Human–AI interaction
Several participants mentioned how AI systems could influence human agency and decision-making. They emphasized the need of striking a balance between using the benefits of AI and protecting human autonomy and control. The increasing integration of AI systems into various aspects of our lives, which can have a significant impact on human agency and decision-making, has raised ethical concerns about AI and human–AI interaction. As AI systems advance, they will be able to influence, if not completely replace, IJOES human decision-making in some fields, prompting concerns about the loss of human autonomy and control. Participants in the study emphasize the need of establishing a balance between using the benefits of AI and maintaining human autonomy and control to ensure that people retain agency and are not overly reliant on AI systems. This balance is essential to prevent possible negative consequences such as over-reliance on AI, diminishing human skills and knowledge and a loss of personal accountability
ENTITY
2 - AI
INTENT
3 - Other
TIMING
2 - Post-deployment
Risk ID
mit590
Domain lineage
5. Human-Computer Interaction
5.2 > Loss of human agency and autonomy
Mitigation strategy
1. Mandate and Implement Human Oversight Frameworks Establish robust, tiered human oversight models (e.g., Human-in-the-Loop, Human-on-the-Loop) to ensure that human professionals retain final authority, accountability, and the capacity for intervention, especially in high-stakes decision-making scenarios, thereby structurally preserving human agency. 2. Enforce Explainability and Transparency Requirements Integrate Explainable AI (XAI) mechanisms and transparency protocols that provide users with clear, contextualized, and understandable rationales for AI outputs. This empowers human users to critically evaluate, trust, and appropriately calibrate their reliance on the system, which is essential for maintaining control and informed decision-making. 3. Prioritize Augmentation in System Design Adopt a human-centered design principle focused on *augmentation* over replacement, configuring AI systems to serve as decision-support tools that enhance human capabilities, judgment, and critical thinking. This design choice actively mitigates the risk of deskilling and long-term erosion of professional autonomy.