Influence, overreliance and dependence (emotional dependence)
Humans might become dependent on generative AI tools in ways similar to their emotional dependence on other technologies, such as smartphones or social networks.
ENTITY
1 - Human
INTENT
2 - Unintentional
TIMING
2 - Post-deployment
Risk ID
mit741
Domain lineage
5. Human-Computer Interaction
5.1 > Overreliance and unsafe use
Mitigation strategy
1. Implement Design Interventions to Foster Intentionality and Self-Regulation: Integrate proactive Human-AI Experience (HAX) design elements—such as "cognitive forcing functions," customizable usage limiters, and prompts for digital detox—to disrupt habitual engagement, thereby empowering users to self-monitor and regulate their interaction frequency and duration with the GenAI system. 2. Mandate Transparency and Enhance User AI Literacy: Systematically employ UI messaging, disclaimers, and educational modules to establish realistic user mental models regarding the GenAI system's non-conscious nature and core limitations (e.g., superficial emotional engagement, lack of real-world reciprocity), thereby mitigating anthropomorphism and the conditions for emotional projection and attachment formation. 3. Establish Context-Specific Ethical and Clinical Guardrails for Vulnerable Populations: Develop and enforce robust policies and technical safeguards, particularly for psychologically vulnerable individuals and minors, that clarify the appropriate use-cases for emotional support, including mechanisms for detecting high-risk communications and referring users to qualified human professional mental health resources to prevent the exacerbation of underlying psychological conditions.