Back to the MIT repository
4. Malicious Actors & Misuse2 - Post-deployment

Technology-facilitated violence

Technology-facilitated violence occurs when algorithmic features enable use of a system for harassment and violence [2, 16, 44, 80, 108], including creation of non-consensual sexual imagery in generative AI... other facets of technology-facilitated violence, include doxxing [79], trolling [14], cyberstalking [14], cyberbullying [14, 98, 204], monitoring and control [44], and online harassment and intimidation [98, 192, 199, 226], under the broader banner of online toxicity

Source: MIT AI Risk Repositorymit149

ENTITY

1 - Human

INTENT

1 - Intentional

TIMING

2 - Post-deployment

Risk ID

mit149

Domain lineage

4. Malicious Actors & Misuse

223 mapped risks

4.3 > Fraud, scams, and targeted manipulation

Mitigation strategy

1. **Implement Safety-by-Design Protocols for Access Control** A mandatory redesign of user authentication and account management systems to prevent unauthorized control. This requires enforcing single-user primary account ownership, ensuring any shared access or secondary user credentialing is explicitly governed by the primary owner, and integrating a robust, auditable mechanism for immediate remote deactivation of all connected devices upon the owner's request (Risk Reduction/Avoidance). 2. **Establish a Dedicated, Gender-Responsive Incident Response Mechanism** Develop a specialized internal protocol for investigating and responding to reports of technology-facilitated violence (TFV) and misuse. This includes: training customer support staff to recognize signs of controlling behavior; creating a rapid-response team to preserve forensic evidence (e.g., remote access logs, location data); and establishing clear internal escalation paths for prompt cooperation with law enforcement and victim support services (Accountability/Response). 3. **Integrate Mandatory Digital Literacy and Abuse Awareness** Deploy educational modules and policy disclosures at the point of product enrollment and within the application interface that explicitly define technology-facilitated abuse, highlight common misuse vectors (e.g., remote monitoring, password sharing), and provide prominent, accessible in-app resources for survivor support and safety guidance (Empowerment/Awareness).

ADDITIONAL EVIDENCE

[She] broke up with [him] due to his controlling behavior. After the break-up, he began to appear where she was. . . One day, while driving her [car], the air conditioner turned off. . . .After a few failed attempts, she figured the unit was broken. . . After a call with the [car's] customer support, she discovered a second person using the [car] app to connect