Malicious use and abuse (sexually explicit content generation)
An illustrative case of malicious use of generative AI models is the creation of explicit sexual images. Generative AI technologies can be employed to produce deepfakes—for instance, superimposing a celebrity’s face onto the body of a performer in an adult film.
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit732
Domain lineage
4. Malicious Actors & Misuse
4.3 > Fraud, scams, and targeted manipulation
Mitigation strategy
1. Implement robust technical guardrails and safety-by-design principles within generative models to intrinsically preclude the creation of non-consensual sexually explicit content, including AIG-CSAM. This process must include continuous adversarial testing (red teaming) to identify and mitigate emergent bypass vectors. 2. Mandate and deploy digital content provenance mechanisms, such as cryptographic watermarking and metadata, for all AI-generated media. This ensures content traceability, which is critical for rapid, high-confidence detection and flagging by AI-powered tools on distribution platforms. 3. Establish and enforce clear legal and regulatory frameworks that impose accountability on model developers and platform providers for negligence in preventing the creation and rapid, non-consensual dissemination of deepfakes, compelling swift takedown and content removal protocols upon victim or law enforcement notification.