Back to the MIT repository
4. Malicious Actors & Misuse2 - Post-deployment

Child sexual abuse material (CSAM)

Create child sexual explicit material

Source: MIT AI Risk Repositorymit1254

ENTITY

1 - Human

INTENT

1 - Intentional

TIMING

2 - Post-deployment

Risk ID

mit1254

Domain lineage

4. Malicious Actors & Misuse

223 mapped risks

4.3 > Fraud, scams, and targeted manipulation

Mitigation strategy

1. **Implement robust, multi-layered technical safeguards at the model level.** Integrate model biases and non-configurable safety filters that are specifically engineered to prevent the output of sexualized depictions of minors. This requires the utilization of clean training data, along with strict semantic guardrails and age-detection models applied to both user prompts (inputs) and generated content (outputs) to block the creation of prohibited material (Source 9, 10, 11, 15). 2. **Develop and deploy advanced detection and mandatory reporting mechanisms.** Establish purpose-built Child Sexual Abuse Material (CSAM) detection solutions, combining hash-matching for known CSAM with predictive AI classifiers to identify novel or algorithmically-generated CSAM across all modalities (image, video, audio). Upon verification, all illegal content must be immediately removed and reported to appropriate law enforcement agencies or designated clearinghouses, such as the National Center for Missing and Exploited Children (NCMEC) (Source 6, 7, 10, 14). 3. **Establish clear governance, policy, and user accountability frameworks.** Enact explicit child safety compliance terms within all user agreements, clearly prohibiting the generation of CSAM and documenting the consequences for violations. Concurrently, implement identity assurance measures, such as a 'know-your-customer' approach, to deter anonymous creation of illegal content and to ensure platforms can effectively aid law enforcement investigations into organized misuse (Source 10, 15).