Information Manipulation
generative AI tools can and will be used to propagate content that is false, misleading, biased, inflammatory, or dangerous. As generative AI tools grow more sophisticated, it will be quicker, cheaper, and easier to produce this content—and existing harmful content can serve as the foundation to produce more
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit510
Domain lineage
4. Malicious Actors & Misuse
4.1 > Disinformation, surveillance, and influence at scale
Mitigation strategy
1. Mandate the implementation of **cryptographically secure content provenance and watermarking standards** (e.g., Content Credentials/C2PA) at the point of content generation to ensure traceability and signal whether media is synthetic. Concurrently, invest in the development and deployment of **advanced, explainable AI (XAI) detection systems** to rapidly verify content authenticity and identify manipulated narratives across platforms. 2. Establish and enforce robust **platform governance and accountability frameworks** that require: a) independent bias audits and diverse dataset requirements for AI training, b) increasing transparency and audit access for recommender algorithms that may amplify misleading content, and c) the adoption of cybersecurity best practices, including threat modeling and red-teaming, into the early stages of generative AI product design. 3. Develop and disseminate comprehensive, sustained **digital literacy and cognitive resilience programs** across educational and community sectors to equip individuals with the critical thinking skills and knowledge necessary to discern and evaluate exogenous cues (e.g., source reliability, provenance signals) for AI-enabled disinformation and to actively report suspicious content.