Disinformation and manipulation of public opinion
AI, particularly general- purpose AI, can be maliciously used for disinformation (351), which for the purpose of this report refers to false information that was generated or spread with the deliberate intent to mislead or deceive. General- purpose AI- generated text can be indistinguishable from genuine human- generated material (352, 353), and may already be disseminated at scale on social media (354). In addition, general- purpose AI systems can be used to not only generate text but also fully synthetic or misleadingly altered images, audio, and video content. General- purpose AI tools might be used to persuade and manipulate people, which could have serious implications for political processes. General- purpose AI systems can be used to generate highly persuasive content at scale. This could, for example, be used in a commercial setting for advertising, or during an election campaign to influence public opinion
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit770
Domain lineage
4. Malicious Actors & Misuse
4.1 > Disinformation, surveillance, and influence at scale
Mitigation strategy
1. Establish a mandatory, harmonized AI governance and risk management framework (e.g., building upon the NIST AI RMF) to ensure the responsible design, development, and deployment of general-purpose AI. This framework must prioritize principles of fairness, transparency, accountability, and security to guide ethical practices and provide a standardized basis for international regulatory enforcement against malicious use. 2. Develop and deploy advanced, multilingual AI models for Foreign Information Manipulation and Interference (FIMI) detection and attribution at scale. This must include leveraging network analysis to identify coordinated inauthentic behavior (bot swarms) and content analysis (NLP) to rapidly flag synthetic content, coupled with Explainable AI (XAI) to provide transparent rationales for flagging decisions and enable efficient human-in-the-loop review. 3. Mandate robust technical transparency and provenance mechanisms for all high-risk AI-generated content (text, audio, video), such as unremovable digital watermarking and content credentials (e.g., C2PA), to allow users and platforms to easily distinguish authentic human-generated media from synthetic material.