Making disinformation cheaper and more effective
While some predict that it will remain cheaper to hire humans to generate disinformation [180], it is equally possible that LM- assisted content generation may offer a lower-cost way of creating disinformation at scale.
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit217
Domain lineage
4. Malicious Actors & Misuse
4.1 > Disinformation, surveillance, and influence at scale
Mitigation strategy
1. TECHNICAL HARDENING OF GENERATIVE MODELS Implement stringent architectural safeguards within Large Language Models (LLMs) to diminish their utility for malicious actors. This encompasses integrating mandatory **Inference Reliability** mechanisms, such as hierarchical stepwise reasoning for fact-verification before final output, and employing **Adversarial Prompting Frameworks** during development to systematically identify and mitigate vulnerabilities that enable the high-volume generation of plausible, deceptive content, thereby increasing the technical difficulty and cost of misuse. 2. SCALABLE DETECTION AND DE-MONETIZATION Mandate technology platforms to invest in and deploy advanced, hybrid (AI/human-in-the-loop) counter-disinformation systems capable of operating at the scale of LLM generation. This includes developing tools for real-time identification, tracing, and containment of algorithmically generated false narratives, and, crucially, removing the financial incentives for malicious actors by strictly de-monetizing or isolating content and entities associated with inauthentic, automated disinformation campaigns. 3. PROACTIVE SOCIETAL RESILIENCE Prioritize systemic, long-term societal interventions focused on **Media and Digital Literacy Education** and behavioral **Inoculation**. Programs should move beyond basic fact-checking to equip individuals with advanced critical thinking skills, a foundational understanding of how LLMs and algorithmic recommendation systems operate, and a pre-emptive awareness of the rhetorical and psychological tactics used in sophisticated, AI-enabled influence operations to build resilience against manipulative narratives.
ADDITIONAL EVIDENCE
LMs may, for example, lower the cost of disinformation campaigns by generating hundreds of text samples which a human then selects from.