Risks from models and algorithms (Risks of unreliable output)
Generative AI can cause hallucinations, meaning that an AI model generates untruthful or unreasonable content but presents it as if it were a fact, leading to biased and misleading information.
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
2 - Post-deployment
Risk ID
mit685
Domain lineage
3. Misinformation
3.1 > False or misleading information
Mitigation strategy
1. Architecturally reinforce knowledge grounding through the implementation of Retrieval-Augmented Generation (RAG) systems, enabling the LLM to access and cross-verify information against external, verified knowledge bases in real-time. This must be paired with rigorous optimization of training datasets, emphasizing quality, diversity, and the elimination of outdated or biased information. 2. Implement advanced prompt engineering techniques to govern output generation, specifically utilizing Chain-of-Thought verification to compel step-by-step reasoning and incorporating Explicit Uncertainty Instructions that mandate the model to express confidence levels or abstain from providing an answer when factual grounding is low. 3. Establish a comprehensive post-deployment governance framework that includes mandatory human oversight and validation for all high-stakes or fact-based generative outputs. This framework must also integrate continuous evaluation loops and automated tools, such as fact-checking APIs, to rigorously test and correct the system's performance and output consistency over time.