Reliability
Generating correct, truthful, and consistent outputs with proper confidence
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
2 - Post-deployment
Risk ID
mit475
Domain lineage
3. Misinformation
3.1 > False or misleading information
Mitigation strategy
1. **Implement Retrieval-Augmented Generation (RAG):** Integrate the LLM with curated, verified external knowledge bases to ground all outputs in factual data, providing source traceability and substantially reducing knowledge-based hallucinations. This is a foundational necessity for high-stakes, knowledge-intensive deployments. 2. **Establish Continuous Uncertainty Quantification and Hallucination Detection:** Incorporate advanced mechanisms, such as contextual grounding checks, confidence scoring, and auxiliary LLM-as-judge evaluations, to continuously monitor output fidelity. This system must enable selective generation (abstention) and automatically escalate low-confidence or non-factual outputs to human-in-the-loop verification processes. 3. **Enforce Structured Reasoning via Prompt Engineering and Data Quality:** Systematically utilize advanced prompt engineering (e.g., Chain-of-Thought, CoV) to mandate explicit, step-by-step reasoning and logical inference before final output generation. Concurrently, maintain rigorous data governance and automated quality checks on all input, training, and fine-tuning datasets to eliminate foundational errors that increase hallucination risk.
ADDITIONAL EVIDENCE
reliability is a major concern because hallucination is currently a well-known problem in LLMs that can hurt the trustworthiness of their outputs significantly, and almost all LLM applications... would be negatively impacted by factually wrong answers. And depending on how high the stake is for the applications, it can cause a wide range of harm, ranging from amusing nonsense to financial or legal disasters