Hallucinations
The inclusion of erroneous information in the outputs from AI systems is not new. Some have cautioned against the introduction of false structures in X-ray or MRI images, and others have warned about made-up academic references. However, as ChatGPT-type tools become available to the general population, the scale of the problem may increase dramatically. Furthermore, it is compounded by the fact that these conversational AIs present true and false information with the same apparent “confidence” instead of declining to answer when they cannot ensure correctness. With less knowledgeable people, this can lead to the heightening of misinformation and potentially dangerous situations. Some have already led to court cases.'
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
2 - Post-deployment
Risk ID
mit58
Domain lineage
3. Misinformation
3.1 > False or misleading information
Mitigation strategy
1. Establish a Retrieval-Augmented Generation (RAG) architecture to anchor the Large Language Model's (LLM) outputs to current, verified external knowledge bases, thereby structurally minimizing reliance on internal, potentially outdated or fabricated, parametric knowledge. 2. Mandate the use of advanced prompt engineering methodologies, including Chain-of-Thought (CoT) prompting to enforce structured logical reasoning, and explicit uncertainty instructions requiring the model to articulate its confidence level or refuse to respond in the absence of verifiable facts. 3. Implement a continuous, post-generation verification framework utilizing automated fact-checking mechanisms or Human-in-the-Loop (HITL) guardrails for critical outputs to cross-validate generated claims against trusted sources prior to dissemination, ensuring that erroneous information is intercepted before impacting end-users.