Factual Errors Injected by External Tools
External tools typically incorporate additional knowledge into the input prompts [122], [178]–[184]. The additional knowledge often originates from public resources such as Web APIs and search engines. As the reliability of external tools is not always ensured, the content returned by external tools may include factual errors, consequently amplifying the hallucination issue.
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
2 - Post-deployment
Risk ID
mit29
Domain lineage
2. Privacy & Security
2.2 > AI system security vulnerabilities and attacks
Mitigation strategy
1. Deploy Retrieval-Augmented Generation (RAG) Architectures with Curated Sources: Integrate the LLM with explicitly verified, high-quality knowledge bases rather than general Web APIs. This shifts the reliance for factual grounding from potentially unreliable public resources to a factually constrained, audited source set, thereby reducing the probability of injecting external errors. 2. Institute a Multi-Stage Factual Validation and Correction Pipeline: Implement automated post-generation checks by utilizing specialized detection mechanisms, such as Fact-Checking Algorithms or LLM-based zero-shot detectors. This pipeline must systematically cross-reference all claims derived from external tools against trusted external evidence, flagging or subjecting low-confidence outputs to an Uncertainty Scoring mechanism that enables polite deferral of the answer. 3. Enforce Strict Provenance Tracing and Expert Human Oversight: Mandate the logging of specific data sources and external tool calls for every factually grounded claim to ensure full traceability and accountability. For high-stakes applications, this requires the integration of a mandatory human-in-the-loop review process where a domain expert validates the factual accuracy of any tool-derived content before the final response is delivered.