Lack of explainability
The explainability of AI systems based on so-called black-box models is often limited. This opaqueness of AI systems can prevent developers from detecting shortcomings in the data or the model itself and decrease the performance and safety levels of the AI system.
ENTITY
3 - Other
INTENT
3 - Other
TIMING
3 - Other
Risk ID
mit1010
Domain lineage
7. AI System Safety, Failures, & Limitations
7.4 > Lack of transparency or interpretability
Mitigation strategy
1. Mandate and Integrate Explainable AI (XAI) Methodologies: Systematically adopt and integrate Model-Agnostic Explanations (e.g., LIME, SHAP values) and/or Model-Specific Interpretability Constraints to generate both local (per-decision) and global (overall behavior) explanations, thereby demystifying 'black-box' decision-making for debugging and validation. 2. Establish a Robust AI Governance and Auditing Framework: Implement comprehensive governance requiring the continuous logging of all decision processes (audit trails), standardized documentation (e.g., Model Cards/Datasheets), and regular independent audits to ensure systematic transparency, accountability, and regulatory compliance throughout the AI lifecycle. 3. Implement Tiered Transparency and Human Validation Loops: Develop interfaces that provide tiered explanations, offering non-technical reasoning for end-users to foster trust while supplying technical feature importance and diagnostic data to expert operators. Concurrently, institute human-in-the-loop validation for high-stakes decisions to ensure contestability and mitigate unforeseen consequences.