Opacity
Stems from the mismatch between mathematical optimization in high-dimensionality characteristic of machine learning and the demands of human-scale reasoning and styles of semantic interpretation.
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
2 - Post-deployment
Risk ID
mit635
Domain lineage
7. AI System Safety, Failures, & Limitations
7.4 > Lack of transparency or interpretability
Mitigation strategy
1. Implement advanced Post-Hoc Explainability Techniques. Utilize model-agnostic methods such as SHAP (SHapley Additive exPlanations) or LIME (Local Interpretable Model-agnostic Explanations) to provide local and global explanations for complex, high-dimensionality machine learning decisions. This directly addresses the technical challenge of reconciling mathematical optimization with human-scale reasoning by generating human-comprehensible rationales for specific outputs or overall model behavior. 2. Establish a Robust AI Transparency and Governance Framework. Develop and enforce a formal framework, such as adopting the NIST AI Risk Management Framework or internal Ethical AI principles, that mandates comprehensive documentation, including standardized Model Cards. This ensures procedural transparency by clearly communicating the AI system's intended uses, training data sources, known limitations, and performance metrics to all stakeholders. 3. Integrate Interpretability and Accountability into the AI Lifecycle. Mandate the use of inherently interpretable model architectures where feasible, and, for opaque models, require human-grounded evaluation to ensure explanations are meaningful for end-users and domain experts. Furthermore, implement clear redress mechanisms and oversight to enable contestability of automated decisions, addressing the ethical and societal demands for fairness and accountability associated with semantic interpretation.