Back to the MIT repository
7. AI System Safety, Failures, & Limitations2 - Post-deployment

Opacity

Stems from the mismatch between mathematical optimization in high-dimensionality characteristic of machine learning and the demands of human-scale reasoning and styles of semantic interpretation.

Source: MIT AI Risk Repositorymit635

ENTITY

2 - AI

INTENT

2 - Unintentional

TIMING

2 - Post-deployment

Risk ID

mit635

Domain lineage

7. AI System Safety, Failures, & Limitations

375 mapped risks

7.4 > Lack of transparency or interpretability

Mitigation strategy

1. Implement advanced Post-Hoc Explainability Techniques. Utilize model-agnostic methods such as SHAP (SHapley Additive exPlanations) or LIME (Local Interpretable Model-agnostic Explanations) to provide local and global explanations for complex, high-dimensionality machine learning decisions. This directly addresses the technical challenge of reconciling mathematical optimization with human-scale reasoning by generating human-comprehensible rationales for specific outputs or overall model behavior. 2. Establish a Robust AI Transparency and Governance Framework. Develop and enforce a formal framework, such as adopting the NIST AI Risk Management Framework or internal Ethical AI principles, that mandates comprehensive documentation, including standardized Model Cards. This ensures procedural transparency by clearly communicating the AI system's intended uses, training data sources, known limitations, and performance metrics to all stakeholders. 3. Integrate Interpretability and Accountability into the AI Lifecycle. Mandate the use of inherently interpretable model architectures where feasible, and, for opaque models, require human-grounded evaluation to ensure explanations are meaningful for end-users and domain experts. Furthermore, implement clear redress mechanisms and oversight to enable contestability of automated decisions, addressing the ethical and societal demands for fairness and accountability associated with semantic interpretation.