Back to the MIT repository
1. Discrimination & Toxicity2 - Post-deployment

Discrimination

When AI is not carefully designed, it can discriminate against certain groups.

Source: MIT AI Risk Repositorymit87

ENTITY

2 - AI

INTENT

2 - Unintentional

TIMING

2 - Post-deployment

Risk ID

mit87

Domain lineage

1. Discrimination & Toxicity

156 mapped risks

1.1 > Unfair discrimination and misrepresentation

Mitigation strategy

1. Implement rigorous data auditing and preprocessing protocols to ensure the training and testing datasets are statistically representative of the target population, are balanced across sensitive groups, and are free from historical or prejudice biases, utilizing techniques such as oversampling, undersampling, or synthetic data generation for underrepresented classes. 2. Establish comprehensive AI governance frameworks that mandate continuous, end-to-end algorithmic audits and impact assessments, utilizing quantitative fairness metrics (e.g., Demographic Parity, Equalized Odds) across diverse subgroups before and after deployment to proactively detect and mitigate discriminatory outcomes. 3. Proactively integrate fairness-by-design principles into the model development lifecycle, employing specialized fairness-aware algorithms, constraint-based optimization, or techniques such as reweighing training instances to minimize the impact of protected characteristics on the system's output while maintaining predictive accuracy. 4. Enhance human oversight through 'Human-in-the-Loop' mechanisms and clear accountability structures, ensuring that human reviewers with diverse perspectives can validate or override AI-driven decisions in high-stakes applications and that the system's logic is made transparent via Explainable AI (XAI) methods.