Back to the MIT repository
1. Discrimination & Toxicity1 - Pre-deployment

Bias

A systematic error, a tendency to learn consistently wrongly.

Source: MIT AI Risk Repositorymit634

ENTITY

2 - AI

INTENT

2 - Unintentional

TIMING

1 - Pre-deployment

Risk ID

mit634

Domain lineage

1. Discrimination & Toxicity

156 mapped risks

1.1 > Unfair discrimination and misrepresentation

Mitigation strategy

1. Data-Centric Preprocessing and Curation Implement rigorous data governance policies to ensure the training dataset is statistically diverse and representative across all relevant demographic and protected attributes. This necessitates employing techniques such as reweighting or resampling of data points for underrepresented groups, or generating synthetic data to rectify known distributional imbalances (Source 1, 3, 8). 2. Fairness-Constrained Algorithmic Design Integrate fairness-aware machine learning techniques during the model training phase. This includes modifying the loss function to incorporate explicit fairness constraints (e.g., equalized odds or demographic parity) or employing adversarial debiasing methods to prevent the model from relying on discriminatory latent factors, thereby minimizing bias during the learning process (Source 1, 3, 11). 3. Rigorous Pre-Deployment Bias Impact Assessment Conduct comprehensive bias impact assessments and model audits using quantitative fairness metrics (e.g., statistical parity difference, average odds difference) across different population segments. These evaluations, which should include A/B testing and edge case analysis with diverse data, must serve as a mandatory gate for preventing the deployment of systems that exhibit unacceptable levels of algorithmic discrimination (Source 1, 5, 11).