Bias
A systematic error, a tendency to learn consistently wrongly.
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
1 - Pre-deployment
Risk ID
mit634
Domain lineage
1. Discrimination & Toxicity
1.1 > Unfair discrimination and misrepresentation
Mitigation strategy
1. Data-Centric Preprocessing and Curation Implement rigorous data governance policies to ensure the training dataset is statistically diverse and representative across all relevant demographic and protected attributes. This necessitates employing techniques such as reweighting or resampling of data points for underrepresented groups, or generating synthetic data to rectify known distributional imbalances (Source 1, 3, 8). 2. Fairness-Constrained Algorithmic Design Integrate fairness-aware machine learning techniques during the model training phase. This includes modifying the loss function to incorporate explicit fairness constraints (e.g., equalized odds or demographic parity) or employing adversarial debiasing methods to prevent the model from relying on discriminatory latent factors, thereby minimizing bias during the learning process (Source 1, 3, 11). 3. Rigorous Pre-Deployment Bias Impact Assessment Conduct comprehensive bias impact assessments and model audits using quantitative fairness metrics (e.g., statistical parity difference, average odds difference) across different population segments. These evaluations, which should include A/B testing and edge case analysis with diverse data, must serve as a mandatory gate for preventing the deployment of systems that exhibit unacceptable levels of algorithmic discrimination (Source 1, 5, 11).