Back to the MIT repository
1. Discrimination & Toxicity1 - Pre-deployment

Decision bias

Decision bias occurs when one group is unfairly advantaged over another due to decisions of the model. This might be caused by biases in the data and also amplified as a result of the model’s training.

Source: MIT AI Risk Repositorymit1317

ENTITY

2 - AI

INTENT

2 - Unintentional

TIMING

1 - Pre-deployment

Risk ID

mit1317

Domain lineage

1. Discrimination & Toxicity

156 mapped risks

1.1 > Unfair discrimination and misrepresentation

Mitigation strategy

1. Systematic Data Curation and Pre-processing Implement rigorous data governance protocols to ensure training datasets are maximally representative and diverse across all protected attributes (e.g., race, gender, age). This necessitates the application of pre-processing techniques such as reweighting underrepresented samples, targeted oversampling of minority groups, or feature perturbation to achieve statistical parity prior to model ingestion. 2. Algorithmic Integration of Fairness Constraints Integrate fairness-aware optimization methodologies during model training. This includes employing techniques like fair regularization, which introduces a penalty term into the loss function to minimize the statistical dependence between sensitive features and model predictions, or utilizing constrained optimization methods to enforce formal fairness criteria (e.g., demographic parity or equalized odds). 3. Establish Robust Bias Governance and Continuous Monitoring Mandate the establishment of an ongoing governance framework, including regular, quantitative bias audits (evaluating performance across demographic subgroups) and the integration of Explainable AI (XAI) to ensure transparency in decision pathways. Furthermore, continuous post-deployment monitoring is required to detect and mitigate bias drift, ensuring long-term adherence to equity standards.