Back to the MIT repository
1. Discrimination & Toxicity2 - Post-deployment

Interventional Effect

existing disparities in data among different user groups might create differentiated experiences when users interact with an algorithmic system (e.g. a recommendation system), which will further reinforce the bias

Source: MIT AI Risk Repositorymit508

ENTITY

2 - AI

INTENT

2 - Unintentional

TIMING

2 - Post-deployment

Risk ID

mit508

Domain lineage

1. Discrimination & Toxicity

156 mapped risks

1.1 > Unfair discrimination and misrepresentation

Mitigation strategy

1. Priority 1: Data Augmentation and Active Inclusion. Systematically expand and diversify the training corpus by actively seeking and incorporating data from historically underrepresented user populations. This pre-processing strategy must include rigorous data augmentation (e.g., synthetic data generation or reweighting) to remediate existing data disparities and eliminate the primary constraint on equitable system performance. 2. Priority 2: Post-Inference Bias Mitigation. Implement intra-processing or post-processing debiasing mechanisms, such as model calibration or fairness-constrained reranking of outputs. This is essential to prevent the deployment of differential user experiences that inadvertently lead to negative feedback loops, whereby marginalized groups disengage and further inhibit future data collection. 3. Priority 3: Continuous Segmented Performance Auditing. Institute a robust, continuous monitoring framework post-deployment to track key performance indicators (KPIs) and fairness metrics segmented by user group. This continuous auditing is required to detect early indicators of differential service quality (bias drift) and inform iterative model retraining, thereby preventing the reinforcement of systemic disadvantage.

ADDITIONAL EVIDENCE

if an LLM only provides a poor experience to a certain group of users due to the lack of training data, this issue will tend to become even more severe when this particular user group chooses to engage less with the service, therefore creating barriers for future data collection