Unfair capability distribution
Performing worse for some groups than others in a way that harms the worse-off group
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
2 - Post-deployment
Risk ID
mit260
Domain lineage
1. Discrimination & Toxicity
1.3 > Unequal performance across groups
Mitigation strategy
1. Mandate Comprehensive Data Governance and Representational Auditing: Implement systematic procedures to audit training, validation, and testing datasets for representational gaps and embedded historical biases. This pre-processing is essential to ensure data is relevant and representative of all target sub-populations, thereby reducing the risk of differential performance derived from biased inputs. 2. Establish Disaggregated Performance Evaluation Metrics: Conduct rigorous and statistical fairness testing across identified protected and vulnerable subgroups. This must include measuring and mitigating disparities in critical error rates (e.g., false positives/negatives) to ensure equitable capability distribution and avoid outcomes that disproportionately harm the worse-off group. 3. Deploy Continuous Algorithmic Fairness Monitoring: Institute a real-time post-market monitoring and vulnerability management process to track system performance disaggregated by group. This continuous oversight is necessary to detect and rapidly remediate any emergent discriminatory effects or degradation in capability for specific groups over time, ensuring sustained trustworthy operation.
ADDITIONAL EVIDENCE
Example: Generating a lower-quality output when given a prompt in a non-English language (Dave, 2023)*