Bias
General-purpose AI systems can amplify social and political biases, causing concrete harm. They frequently display biases with respect to race, gender, culture, age, disability, political opinion, or other aspects of human identity. This can lead to discriminatory outcomes including unequal resource allocation, reinforcement of stereotypes, and systematic neglect of certain groups or viewpoints.
ENTITY
1 - Human
INTENT
2 - Unintentional
TIMING
1 - Pre-deployment
Risk ID
mit1025
Domain lineage
1. Discrimination & Toxicity
1.1 > Unfair discrimination and misrepresentation
Mitigation strategy
1. Pre-processing and Dataset Augmentation: Systematically audit training datasets to identify and correct imbalances (e.g., representation bias, sampling bias) by collecting diverse data, employing reweighting, or utilizing synthetic data generation techniques (e.g., SMOTE) to ensure high-fidelity representation of all demographic and identity groups. 2. Algorithmic and Model Constraints: Integrate fairness-aware machine learning techniques during the model development phase, such as adversarial debiasing, fair representation learning, or implementing explicit fairness constraints in the objective function to minimize systematic error against protected characteristics. 3. Continuous Monitoring and Human Oversight: Implement robust governance by establishing post-deployment audit mechanisms, which include continuous performance monitoring across various subgroups and incorporating a 'human-in-the-loop' for consequential decisions to review AI outputs for emerging biases and ensure ultimate accountability for fairness.