Harmful Bias or Homogenization
Amplification and exacerbation of historical, societal, and systemic biases; performance disparities8 between sub-groups or languages, possibly due to non-representative training data, that result in discrimination, amplification of biases, or incorrect presumptions about performance; undesired homogeneity that skews system or model outputs, which may be erroneous, lead to ill-founded decision-making, or amplify harmful biases.
ENTITY
3 - Other
INTENT
2 - Unintentional
TIMING
3 - Other
Risk ID
mit761
Domain lineage
1. Discrimination & Toxicity
1.1 > Unfair discrimination and misrepresentation
Mitigation strategy
1. Systematically apply **Pre-processing Data Bias Mitigation Techniques**. This requires rigorous auditing of training and fine-tuning datasets for demographic underrepresentation or overrepresentation based on sensitive attributes (e.g., race, gender, age, geographic location). Techniques such as data reweighing or resampling must be implemented upstream to ensure the data is non-homogenous and equitably representative of the intended population, thereby directly addressing the primary source of historical and societal bias amplification. 2. Implement **Algorithmic Fairness Controls and Continuous Model Auditing**. This involves integrating in-processing techniques to modify learning algorithms for fairness assurance, alongside continuous cross-validation of hyperparameters (e.g., model capacity and regularization) to actively reduce the potential for machine learning mechanics to amplify pre-existing biases, ensuring consistent performance and reduced disparity across all identified sub-groups. 3. Establish a **Diverse and Culturally Aware AI Governance Structure**. This mandates ensuring diversity within AI development and review teams (including clinical experts, data scientists, and members of underrepresented populations) starting from the conception phase. Furthermore, all personnel must receive mandatory training to recognize and actively mitigate their own implicit biases and cultural assumptions to prevent these unexamined prejudices from influencing the design of the research questions, algorithms, and system evaluation metrics.