Quality of training data
The quality of training data is another challenge faced by generative AI. The quality of generative AI models largely depends on the quality of the training data (Dwivedi et al., 2023; Su & Yang, 2023). Any factual errors, unbalanced information sources, or biases embedded in the training data may be reflected in the output of the model. Generative AI models, such as ChatGPT or Stable Diffusion which is a text-to-image model, often require large amounts of training data (Gozalo-Brizuela & Garrido-Merchan, 2023). It is important to not only have high-quality training datasets but also have complete and balanced datasets.
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
1 - Pre-deployment
Risk ID
mit542
Domain lineage
7. AI System Safety, Failures, & Limitations
7.3 > Lack of capability or robustness
Mitigation strategy
1. Perform Comprehensive Data Provenance and Quality Audits: Rigorously assess all training data for factual accuracy, statistical imbalances, and inherent biases (e.g., demographic, cultural). Implement data sanitization and cleansing techniques, such as statistical outlier detection and the application of differential privacy, to ensure the dataset is high-quality, complete, and free from contamination prior to model ingestion. 2. Integrate Diversity and Fairness Metrics into Data Curation: Establish guidelines and conduct audits to ensure the workforce involved in data annotation and labeling is diverse and representative of the intended user population. This process actively mitigates the introduction of subtle, human-generated biases during the preparation phase that could be amplified by the model. 3. Establish Continuous Bias and Hallucination Monitoring: Deploy real-time system monitoring, post-deployment fairness audits, and explainable AI (XAI) tools to track model performance, identify unusual behavior, and detect outputs that reflect training data flaws, such as inaccurate or biased generation. Implement human-in-the-loop validation checkpoints for high-risk outputs to provide critical oversight.