Technical and operational risks
To date, technical limitations and vulnerabilities are present in most generative AI models in various contexts. Consequently, malicious users find it easier to breach an AI system’s safety and ethical guardrails to execute harmful actions.223 Normal user behavior—actions within an AI system’s intended use—can also lead to harmful outcomes. Whether these harmful outcomes result from normal or malicious use, they stem from the inherent limitations of current technology, which future advancements may overcome. This section examines the technical vulnerabilities that can affect AI models, the tendency of generative AI models to generate inaccurate information, and the inherent opacity of these AI systems, which complicates the understanding and mitigation of these difficulties.
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
3 - Other
Risk ID
mit722
Domain lineage
7. AI System Safety, Failures, & Limitations
7.3 > Lack of capability or robustness
Mitigation strategy
1. Implement robust adversarial training and rigorous vulnerability testing throughout the model lifecycle to enhance system robustness against adversarial attacks and ensure reliable operation under varied conditions. 2. Establish continuous, real-time monitoring of model inputs, outputs, and performance metrics, integrating a robust vulnerability management process for the prompt detection and remediation of security flaws and behavioral anomalies. 3. Enforce secure model development and deployment practices, including secure coding standards, cryptographic controls for data integrity, and secure deployment pipelines to address inherent technical limitations and prevent the introduction of new vulnerabilities.