Risks from AI systems (Risks of exploitation through defects and backdoors)
The standardized API, feature libraries, toolkits used in the design, training, and verification stages of AI algorithms and models, development interfaces, and execution platforms may contain logical flaws and vulnerabilities. These weaknesses can be exploited, and in some cases, backdoors can be intentionally embedded, posing significant risks of being triggered and used for attacks.
ENTITY
1 - Human
INTENT
3 - Other
TIMING
3 - Other
Risk ID
mit691
Domain lineage
2. Privacy & Security
2.2 > AI system security vulnerabilities and attacks
Mitigation strategy
1. Mandate the cryptographic signing and rigorous integrity verification of all model artifacts (weights, configurations) and third-party dependencies (libraries, APIs) across the entire AI supply chain to prevent unauthorized modification or the introduction of malicious code. Concurrently, establish immutable data provenance and lineage tracking to safeguard against data poisoning. 2. Enforce the principle of 'Secure by Design' on all development interfaces and execution platforms by deploying an API Gateway for centralized access control, robust input/output validation, and threat mitigation. Implement Zero Trust access control with strict least-privilege principles for all model and data interactions. 3. Institute mandatory, continuous behavioral monitoring of the deployed AI system and its APIs to detect anomalous activities indicative of backdoor activation or functional exploitation in real-time. This should be supplemented by adversarial red teaming and security testing tailored to expose AI-specific vulnerabilities.