Unauthorized manipulation of AI
AI machines could be hacked and misused, e.g. manipulating an airport luggage screening system to smuggle weapons
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit122
Domain lineage
4. Malicious Actors & Misuse
4.2 > Cyberattacks, weapon development or use, and mass harm
Mitigation strategy
1. Establish Comprehensive AI Governance Frameworks and Security Compliance: Incorporate mandatory AI-specific risk assessments, model oversight, and audit trails into the organizational structure. This foundational step ensures all subsequent technical defenses are guided by a robust policy to address attack surfaces and define appropriate incident response protocols. 2. Deploy AI Firewalls for Input/Output Filtering and Validation: Implement a security layer dedicated to analyzing the intent of all prompts, actions, and outputs of the AI system. This layer must enforce policy guardrails to neutralize malicious inputs (e.g., prompt injections) and validate all outputs to prevent unauthorized actions or data leakage before harm occurs. 3. Implement Identity-Centric Security and Least Privilege Access for AI Agents: Treat AI agents as first-class identities with strictly enforced access controls and lifecycle management. Systematically implement the principle of least privilege, ensuring agents only possess the permissions necessary for their designated function, thereby limiting the potential impact of a successful compromise or unauthorized manipulation.