Cyber offence
General- purpose AI systems could uplift the cyber expertise of individuals, making it easier for malicious users to conduct effective cyber- attacks, as well as providing a tool that can be used in cyber defence. General- purpose AI systems can be used to automate and scale some types of cyber operations, such as social engineering attacks.
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit771
Domain lineage
4. Malicious Actors & Misuse
4.2 > Cyberattacks, weapon development or use, and mass harm
Mitigation strategy
1. Implement stringent technical safeguards, including input validation, sanitization, and runtime behavioral monitoring, on general-purpose AI systems to prevent prompt injection and jailbreaking attempts that force the generation of malicious code or instructions. 2. Foster adaptive defense capabilities and intelligence-led security strategies, including mass training of cybersecurity professionals in defensive AI tools, to effectively counter the automation and scaling of sophisticated, AI-driven attacks. 3. Mandate proactive adversarial training and red-teaming exercises to stress-test AI models against novel input manipulation and automated exploitation techniques, ensuring system robustness before deployment.