Misuse of AI systems to assist in the creation of weapons
AI systems may be misused to aid in the creation of weapons, such as chemical, biological, radiological, and nuclear (CBRN) weapons, or augment the abilities of existing weapons, such as providing autonomous capabilities to unmanned weapon systems. Current systems do not significantly aid a malicious actor in these tasks, but they do show early signs [117]. This risk can sometimes be mitigated with input and output filtering, but is still susceptible to adversarial techniques (such as jailbreaking or paraphrasing).
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit1196
Domain lineage
4. Malicious Actors & Misuse
4.2 > Cyberattacks, weapon development or use, and mass harm
Mitigation strategy
1. Mandate comprehensive AI Security Compliance programs, particularly for high-risk dual-use systems, through a combination of regulatory oversight, international cooperation, and the establishment of shared best practices to systematically reduce the risk of malicious actor access and capability proliferation. 2. Implement advanced model hardening techniques, including fine-tuning (e.g., RLHF/RLAIF) and adversarial training, to condition AI systems to reliably reject and decline prompts related to the conceptualization or execution of weapon development or high-consequence cyber-activity. 3. Apply robust, multi-layered technical controls such as input/output filtering and validation, alongside central access licensing models, to actively detect, sanitize, and restrict the flow of hazardous information that could facilitate the procurement or weaponization of dangerous substances.