Warfare and Physical Harm
The use of AI in warfare is highly alarming and may pose dangers to human safety (Hendrycks et al., 2023). Autonomous drone warfare is being aggressively pursued as a tactic in the current war in Ukraine (Meaker, 2023), and may already have been used on human targets (Hambling, 2023). The use of AI- based facial recognition has been documented in the targeting of Palestinians in Gaza (International, 2023). LLMs have already been productized in limited ways for the purposes of warfare planning (Tarantola, 2023). Furthermore, active research is being carried out to develop multimodal-LLMs that can act as ‘brains’ for general-purpose robots (Ahn et al., 2022; 2024). Due to the ‘general-purpose’ nature of such advances, it will likely be cost-effective and practical to adapt them for creating more advanced autonomous weapons
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit1492
Domain lineage
4. Malicious Actors & Misuse
4.2 > Cyberattacks, weapon development or use, and mass harm
Mitigation strategy
1. Mandate the establishment of comprehensive, internationally aligned ethical and legal frameworks that enforce meaningful human control ("human-in-the-loop" or "human-on-the-loop") over the deployment of lethal autonomous weapon systems, thereby prioritizing human judgment and accountability in the use of force. 2. Implement rigorous security and resilience protocols, including advanced cybersecurity, continuous red-teaming, and adversarial testing, to detect and mitigate vulnerabilities such as data poisoning or adversarial attacks that could lead to technical failures or inadvertent escalation. 3. Engage in proactive international diplomacy to establish binding treaties, shared technical standards, and confidence-building measures that limit the proliferation of offensive AI-enabled capabilities and promote collaborative mechanisms for managing catastrophic AI failure risks.