Purposeful or malicious harm
EAI systems present distinct physical risks due to their embodiment in the physical world. EAI technologies have already been designed and deployed with lethal intent, such as AI-controlled drones [52, 53]. However, fully autonomous military robots, often integrated with bespoke AI architectures [54, 55], are not yet widely used in combat. While highly or fully autonomous warfare is distinctly possible in the future [56], immediate risks arise from commercially available EAI systems, including AI-controlled quadrupeds and autonomous driving assistants.
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit1424
Domain lineage
4. Malicious Actors & Misuse
4.2 > Cyberattacks, weapon development or use, and mass harm
Mitigation strategy
1. **Implement and Enforce Human-in-the-Loop (HITL) Protocols for Lethal Action** Prioritize establishing and mandating robust governance frameworks, such as the Human-in-the-Loop (HITL) or Human-on-the-Loop (HOTL) models, across all deployed EAI systems with potential for lethal or mass harm. This involves setting clear international and national policy limits on the types of targets and scope of engagement to ensure human operators retain ultimate, meaningful judgement and control over the use of force throughout the weapon system's lifecycle, thereby mitigating the risk of intentional or unintentional misuse by malicious actors. 2. **Enhance System Resilience and Assurance through Rigorous V\&V** Require and implement rigorous verification and validation (V\&V) protocols, along with comprehensive safety engineering design reliability, to ensure EAI systems operate as intended and minimize susceptibility to exploitation. This includes cyber-hardening the autonomous platforms via strong encryption, security measures to prevent model and data theft from edge devices, and conducting adversarial testing to proactively address vulnerabilities to hacking, spoofing, and other forms of malicious input or compromise. 3. **Develop and Integrate Advanced Counter-EAI Defense Systems** Invest in and deploy sophisticated, multi-layered Counter-Unmanned Aerial Systems (C-UAS) and counter-EAI technologies that are capable of detecting, tracking, and neutralizing malicious autonomous platforms. Mitigation strategies should emphasize non-kinetic "soft-kill" techniques, such as cyber-based takeover, signal manipulation, and targeted jamming, to safely neutralize the threat and enable forensic recovery, which is critical for non-state actor or non-permissive environment scenarios.