Physical Harm and Injury Risks
The integration of general-purpose AI models into embodied systems creates direct physical threats through malicious exploitation of autonomous decision-making capabilities in real-world environments. The risk lies in embodied models' capacity for autonomous action and real-world interaction, and when these capabilities are maliciously exploited they may trigger a series of serious consequences.18
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit1447
Domain lineage
4. Malicious Actors & Misuse
4.2 > Cyberattacks, weapon development or use, and mass harm
Mitigation strategy
1. Enhance Adversarial Robustness and Sensor Redundancy: Implement rigorous adversarial testing and defense strategies to ensure core machine learning models are robust against maliciously crafted inputs, such as in-context backdoors or word injections, and deploy sensor redundancy (e.g., cross-checking multiple sensor feeds) to prevent physical spoofing or manipulation from causing misperception and unsafe autonomous action. 2. Integrate Proactive Anomaly Detection and Emergency Mitigation: Establish real-time, multimodal anomaly and hazard detection systems within the robot's decision-making framework, linking detected hazardous or conflict states to automated, safety-critical mitigation actions such as emergency stops, system disengagement, or immediate path replanning to contain and prevent physical harm. 3. Apply Secure-by-Design Principles and System Isolation: Adopt a secure-by-design development lifecycle that includes comprehensive threat modeling and secure coding practices, and isolate safety-critical control networks (like in-vehicle CAN buses) from less secure components, ensuring strong authentication is required to prevent malicious actors from impersonating critical Electronic Control Units (ECUs) and hijacking autonomous capabilities.