Other Critical Infrastructure Control Systems
General-purpose AI deployed in power grid management, water treatment facilities, telecommunications networks, or transportation coordination systems could misinterpret operational data, fail to anticipate cascading failure modes, or make control decisions that destabilize interconnected infrastructure networks. Infrastructure failures could result in widespread blackouts, contaminated water supplies, communications breakdowns, and the collapse of essential services supporting hundreds of thousands of people.
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
2 - Post-deployment
Risk ID
mit1455
Domain lineage
7. AI System Safety, Failures, & Limitations
7.3 > Lack of capability or robustness
Mitigation strategy
1. Implement layered cyber-physical segmentation and mandatory human-in-the-loop (HITL) overrides for all AI control systems. Isolate Operational Technology (OT) networks from the general IT environment and ensure the system's architecture includes clearly defined, non-AI-dependent manual control pathways (kill switches) that can immediately override or halt destabilizing autonomous AI decisions. 2. Deploy predictive modeling systems, such as Digital Twins, and continuous behavioral analytics for the AI itself. Utilize high-fidelity emulation environments (digital twins) of the critical infrastructure network to anticipate and model cascading failure modes that the AI may fail to recognize. Simultaneously, monitor the AI's real-time outputs and performance against established safety thresholds to detect subtle anomalies or drift in decision-making that may precede a system failure. 3. Establish a robust AI Risk Management Framework (AI RMF) throughout the entire system lifecycle, integrated with existing cybersecurity protocols. Adopt a framework (e.g., NIST AI RMF) to mandate "secure-by-design" principles from the initial phase. This includes comprehensive adversarial testing (AI red-teaming) and post-deployment audits to continuously validate the AI's robustness against both intentional threats and unintentional failures, ensuring compliance and long-term trustworthiness.