Extintion
Risk to the existence of humanity.
ENTITY
3 - Other
INTENT
3 - Other
TIMING
2 - Post-deployment
Risk ID
mit638
Domain lineage
7. AI System Safety, Failures, & Limitations
7.2 > AI possessing dangerous capabilities
Mitigation strategy
1. Implement a mandatory regulatory framework for **gating critical AI experiments** (Frontier AI), requiring developers to demonstrate affirmative, verifiable evidence of system safety and alignment (e.g., adversarial robustness, goal fidelity, and resistance to deceptive behavior) before any deployment, analogous to standards in high-risk industries. 2. Establish an **international governance mechanism**—such as a Multinational AGI Consortium—to enforce globally coordinated policies, including a **compute limitation threshold** on the training of the most powerful AI systems, thereby mitigating the competitive "AI race" dynamics that compromise safety. 3. Impose **strict access controls** and **biosecurity protocols** on AI systems possessing dual-use capabilities (e.g., biological or chemical research), including the technical removal of dangerous capabilities from general-purpose models to prevent catastrophic malicious use by rogue actors.