Back to the MIT repository
4. Malicious Actors & Misuse2 - Post-deployment

Ability to enhance and modify pathogens

AI can be used to enhance pathogens, making them more lethal or resistant to treatments.

Source: MIT AI Risk Repositorymit1046

ENTITY

1 - Human

INTENT

1 - Intentional

TIMING

2 - Post-deployment

Risk ID

mit1046

Domain lineage

4. Malicious Actors & Misuse

223 mapped risks

4.2 > Cyberattacks, weapon development or use, and mass harm

Mitigation strategy

1. Layered AI Model Safeguards: Implement stringent access controls and technical safety measures on advanced Biological Design Tools (BDTs) and general-purpose Large Language Models (LLMs). This includes rigorous pre-deployment evaluations to measure and restrict capabilities that could facilitate pathogen enhancement, employing model distillation to limit dual-use knowledge, and applying robust content filters to prevent the generation of harmful biological protocols. 2. Mandatory Biosecurity Screening for Gene Synthesis: Enforce comprehensive and globally coordinated "Know Your Customer" protocols and nucleic acid sequence screening across all gene synthesis providers. This is a critical physical chokepoint that must be continuously updated to detect and flag genetic sequences corresponding to known or AI-generated/enhanced pathogens, thereby preventing the physical creation of the harmful agent. 3. Dynamic Risk Assessment and Adaptive Governance: Establish a continuous, outcome-based risk assessment framework, prioritizing epidemic and pandemic-level risks, to monitor the rapidly evolving capabilities of AI in biotechnology. This assessment must inform agile, multi-layered governance that includes regulations on dual-use biological data sharing and the rigorous oversight of gain-of-function research enabled by AI tools.