Misuse Risks
However, even if a model is entirely trustworthy and reliable, Misuse or Systemic Risks remain. General purpose AI models may present significant risks to society if this technology is misused by malicious actors to produce harmful outcomes. Misuse Risks span across Cyber Crime, Biosecurity Threats and Politically Motivated Misuse.
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit840
Domain lineage
4. Malicious Actors & Misuse
4.0 > Malicious use
Mitigation strategy
1. Mandate rigorous, continuous Adversarial Testing and Capability Measurement Routinely engage independent experts (red teams) to conduct open-ended experimentation and systematically identify emergent model capabilities that could enable malicious misuse, particularly regarding high-consequence domains such as *Cyber Crime* and *Biosecurity Threats*. The efficacy of existing safeguards must be continuously monitored and refined based on the findings from these adversarial tests and real-world misuse detection data. 2. Implement Granular Input/Output Validation and Access Controls Deploy technical safeguards to limit the model's ability to facilitate harmful outcomes, including strict input/output validation, prompt injection defenses, and limiting the model's access to sensitive data or high-risk functions (e.g., code execution, external APIs). This is crucial for mitigating threats related to *Cyber Crime* and unauthorized data access, by ensuring the model operates within intended boundaries. 3. Establish Misuse-Focused Threat Profiles and Collaboration Frameworks Before training and deployment, create and maintain detailed threat profiles that identify potential malicious actors (e.g., state-affiliated groups for *Politically Motivated Misuse*) and forecast the specific harmful activities they may attempt. Collaborate with industry partners, regulators, and security agencies to share information on detected misuse tactics, thereby strengthening collective defense and promoting systemic risk reduction across the ecosystem.