Back to the MIT repository
4. Malicious Actors & Misuse2 - Post-deployment

Malicious Use

AI systems reducing the costs and facilitating activities of actors trying to cause harm (e.g. fraud, weapons)

Source: MIT AI Risk Repositorymit269

ENTITY

2 - AI

INTENT

3 - Other

TIMING

2 - Post-deployment

Risk ID

mit269

Domain lineage

4. Malicious Actors & Misuse

223 mapped risks

4.0 > Malicious use

Mitigation strategy

1. Establish a Layered Security Framework with Strict Access Controls Implement robust Multi-Factor Authentication (MFA) and the principle of least privilege (Zero Trust architecture) for all access points to AI models, training datasets, and deployment infrastructure. For generative AI systems prone to impersonation, integrate advanced identity-proofing mechanisms such as liveness detection and behavioral biometrics to counter deepfake-based fraudulent access attempts. 2. Fortify Model Resilience Through Adversarial Training and Validation Mandate adversarial training during the model development lifecycle to proactively expose the AI system to manipulated inputs, thereby strengthening its defense against evasion attacks. Concurrently, deploy automated data validation pipelines and redundant dataset checks to ensure data integrity and prevent model corruption via poisoning attacks, which would otherwise facilitate sustained malicious use. 3. Implement Real-Time Continuous Monitoring and Auditable Oversight Deploy continuous monitoring solutions that track input patterns, output anomalies, and system performance in real-time to detect suspicious activity indicative of model extraction attempts or unauthorized generation of harmful content. Maintain comprehensive audit trails and logs to ensure full traceability and accountability for all model decisions and system behaviors, facilitating rapid post-deployment incident response.

ADDITIONAL EVIDENCE

Example: AI systems can generate deepfake images cheaply, at scale (Amoroso et al., 2023)