Cybercrime
Closely related to discussions surrounding security and harmful content, the field of cybersecurity investigates how generative AI is misused for fraudulent online activities. A particular focus lies on social engineering attacks, for instance by utilizing generative AI to impersonate humans, creating fake identities, cloning voices, or crafting phishing messages. Another prevalent concern is the use of LLMs for generating malicious code or hacking.
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit79
Domain lineage
4. Malicious Actors & Misuse
4.2 > Cyberattacks, weapon development or use, and mass harm
Mitigation strategy
1. Implement a multi-layered defense against social engineering by mandating continuous, scenario-based security awareness training (SAT) focused on AI-driven deception (e.g., deepfakes, sophisticated phishing) and enforcing organization-wide Multi-Factor Authentication (MFA) and Zero-Trust access policies for all critical systems and GenAI endpoints. 2. Establish comprehensive LLM SecOps practices, including rigorous input filtering and sanitization to preempt prompt injection attacks, combined with real-time monitoring of model activity and outputs to detect and block the generation of malicious code or unauthorized access to sensitive data. 3. Institute a formal Generative AI Incident Response Plan (GenAI-IRP) that incorporates AI-specific threat containment and forensic procedures. This plan must be subject to quarterly testing and updating, ensuring the organization is prepared to manage the unique technical and reputational consequences of AI-enabled cyber incidents.