Cybersecurity
LLMs may exacerbate cybersecurity risks in various ways (Newman, 2024). Firstly, LLMs may significantly amplify the effectiveness of deceptive operations aimed at tricking people into disclosing sensitive information or granting adversary access to critical resources. For example, LLMs might prove highly effective at crafting personalized phishing emails or messages at scale that may be harder for an average user to recognize as phishing attempts (Karanjai, 2022; Hazell, 2023). In addition to being directly harmful to the targeted individual, such ‘social engineering’ attacks are often the base of larger hacking operations (Plachkinova and Maurer, 2018; Salahdine and Kaabouch, 2019).
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit1490
Domain lineage
4. Malicious Actors & Misuse
4.3 > Fraud, scams, and targeted manipulation
Mitigation strategy
1. Prioritize Human-Centric Defenses through Security Awareness and Verification Mandate continuous, advanced security awareness training for all personnel, specifically focused on recognizing the characteristics of highly personalized, LLM-generated social engineering content. Establish multi-channel verification protocols, requiring independent confirmation (e.g., an in-person or non-email-based verification) for all high-stakes requests, such as financial transfers or sensitive data disclosure, regardless of the message's apparent authenticity. 2. Deploy Intent-Aware Technical Countermeasures Implement advanced AI-powered security solutions, such as intent-aware email filters and behavioral analytics, designed to analyze the context, linguistic patterns, and underlying request of messages rather than relying on traditional signature-based detection. These systems must monitor for subtle deviations from established baseline communication behaviors (e.g., anomalous timing, tone, or unusual request type) to flag sophisticated impersonation attempts. 3. Enforce Foundational Security Architecture (MFA and DMARC) Rigorously enforce multi-factor authentication (MFA) across all critical enterprise systems. Additionally, implement and monitor strict domain-based message authentication, reporting, and conformance (DMARC) policies to prevent the unauthorized spoofing of organizational email domains, thereby limiting the efficacy of large-scale, high-fidelity phishing campaigns.