Enabling malicious actors and harmful actions
Some uses of AI have been deeply concerning, namely voice cloning [58] and the generation of deep fake videos [59]. For example, in March 2022, in the early days of the Russian invasion of Ukraine, hackers broadcast via the Ukrainian news website Ukraine 24 a deep fake video of President Volodymyr Zelensky capitulating and calling on his soldiers to lay down their weapons [60]. The necessary software to create these fakes is readily available on the Internet, and the hardware requirements are modest by today’s standards [61]. Other nefarious uses of AI include accelerating password cracking [62] or enabling otherwise unskilled people to create software exploits [63, 64], or effective phishing e-mails [65]. Although some believe that powerful AI models should be prevented from running on personal computers to retain some control, others demonstrate how inglorious that effort may be [66]. Furthermore, as ChatGPT-type systems evolve from conversational systems to agents, capable of acting autonomously and performing tasks with little human intervention, like Auto-GPT [67], new risks emerge.
ENTITY
3 - Other
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit61
Domain lineage
4. Malicious Actors & Misuse
4.0 > Malicious use
Mitigation strategy
1. Mandate the universal application of Multi-Factor Authentication (MFA) and the enforcement of high-entropy, unique password policies across all critical organizational and user-facing accounts to neutralize the effectiveness of AI-accelerated password cracking and credential stuffing attacks. 2. Implement a multi-layered defense system comprising AI-powered detection technologies—such as deepfake recognition and digital watermarking protocols—to verify media authenticity and concurrently integrate robust input/output guardrails within generative AI systems to resist adversarial manipulation, including sophisticated prompt injections. 3. Establish an explicit AI governance framework for autonomous systems that mandates human-in-the-loop oversight (e.g., requiring explicit confirmation for high-consequence actions), enforces strict access control mechanisms for agents, and requires comprehensive logging and traceability of all agent actions and decisions to ensure accountability and enable forensic analysis of unintended or malicious behavior.