Unintended outbound communication by AI systems
AI systems that have the broad ability to connect to a network to obtain infor- mation could also end up sending data outbound in ways that neither providers, deployers, or end users intended [138]. This can happen if there is no whitelisting of communication channels (such as network connections or allowed protocols). In general, this can occur if the deployment of the AI system violates the prin- ciple of least privilege. Such outbound communication may lead to leakage of confidential data, or the AI system performing unwanted actions like sending emails or ordering goods on the internet.
ENTITY
2 - AI
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit1163
Domain lineage
7. AI System Safety, Failures, & Limitations
7.2 > AI possessing dangerous capabilities
Mitigation strategy
1. **Strict Enforcement of the Principle of Least Privilege (PoLP)** Apply granular, context-aware access controls to the AI system's runtime environment, ensuring its access permissions—particularly concerning network, file system, and API access—are strictly limited to the minimum set required for its intended function. The AI model's authority should be configured to inherit from and never exceed the permissions of the user or calling service interacting with it to prevent unauthorized actions and privilege escalation. 2. **Mandatory Outbound Communication Whitelisting** Establish a default-deny network security posture by mandating explicit whitelisting (allowlisting) of all approved network connections, IP addresses, domains, and protocols required for the AI system's operation. Any attempt by the AI system to establish communication outside this predefined and rigorously audited set of channels must be automatically blocked and logged. 3. **Continuous Behavioral Monitoring and Automated Containment** Integrate behavioral AI engines to continuously profile and monitor the AI system's process activity, API usage, and network traffic for deviations from its established baseline of normal operations. This monitoring system must be paired with an automated response capability to immediately quarantine the AI workload or disconnect its outbound connections upon the detection of anomalous or unauthorized communication attempts.