Information & Safety Harms
AI systems leaking, reproducing, generating or inferring sensitive, private, or hazardous information
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
2 - Post-deployment
Risk ID
mit266
Domain lineage
2. Privacy & Security
2.1 > Compromise of privacy by leaking or correctly inferring sensitive information
Mitigation strategy
1. Implement Robust Data Security Architectures Enforce end-to-end encryption for all sensitive data at rest and in transit. This must be coupled with strict, role-based access control (RBAC) and multi-factor authentication (MFA) across all stages of the AI lifecycle (data pipeline, training environments, and deployment infrastructure) to prevent unauthorized access. 2. Employ Differential Privacy Mechanisms Integrate mathematically rigorous Differential Privacy (DP) into the model training and output generation processes. DP ensures that the presence or absence of any single individual's data point does not significantly affect the aggregate output, thus providing quantifiable assurance against privacy-compromising data inference and leakage. 3. Deploy Advanced Data Loss Prevention (DLP) Solutions Implement real-time DLP systems with semantic risk detection to monitor and control information flow during inference and user interaction. These systems must be capable of preemptively blocking, redacting, or alerting on the entry of sensitive data into models (prompt protection) and the exfiltration of private or confidential information from model outputs.
ADDITIONAL EVIDENCE
Example: An AI system leaks private images from the training data (Carlini et al., 2023a)