Risks to privacy
General- purpose AI models or systems can ‘leak’ information about individuals whose data was used in training. For future models trained on sensitive personal data like health or financial data, this may lead to particularly serious privacy leaks. General- purpose AI models could enhance privacy abuse. For instance, Large Language Models might facilitate more efficient and effective search for sensitive data (for example, on internet text or in breached data leaks), and also enable users to infer sensitive information about individuals.
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
2 - Post-deployment
Risk ID
mit781
Domain lineage
2. Privacy & Security
2.1 > Compromise of privacy by leaking or correctly inferring sensitive information
Mitigation strategy
1. Prioritize Data Minimization and Privacy-Preserving Techniques: Employ data minimization principles, collecting and using only the strictly necessary data. When sensitive data is required, apply robust anonymization, pseudonymization, or synthetic data generation techniques to preclude the compromise of personally identifiable information (PII) in training and input datasets. 2. Deploy Advanced Data Loss Prevention (DLP) and Secure Platforms: Implement next-generation DLP solutions with real-time prompt analysis to prevent the unauthorized input of confidential or sensitive corporate data into general-purpose AI models. Mandate the use of enterprise-grade AI platforms that contractually prohibit the use of customer inputs for model training, establishing a critical legal and technical boundary. 3. Establish Continuous Auditing and Behavioral Monitoring: Systematically audit the usage logs and outputs of deployed AI systems. Utilize anomaly detection and behavioral monitoring tools to proactively identify and flag unusual data extraction activities, potential policy violations, or inadvertent exposure of sensitive information, enabling rapid incident response.