Exposing personal information
When personal identifiable information (PII) or sensitive personal information (SPI) are used in training data, fine-tuning data, or as part of the prompt, models might reveal that data in the generated output. Revealing personal information is a type of data leakage.
ENTITY
2 - AI
INTENT
2 - Unintentional
TIMING
2 - Post-deployment
Risk ID
mit1318
Domain lineage
2. Privacy & Security
2.1 > Compromise of privacy by leaking or correctly inferring sensitive information
Mitigation strategy
1. Mandate the implementation of advanced Data Loss Prevention (DLP) solutions, inclusive of real-time AI Prompt Protection, to automatically detect, redact, or block the submission of Personal Identifiable Information (PII) or Sensitive Personal Information (SPI) into model input channels. This must be complemented by rigorous data anonymization, pseudonymization, and minimization processes for all source data utilized in model training and fine-tuning to preclude the possibility of data memorization and subsequent output leakage. 2. Establish a foundational security posture by requiring end-to-end data encryption—at rest and in transit—for all sensitive datasets and model components. Concurrently, institute stringent Role-Based Access Controls (RBAC) integrated with Multi-Factor Authentication (MFA) across the entire AI pipeline to enforce the principle of least privilege, thereby limiting system and data access solely to verified, authorized personnel. 3. Develop and enforce a comprehensive AI Governance Policy that explicitly defines and prohibits the input of proprietary or sensitive information into unapproved AI services. This policy must be reinforced by continuous, role-specific employee security awareness training to mitigate the risk of human error, coupled with mandatory, routine audits of AI tool usage logs and model outputs to proactively detect and remediate potential compliance violations or inadvertent data exposure.