Confidential data in prompt
Confidential information might be included as a part of the prompt that is sent to the model.
ENTITY
3 - Other
INTENT
2 - Unintentional
TIMING
2 - Post-deployment
Risk ID
mit1295
Domain lineage
2. Privacy & Security
2.1 > Compromise of privacy by leaking or correctly inferring sensitive information
Mitigation strategy
1. Implement automated data redaction, anonymization, and tokenization using Data Loss Prevention (DLP) tools at the AI gateway or client-side to proactively identify and remove sensitive information (e.g., PII, financial data, corporate secrets) from the prompt before it is processed by the Large Language Model. 2. Utilize Confidential Computing (CC) and Trusted Execution Environments (TEEs) to protect the user prompt during inference. This architectural control ensures the prompt remains confidential, even from the cloud provider and the LLM service provider, as it is processed in a secured memory enclave. 3. Establish a mandatory user education and awareness program that clearly outlines the organizational policy prohibiting the submission of sensitive or confidential data into the LLM system, reinforcing the principle of least privilege and data minimization for all user inputs.