Information Science Risks
These risks pertain to the misuse, misinterpretation, or leakage of data, which can lead to erroneous conclusions or the unintentional dissemination of sensitive information, such as private patient data or proprietary research. Recent research has demonstrated how LLMs can be exploited to generate malicious medical literature that poisons knowledge graphs, potentially manipulating downstream biomedical applications and compromising the integrity of medical knowledge discovery [28]. Such risks are pervasive across all scientific domains.
ENTITY
3 - Other
INTENT
3 - Other
TIMING
2 - Post-deployment
Risk ID
mit1440
Domain lineage
2. Privacy & Security
2.1 > Compromise of privacy by leaking or correctly inferring sensitive information
Mitigation strategy
1. **Implement Privacy-Preserving Techniques and Data Minimization** Employ advanced privacy architectures such as **Differential Privacy** or **Data Masking/Anonymization** on all sensitive datasets, especially private patient data, *prior* to LLM training and inference. Concurrently, enforce a strict **Data Minimization** strategy, collecting and retaining only the information strictly necessary for model function, coupled with mandatory **Encryption** for data both at rest (storage) and in transit (transmission). 2. **Establish Continuous Knowledge Integrity Verification** Develop and deploy a rigorous integrity-focused governance layer for all knowledge bases, including Knowledge Graphs (KGs) and Retrieval-Augmented Generation (RAG) sources. This layer must utilize **Audit Trails** and regular, targeted **Security Audits** to detect subtle **Data Poisoning** or corruption (e.g., malicious literature, perturbation triples) that could compromise the accuracy and reliability of scientific or medical knowledge used by the agent. 3. **Enforce Principle of Least Privilege with Zero-Trust Access Controls** Apply a comprehensive **Role-Based Access Control (RBAC)** framework across the entire LLM system lifecycle, strictly adhering to the **Principle of Least Privilege** to limit data access to only authorized personnel and processes on a need-to-know basis. This system must operate under a **Zero-Trust Architecture** model and be supported by **Real-time Monitoring and Logging** to immediately identify and alert on suspicious activity, unauthorized access attempts, or potential data misuse.