Software Security Issues
The software development toolchain of LLMs is complex and could bring threats to the developed LLM.
ENTITY
3 - Other
INTENT
3 - Other
TIMING
1 - Pre-deployment
Risk ID
mit19
Domain lineage
2. Privacy & Security
2.2 > AI system security vulnerabilities and attacks
Mitigation strategy
1. Establish a comprehensive Software Bill of Materials (SBOM) for all machine learning components, external libraries, and dependencies to ensure continuous visibility and verifiable provenance across the entire LLM development toolchain. 2. Implement a rigorous process for the vetting of all third-party models and open-source components, including continuous scanning for known vulnerabilities and the systematic application of security patches to mitigate risks associated with outdated or insecure dependencies. 3. Employ strict runtime isolation mechanisms, such as least-privilege sandboxing, for all external tools and code execution capabilities accessed by the LLM to prevent remote code execution or unauthorized system access via compromised toolchain elements.