Dual use science risks
General- purpose AI systems could accelerate advances in a range of scientific endeavours, from training new scientists to enabling faster research workflows. While these capabilities could have numerous beneficial applications, some experts have expressed concern that they could be used for malicious purposes, especially if further capabilities are developed soon before appropriate countermeasures are put in place. There are two avenues by which general- purpose AI systems could, speculatively, facilitate malicious use in the life sciences: firstly by providing increased access to information and expertise relevant to malicious use, and secondly by increasing the ceiling of capabilities, which may enable the development of more harmful versions of existing threats or, eventually, lead to novel threats (404, 405).
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit772
Domain lineage
4. Malicious Actors & Misuse
4.2 > Cyberattacks, weapon development or use, and mass harm
Mitigation strategy
1. Prioritize Pre-deployment Misuse Risk Evaluation: Mandate rigorous, standardized evaluations of dual-use foundation models to measure and mitigate high-consequence biological and cyber misuses, such as facilitating bioweapon development or enabling autonomous cyberattacks, prior to release. This includes establishing a marginal risk framework to assess the extent to which a model exacerbates risks beyond pre-existing technologies 2. Establish Comprehensive and Multilateral Dual-Use Governance: Develop and enforce an international, multilateral governance framework to manage misuse risks across the entire AI supply chain, from model development to deployment. This framework must ensure accountability and transparency, aligning regulatory measures with the promotion of open-source and open-data principles to avoid unduly stifling legitimate scientific innovation 3. Enhance Defensive AI Capabilities: Invest in and deploy advanced, adaptive Defensive AI systems for real-time threat detection, anomaly identification, and automated incident response against sophisticated, AI-powered adversarial techniques. This involves integrating AI-native frameworks to detect synthetic media and continuously monitor for the subtle, high-volume activity characteristic of machine-speed cyberattacks