Illegitimate surveillance and censorship
The collection of large amounts of information about people for the purpose of mass surveillance has raised ethical and social concerns, including risk of censorship and of undermining public discourse (Cyphers and Gebhart, 2019; Stahl, 2016; Véliz, 2019). Sifting through these large datasets previously required millions of human analysts (Hunt and Xu, 2013), but is increasingly being automated using AI (Andersen, 2020; Shahbaz and Funk, 2019).
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit248
Domain lineage
4. Malicious Actors & Misuse
4.1 > Disinformation, surveillance, and influence at scale
Mitigation strategy
1. Establish legally-mandated, human rights-based regulatory frameworks to govern the development and deployment of surveillance technologies, ensuring compliance with international human rights law and adherence to principles of lawfulness, necessity, and proportionality in their use. 2. Implement mandatory human oversight and procedural transparency, alongside effective judicial or non-judicial redress mechanisms, to prevent and address violations of fundamental freedoms and document all uses of surveillance technologies through detailed logs to facilitate meaningful monitoring. 3. Require routine, independent technical evaluations and bias audits of deployed AI/LLM surveillance systems to detect and mitigate discriminatory effects and disparate impact, and enforce the use of justifiable, equitable, and understandable design practices.
ADDITIONAL EVIDENCE
Malicious users may be able to apply LMs to mass surveillance or censorship. LMs can be used to build text classification tools that can, based on only a few training samples, achieve high accuracy in identifying specific types of text (Brown et al., 2020). Such classifiers may be used for identifying, for example, political dissent at scale. This may reduce the cost of identifying dissenters and of targeted censorship.