Misuse
The misuse of generative AI refers to any deliberate use that could result in harmful, unethical or inappropriate outcomes (Brundage et al., 2020). A prominent field that faces the threat of misuse is education. Cotton et al. (2023) have raised concerns over academic integrity in the era of ChatGPT. ChatGPT can be used as a high-tech plagiarism tool that identifies patterns from large corpora to generate content (Gefen & Arinze, 2023). Given that generative AI such as ChatGPT can generate high-quality answers within seconds, unmotivated students may not devote time and effort to work on their assignments and essays. Hence, in the era of generative AI, the originality of the work done by students could be difficult to assess. Text written by ChatGPT is regarded as plagiarism and is not acceptable (Thorp, 2023). Another form of misuse is cheating in examinations. If students have access to digital devices during examinations, they can resort to using ChatGPT to assist them in answering the questions. To address potential misuse in education, AI-generated content detectors such as Turnitin could be used and strict proctoring measures will need to be deployed (Susnjak, 2022). However, the challenges go beyond content detection and examination proctoring as the line between what is considered appropriate versus inappropriate use of ChatGPT could be fuzzy.
ENTITY
1 - Human
INTENT
1 - Intentional
TIMING
2 - Post-deployment
Risk ID
mit537
Domain lineage
4. Malicious Actors & Misuse
4.3 > Fraud, scams, and targeted manipulation
Mitigation strategy
1. Establish and rigorously enforce clear institutional policies and academic integrity standards specifically governing the permissible and prohibited use of generative AI tools in coursework and research. 2. Deploy advanced AI-generated content detection technologies (e.g., Turnitin, Copyleaks) and mandate the consistent labeling or citation of all AI-assisted content to ensure authenticity and authorship attribution. 3. Implement enhanced proctoring protocols and controlled, secure assessment environments (both in-person and remote) to minimize opportunities for the unauthorized use of generative AI during high-stakes examinations.