AI-rulemaking for human behaviour
AI rulemaking for humans can be the result of the decision process of an AI system when the information computed is used to restrict or direct human behavior. The decision process of AI is rational and depends on the baseline programming. Without the access to emotions or a consciousness, decisions of an AI algorithm might be good to reach a certain specified goal, but might have unintended consequences for the humans involved (Banerjee et al., 2017).
ENTITY
2 - AI
INTENT
3 - Other
TIMING
2 - Post-deployment
Risk ID
mit326
Domain lineage
7. AI System Safety, Failures, & Limitations
7.3 > Lack of capability or robustness
Mitigation strategy
1. Prioritize the implementation of robust human-in-the-loop oversight mechanisms that explicitly grant human operators the authority and technical means to disregard, override, or reverse the AI system's output or restrictive action, alongside a fail-safe capability to interrupt the system's operation 2. Establish a formal, multi-dimensional evaluation framework for assessing second-order and unintended ethical, societal, and operational consequences stemming from the AI's 'rulemaking' behavior before and during deployment to proactively mitigate unforeseen harms 3. Enhance the transparency and interpretability of the AI's decision process through explainable AI (XAI) tools, enabling human overseers to understand the system's rationale, capacities, and limitations, thereby countering automation bias and facilitating informed intervention