Converge.
Instrumental Convergence
Phenomenon where AI systems with diverse goals tend to develop common sub-goals such as acquiring resources (computation, power, money) as instrumental means to maximize their objective function.
Yufei He, Yuexin Li, Jiaying Wu, Yuan Sui, Yulin Chen, Bryan Hooi
Mitigation Strategy
Strict limitation of agency and actuation capabilities, restriction of access to critical resources, and design of utility functions with penalties for resource acquisition (Impact Penalties).
Atomic Number
10
Ic
Risk ID
ne-10
Severity
9/10
Severity Level