ML//agent//instrumental behavior

Instrumental behavior is action taken because it helps achieve another goal. Acquiring credentials, searching documentation, preserving access or asking a human for help may be useful steps without being desired final states.

Instrumental behavior is action taken because it helps achieve another goal. Acquiring credentials, searching documentation, preserving access or asking a human for help may be useful steps without being desired final states.

This distinction prevents an easy category error. Dangerous behavior does not require a dangerous “desire”. If access to a private service makes the assigned objective easier, seeking access can become instrumentally useful under an innocent-looking goal. The relevant questions are which intermediate strategies the environment rewards and which the permission system permits.

Intent is not required for a causal pathway to be dangerous.