ML//agent//instrumental behavior

Instrumental behavior is action taken because it helps achieve another goal. Acquiring credentials, searching documentation, preserving access or asking a human for help may be useful steps without being desired final states.

Instrumental behavior is action taken because it helps achieve another goal. Acquiring credentials, searching documentation, preserving access or asking a human for help may be useful steps without being desired final states.

This distinction prevents an easy category error. Dangerous behavior does not require a dangerous desire. If access to a private service makes the assigned objective easier, seeking access can become instrumentally useful under an innocent-looking goal. The relevant questions are which intermediate strategies the environment rewards and which the permission system permits; removing unnecessary capabilities (minimum privilege) makes many harmful instrumental strategies physically unavailable.

Intent is not required for a causal pathway to be dangerous.