ML//capability evaluation

A capability evaluation is a test of what a system can accomplish under specified conditions, run to expose the upper envelope of what it can do rather than to mimic an ordinary product. For that purpose evaluators may intentionally provide strong tools, long horizons and fewer safeguards; a cybersecurity evaluation, for instance, often measures maximum offensive capability apart from the restrictions of a normal deployment.

A capability evaluation is a test of what a system can accomplish under specified conditions, run to expose the upper envelope of what it can do rather than to mimic an ordinary product. For that purpose evaluators may intentionally provide strong tools, long horizons and fewer safeguards; a cybersecurity evaluation, for instance, often measures maximum offensive capability apart from the restrictions of a normal deployment.

This is useful science and dangerous headline material. Demonstrated capability is one ingredient of risk, alongside access, frequency, incentives, containment and real-world reliability. A model that succeeds once in a permissive laboratory has not thereby gained silent access to production.

Capability asks whether the system can do something. Deployment risk asks whether it can do it here, often enough, and with what consequences.