ML//agent//evaluation harness

An evaluation harness is the machinery that presents tasks, provisions environments, exposes tools, enforces limits and computes scores. For agents, it is part examiner, part laboratory and part operating environment.

An evaluation harness is the machinery that presents tasks, provisions environments, exposes tools, enforces limits and computes scores. For agents, it is part examiner, part laboratory and part operating environment.

That means an agent score never belongs cleanly to the model. CPU, memory, timeouts, network policy, tool ergonomics and scoring bugs can all change measured performance. A suspicious trajectory may reveal model behavior, harness behavior or an interaction between both.

If the laboratory changes the phenomenon, “same test” is only a label.