systems engineering//verification

Verification is the work of demonstrating that a built system meets its requirements, at every level at which it was specified, and it is used to find each error at the stage where it is cheapest to fix: in a model, in compiled software, on the real processor, with the real electronics, on a bench, in the field. An error costs more the later it appears: a minute in simulation, an afternoon on the bench, a drone in the field, a fleet and a customer in production. Its sibling, validation, asks the other question, whether the requirements themselves describe what the user needed; verification checks the system against the spec, validation the spec against the world.

Verification is the work of demonstrating that a built system meets its requirements, at every level at which it was specified, and it is used to find each error at the stage where it is cheapest to fix: in a model, in compiled software, on the real processor, with the real electronics, on a bench, in the field. An error costs more the later it appears: a minute in simulation, an afternoon on the bench, a drone in the field, a fleet and a customer in production. Its sibling, validation, asks the other question, whether the requirements themselves describe what the user needed; verification checks the system against the spec, validation the spec against the world.

In the V-model verification is the right side of the V, each level checked against its mirror on the left: units against the design, the integrated system against the architecture, the whole against its requirements. For a controller or an autopilot the levels map onto the pyramid of X-in-the-loop testing, where more of the system is real at each step up and each step finds a different class of error, from design mistakes in model-in-the-loop to timing and bus faults in hardware-in-the-loop.

Three practices multiply what the pyramid is worth. Log replay runs new estimation code on recorded real data, the cheapest test with reality in it. Fault injection exercises the failure paths that normal tests never reach. Continuous integration keeps both running on every change, with each past breakage kept as a test.

The test bench has to be verified too. A simulator never compared against real logs is an opinion with nice graphics; the sim-to-real gap has to be measured against flight logs, and a simulator used as evidence is itself validated (simulation).

Testing samples behaviour; it cannot prove absence of errors. Where the stakes justify it, static analysis and formal methods prove properties of code or models for all inputs, and safety standards demand structural coverage to show the tests at least exercised the code (DO-178C).

What is deployed is what must be verified: a model quantized for the edge, preprocessing rewritten for production or a prototype sensor swapped for the series one each make a different system from the one tested (training-serving skew).

Verification is the half of systems engineering that turns a design into evidence.