mathematics//decision theory//value of information
The value of information is a measure of how much an observation improves the next decision, net of what it costs to acquire, and it is what an engineer or an agent weighs before taking another measurement: a second camera, a manual inspection, one more radar sweep, opening one more file (the criterion behind active perception). In decision theory it is the expected cost of deciding now minus the expected cost of first observing \(y\) and then deciding the best for each possible result:
The value of information is a measure of how much an observation improves the next decision, net of what it costs to acquire, and it is what an engineer or an agent weighs before taking another measurement: a second camera, a manual inspection, one more radar sweep, opening one more file (the criterion behind active perception). In decision theory it is the expected cost of deciding now minus the expected cost of first observing yyy and then deciding the best for each possible result:
VOI=minaE[C(a,s)]−Ey[minaE[C(a,s)∣y]]\mathrm{VOI} = \min_{a} \mathbb{E}[C(a,s)] - \mathbb{E}_{y}\Big[\min_{a} \mathbb{E}[C(a,s)\mid y]\Big]VOI=aminE[C(a,s)]−Ey[aminE[C(a,s)∣y]]
An inspection line sees parts arrive 2% defective. Rejecting a good part costs 3 euros, shipping a bad one 300. With no further information the best policy is to reject everything (2.94 euros per part, against 6 for accepting everything). A perfect inspection would bring the cost to zero, so it is worth up to 2.94 euros per part: that is the expected value of perfect information, the ceiling no test can exceed. An imperfect inspection is worth less, computed the same way with its error rates.
An observation is worth what it changes in the decision.
If every possible result would lead to the same action, the observation is worth nothing, however precise it is, so do not look.
Value is stage-dependent. A detour to look matters before committing to a corridor and not after; in a POMDP this becomes sequential, with actions chosen for what they reveal, and in sequential allocation the observation decides whether the next drone needs to fly at all.
Information value is not byte count. Before opening an 8,000-line file, a programmer usually searches for the symbol, and the eight-line match can be worth more because it removes the relevant uncertainty while preserving attention for later evidence. For a software agent the cost includes tokens, latency, money, tool calls, risk and context pollution; broad search pays when the location is unknown, focused reading once candidates exist, tests once there is a falsifiable hypothesis.
More data is not better data. The useful quantity is uncertainty removed per unit of a constrained budget, and the information measured by information theory becomes valuable only relative to a decision and its costs: a sensor with high mutual information about a variable that changes no decision is worth nothing to the plant.