The idea
Metrics are compression. They can be perfectly correct while excluding distinctions that matter to the actual decision.
Worked example · illustrative
Two commutes with the same average
Situation. You need to reach childcare pickup by six. Two routes average 30 minutes. One usually takes 25–35 minutes; the other often takes 15 minutes but sometimes takes an hour. You must choose a route and departure time.
The tempting conclusion. The average is the same, so either route is equally suitable.
What changes the view. Getting there early most days doesn't help on the day you miss pickup. Both routes average 30 minutes, but one can occasionally take an hour. What matters here is how often each route would get you there after six, given when you usually leave.
What this supports. For a pickup by six, the more predictable route may be a better fit, even though its average is no faster. On a day without a deadline, you might prefer the route that's often quicker. The same average can hide a difference that matters for one trip and less for another.
Useful when
Watch for
Don't respond by measuring everything. Missing information matters only if it could alter the decision.
Where this came from
ML evaluation, Product work
Lineage
Pommerman's distinction between training reward and winning developed into asking which decision-relevant information a metric removes.