The idea
A result is more convincing when the surrounding evidence has a structure consistent with the explanation. An isolated maximum may be real, but it demands explanation.
Worked example · illustrative
Choosing a price for a fundraising lunch
Situation. A community group sells lunch at three weekend events to raise funds. They try a different price each time, and the middle price brings in much more money than the other two. Now they’re planning another lunch and wondering what to charge.
The tempting conclusion. The middle price seems to be the sweet spot. Why not use it again?
What changes the view. The day with the strongest sales also had a festival nearby, so there may simply have been more people around. If the price was the reason for the success, you’d expect it to do well on other days with similar crowds. Attendance and sales per visitor could help make sense of what happened, as could trying that price again on a day without a festival.
What this supports. The middle price still looks promising, but one busy day leaves some uncertainty about why it worked. A strong showing with a more typical crowd would make the price explanation more convincing. It could be a price people respond especially well to; the festival just makes that harder to tell from this result alone.
Useful when
Watch for
Real phenomena can have thresholds and discontinuities; smoothness is evidence, not proof.
Where this came from
Quantitative trading, Machine learning
Lineage
Skepticism about unusually good quantitative research results developed into examining the structure of surrounding evidence.