The Shape of the Evidence
What would I expect to see around this result if my explanation were actually right?
Use when a result looks convincing, but you’re unsure why.
From: Quantitative trading, Machine learningToolbox
Find a useful question for the decision you’re facing.
Start with a situation you recognize. Each tool explores a question through an example.
Find your situation
A result looks impressive, but I’m unsure whether to trust it.
The Shape of the Evidence →Check whether the surrounding observations fit the explanation.
Another angle for A result looks impressive, but I’m unsure whether to trust it.
What Did the Metric Throw Away? →The headline number may hide differences that matter.
I understand the findings, but I’m unsure what we should do.
Decision-Complete →Examine the reasoning connecting the findings to a practical choice.
Another angle for I understand the findings, but I’m unsure what we should do.
What Is Worth Learning? →That reasoning reveals an unanswered question, and you need to decide what to investigate.
We keep gathering information without getting closer to a decision.
What Is Worth Learning? →Identify which uncertainty could actually change the choice.
Another angle for We keep gathering information without getting closer to a decision.
Two Clocks →Further learning may help, but you need to consider what waiting could cost.
I’m unsure whether to act now or wait.
Two Clocks →Consider how information and available options change during the delay.
Another angle for I’m unsure whether to act now or wait.
Position Before You Know →You cannot commit yet, but could prepare in ways that help across plausible outcomes.
More situations (2)
Something feels wrong, but I can’t explain why.
From Intuition to Argument →Identify what your concern is noticing and make the reasoning inspectable.
Another angle for Something feels wrong, but I can’t explain why.
Look at the Boundary →The concern involves different areas of expertise, and understanding their connection may reveal the problem.
A detailed plan includes things we haven’t actually agreed on.
Surface the Hidden Decisions →Separate agreed commitments from assumptions and suggestions.
Another angle for A detailed plan includes things we haven’t actually agreed on.
Change the Representation →People agree on the wording but understand its scope or consequences differently.
Browse all decision tools
What are you dealing with?
More filters
What Did the Metric Throw Away?
Could information removed by this metric change the decision?
Use when the headline number doesn’t tell the whole story.
From: ML evaluation, Product workLook at the Boundary
What might be visible from the other domain that isn't visible from mine?
Use when people see different problems from different sides.
From: Startups, Engineering, Parenting, Knowledge-boundary researchChange the Representation
Is this actually small, or does it only look small in the representation I'm using?
Use when the same change looks small to one person and large to another.
From: Engineering, Product workFrom Intuition to Argument
What exactly is my intuition noticing, and why does it matter to this decision?
Use when something feels wrong, but you can’t yet explain it.
From: Research, Startups, Technical judgmentDecision-Complete
Can I construct a defensible path from the evidence to the decision?
Use when you have findings, but no clear recommendation.
From: Academic research → applied R&DPreserve the Option
What does this decision make possible or impossible afterward?
Use when a choice now could limit what you can do later.
From: Startups, Research and development, Star TrekTwo Clocks
What happens to the decision while I’m waiting?
Use when waiting could help, but your options may disappear.
From: StartupsWhat Is Worth Learning?
What could I learn that would change my choice, and what’s a simple, reliable way to find out?
Use when you’re gathering information without getting closer to a choice.
From: Research, Product workPosition Before You Know
What can I do now that leaves me well positioned for the next states that matter?
Use when you can prepare now, even if you can’t commit yet.
From: FootballExplore, Then Exploit
Do I know where the gains are likely to be, or am I already optimizing one path?
Use when you’re unsure where to focus your effort.
From: Applied machine learning, ResearchLook Before You Model
What can I learn about the problem before choosing the solution?
Use when you’re choosing a solution before understanding the problem.
From: Applied machine learningStart Simple
What is the simplest approach that would teach me whether the core idea works?
Use when you’re making a solution complicated before trying the basics.
From: Quantitative research, Machine learningWhich Game Are We Playing?
What am I assuming the other person is trying to achieve?
Use when you haven’t checked what matters to the other person.
From: Game theory, Multi-agent learningThe Right Level of Reasoning
How much strategic reasoning does the evidence justify here?
Use when you may be reading too much strategy into someone’s behavior.
From: Game theory, Multi-agent learning, The Prestige, SuccessionStart With a Hypothesis
What did I expect to observe before I saw the result?
Use when you keep trying things until the results look good.
From: Quantitative research, Finance, Machine learningSolve the Constrained Problem
What solution works under the constraints that actually exist?
Use when your preferred solution won’t fit the time, money, or rules.
From: Pommerman, Competitions, Applied machine learningProductive Struggle
Will helping now improve the immediate outcome but reduce the learning?
Use when you could help someone now, but they also need to learn.
From: Mentoring, SupervisionRaise the Resolution of Feedback
Have I made the work concrete enough for someone to give me the kind of feedback I want?
Use when you want useful feedback, but only get vague reactions.
From: PhD supervision, Internships, WritingConnect Work to Consequences
What feedback would help me understand what happened because of my work?
Use when the work looks good, but you’re unsure whether it helped.
From: Research, Product work, Startups, MentoringSurface the Hidden Decisions
Which decisions does this document appear to settle that we haven’t actually made?
Use when a detailed plan may be mistaken for an agreed plan.
From: Product work, AI-assisted writingNo tools match both filters. Try another problem or source.