Use this when you’re discussing an interview and need to distinguish observed behavior from expectations about future work.
You’ll be able to describe what an answer supports, what remains untested, and a useful follow-up check.
01 / The problem
From observation to prediction
A technical interview shows how someone responds to a particular problem. Hiring asks us to predict their contribution on real work, with our team’s support and constraints.
The jump from an encouraging answer to a broad expectation can hide assumptions. Separate what you saw, what it suggests, and what remains untested.
02 / The distinction
What you observed is evidence. What you expect them to do at work is a prediction. Keep the uncertainty visible.
Evidence
What did we observe?
Expectation
What work might they handle, with what support?
Uncertainty
What remains untested, and how would we check it?
03 / Examples
Compare the evidence
Choose an interview scenario. Each tab shows a different scenario; compare the evidence and what it supports.
Two illustrative scenarios.
ML evaluation
You’re hiring an ML engineer who will help decide whether models are ready to ship. You show a fraud-detection model with 92% accuracy and ask whether to deploy it. The candidate warns of possible data leakage: the test might use later customer disputes to predict whether an earlier payment was fraudulent. Those disputes would not yet be known when the model checks a new payment, so the test score could be misleading.
A tempting conclusion: “They spotted leakage. They have good evaluation judgment.”
That could be true. But naming a familiar problem and knowing what to do about it are different pieces of evidence.
What happens after they name the problem?
They name the risk
They act on the risk
What happened
They mention possible leakage, then recommend deploying because 92% sounds high.
They ask whether the test used information available at payment time, propose retesting with only that information on later payments, and hold off on recommending deployment until that check is done.
What this supports
In this answer, they recommended release without resolving a risk they had identified.
That raises a specific concern about their use of evaluation evidence. You don’t yet know how they would respond to guidance.
They connect a concern to a test and a release condition. That supports a more specific expectation: they may question an unreliable evaluation before recommending release.
They name the risk
What happened
They mention possible leakage, then recommend deploying because 92% sounds high.
What this supports
In this answer, they recommended release without resolving a risk they had identified.
That raises a specific concern about their use of evaluation evidence. You don’t yet know how they would respond to guidance.
They act on the risk
What happened
They ask whether the test used information available at payment time, propose retesting with only that information on later payments, and hold off on recommending deployment until that check is done.
What this supports
They connect a concern to a test and a release condition. That supports a more specific expectation: they may question an unreliable evaluation before recommending release.
When they name the risk, they set aside an identified concern when recommending release. When they act on the risk, they turn that concern into a check before release. These are different observations about judgment in this discussion. Whether the candidate can revise their reasoning with guidance or apply it to a new problem remains uncertain.
Example wording for the hiring discussion
Use these sample statements to describe each response to the hiring team: what happened, what it suggests about future work, and what remains uncertain.
They name the risk
“They identified possible leakage but recommended release without resolving it. That raises a concern about how they used evaluation evidence in this answer. I’d need to see them address that concern before trusting a release recommendation. I don’t yet know how they would revise their reasoning with guidance.”
A useful follow-up: Ask what information would be available when a payment arrives, and what test result would change their recommendation. Note any hints you provide and whether they use them.
They act on the risk
“In this case, they proposed retesting with information available at payment time and waiting for that result before recommending release. I’d expect them to contribute to model reviews, with help on our domain’s metrics. I haven’t yet seen whether they do the same with an unfamiliar problem.”
A useful follow-up: Give them a different evaluation scenario and ask what would make them change their recommendation. Look for whether they adapt the reasoning rather than repeat the same test.
System design
You’re hiring a backend engineer to work on a service that uses a message queue. The candidate hasn’t used your queue product. A queue may deliver the same payment request more than once; without a safeguard, that could charge a customer twice. You still need to understand whether they can reason about failures, repeated messages, and recovery.
A tempting conclusion: “They don’t know our stack. They’ll struggle with reliability.”
Unfamiliarity with a product is visible. Their ability to reason about reliability needs its own evidence.
Two different reasons you might lack confidence
You haven’t tested the capability
You observed a problem in the reasoning
What happened
You asked which queue products they had used, but never discussed delivery failures or repeated messages.
You explained that the queue can deliver a message more than once.
Even after discussing the consequences, they kept a design that could charge the customer twice and offered no way to prevent or detect it.
What this supports
You know they lack experience with this product. You don’t yet know how they reason about these failures.
You have evidence of difficulty handling this failure mode in this discussion. The concern is more specific than unfamiliarity with the product.
You haven’t tested the capability
What happened
You asked which queue products they had used, but never discussed delivery failures or repeated messages.
What this supports
You know they lack experience with this product. You don’t yet know how they reason about these failures.
You observed a problem in the reasoning
What happened
You explained that the queue can deliver a message more than once.
Even after discussing the consequences, they kept a design that could charge the customer twice and offered no way to prevent or detect it.
What this supports
You have evidence of difficulty handling this failure mode in this discussion. The concern is more specific than unfamiliarity with the product.
An interview that never tested failure handling leaves a question open. An answer that overlooks a failure after it has been explained gives you something concrete to examine. Neither warrants a broad claim about every aspect of the candidate’s ability.
Example wording for the hiring discussion
Use these sample statements to describe each response to the hiring team: what happened, what it suggests about future work, and what remains uncertain.
What the interviewer supplied matters. When you observed a problem in the reasoning, you had explained repeated delivery. Assess how they used that explanation; this does not show whether they would have identified the risk independently.
The role changes the forecast. Judge the answer against the role’s responsibilities and the support the team can provide.
You haven’t tested the capability
“They’ll need to learn our queue. Before judging their reliability skills, I’d discuss a repeated-payment scenario and ask how they would prevent charging a customer twice. Product experience and failure reasoning are separate questions.”
A useful follow-up: Explain the delivery behavior without requiring product-specific vocabulary. Ask them to trace one message through a failure and recovery. Then assess the reasoning against the role’s expectations.
You observed a problem in the reasoning
“After duplicate delivery and its consequences were explained, they kept a design that could charge a customer twice. For a junior role with design reviews, I’d expect them to need guidance on this failure mode. For a senior role leading payment design, this raises a stronger concern. I haven’t yet established whether this is a local gap or a broader difficulty.”
A useful follow-up: Give targeted feedback and ask them to revise the design, then trace a repeated message through failure and recovery. This checks how they use support. A similar, unfamiliar scenario can separately test whether the reasoning transfers.
When a candidate raises a concern, look at what happens next. A proposed check and a recommendation that depends on its result show more than naming the concern alone.
Identify the actual gap.
Not asking about a capability leaves it untested. Watching someone struggle after the problem is explained gives you a specific difficulty to investigate. These need different follow-ups.
Account for the prompt.
What the candidate demonstrated depends partly on what the interviewer explained, hinted at, or left unstated.
Keep predictions conditional.
One interview response does not establish performance across unfamiliar problems. Name the role, support, and conditions your expectation assumes so it can be questioned and revisited.
Wording to take with you
We observed [specific behavior]. This suggests [a specific expectation or concern] about [work], given [conditions and support]. We’re still unsure about [gap], which we could check by [next step].
If the capability wasn’t tested, name that gap directly:
We haven’t tested [capability], so we don’t yet have enough evidence to predict [how they would handle specific work]. We could check it by [next step].
Check the standard, too. If you call a gap “learnable” for one candidate and “disqualifying” for another, explain which requirement of the role justifies the difference.