The Right Level of Reasoning
Sometimes It’s Not That Simple
We can spend a surprising amount of time trying to understand why other people do things.
A customer does not reply. A vendor changes its pricing. Someone on a team suddenly disagrees with a decision. A competitor launches a product similar to ours.
Sometimes the obvious explanation is enough. Sometimes it isn't.
The difficulty is that other people are not static parts of the environment. They have goals, information, constraints, and beliefs. They react to what happens. Sometimes they are also trying to understand us at the same time.
Game theory has a long history of studying these strategic interactions.
Years ago, I reviewed research on a related problem: how machine-learning agents behave in environments with other agents. One question was how to classify increasingly sophisticated ways of accounting for what the other agents were doing.
Translated from machines to people, the progression looks something like this:
1. The world happens to me.
Treat other people's behavior as part of the environment.
2. The world has changed.
Notice that patterns have shifted and adapt.
3. People have types.
Recognize recurring behaviors and respond accordingly.
4. People have reasons.
Infer goals, incentives, information, and constraints.
5. People have models of me.
Understand that your actions change their beliefs, and therefore their future actions.
Read from top to bottom, this looks like a ladder, or a progression in reasoning sophistication.
And often, but not always, it is useful to move up that ladder.
More sophisticated reasoning is not necessarily better reasoning.
The Prestige and Succession: Two Sides of Strategic Reasoning
Movies and series give us particularly good examples of this problem. Much of what makes their characters interesting is strategic interaction: people trying to understand what others want, what they know, and what they think everyone else will do.
The Prestige and Succession show opposite sides of the problem particularly well.
The Prestige (It’s Not That Simple)
The Prestige is a movie about two rival magicians, Borden and Angier, obsessed with understanding and surpassing each other's illusions.
Borden develops a trick called The Transported Man. He enters one side of the stage and appears almost immediately at the other.
Angier becomes obsessed with discovering how it works.
Cutter, the engineer who designs Angier's illusions, gives him an explanation almost immediately:
Cutter: “He uses a double.”
Angier: “It's not that simple. This is a complex illusion.”
We have a very simple explanation. But Angier, amazed by what he has seen, searches for something more sophisticated.
Cutter's basic explanation was right.
Angier rejects the correct model because it seems too simple.
Maybe the lesson is: Uncertainty makes complexity attractive.
When we don't understand why something is happening, a more sophisticated explanation can feel like progress.
A Customer Doesn't Reply
The ordinary version happens constantly.
Suppose a customer does not reply to an email.
- Maybe they are busy.
- Or they are evaluating another vendor.
- Or they are deliberately slowing the process because they know we want to close the deal.
- Or perhaps they expect us to interpret the delay as a lack of interest, making us more willing to improve our offer.
Every additional explanation can be plausible.
But plausibility is not evidence.
The fact that we can construct the more sophisticated explanation does not make it correct.
Succession (But Sometimes It Really Isn’t That Simple)
Succession follows the Roy family, owners of a large media company, and much of the series revolves around the question of who will eventually replace Logan Roy, the company's founder and CEO.
It is also a series full of people trying to understand what other people really want.
At one point, Logan tells his daughter Shiv that he wants her to succeed him as CEO.
The simple model is straightforward:
Logan wants Shiv to become CEO.
But there are other possible models.
- Perhaps Logan wants to bring Shiv back into the company.
- Perhaps he wants to pull her away from politics.
- Perhaps he wants to keep her close enough to control her.
- Perhaps he wants to neutralize a potential threat.
- Perhaps several of these things are true at once.
In the world of Succession, taking what someone says literally can be a serious mistake.
The difficult problem is deciding how much reasoning the situation actually requires.
Choosing the Model
Imagine several possible models of another person's behavior.
- At the simplest level, nothing strategic is happening.
- At another level, they are reacting to recent events.
- At another, their behavior reflects relatively stable incentives or constraints.
- Go further and they may be anticipating what we will do.
- Further still, they may be thinking about what we think they will do.
Continuing to add levels can produce a cycle where each explanation creates another possible explanation.
Sometimes the way out is not another level of reasoning. It is another observation.
If a customer says they are interested, I can spend hours reasoning about what “interested” really means.
Or I can see whether they will schedule the next meeting.
Actions can distinguish between explanations that words and more reasoning cannot.
The Experienced-Person Trap
There is another reason I find this problem interesting.
Experience makes us better at seeing explanations that we would previously have missed.
After enough negotiations, we recognize anchoring.
After enough customer conversations, we recognize polite interest that is unlikely to go anywhere.
That is real learning.
But it creates another failure mode.
We can stop believing simple explanations.
The skill, then, is not simply moving upward through increasingly sophisticated models.
It is being able to move up and down.
Sometimes another level of reasoning is exactly what is needed.
Other times it only gives us another plausible story.
Strategic sophistication is not thinking the maximum number of steps ahead.
It is knowing how many steps ahead the situation requires.
Sometimes, as Angier says, it's not that simple.
And sometimes it is.