Assumption Testing
What is Assumption Testing?
It is a tool for surfacing the key assumptions underlying the
attractiveness of a new business concept and using data to
assess the likelihood that these assumptions are true.
The approach acknowledges that any new business concept
is actually a hypothesis: a well-informed guess about what
customers desire and what they will value.
• The first step: Identify and articulate the assumptions.
• The second: Choose between two options for testing them
• New data in a field experiment in the marketplace (expensive) or
• Use existing data to conduct an analytical thought experiment
without going into the marketplace.
How Assumption Testing De-risks Project
• Why projects fail?- It is always because reality turns out to be
different than you thought it would be
• Perhaps customers don't want a new offering
• Your firm can't execute it
• Partners don't like it
• Competitors copy it quickly
• Launching new concepts- Risky and Expensive approach
• Minimize risk and expenditure by market testing only those concepts
that pass a set of initial thought tests.
Steps
1. Lay out the generic business tests your new concept must “pass” in order to
move forward.
New idea = attractive + viable business
The value test: Customers will buy it—at a price that works
The execution test: You can create and deliver it—at a cost that works
The scale test: If you pass 1 and 2, eventually (the sooner the better) you can build a level
of volume that makes it worthwhile
The defensibility test: After you do all the work involved in steps 1 through 3, competitors
can't easily copy you
“Think valuable, doable, scalable, and defensible”
2. Lay out the specific business tests your new concept must
“pass” in order to move forward.
These relate to your firm and your particular situation.
What are the important strategic goals you are trying to
accomplish with this new concept?
What assumptions are you making about how and why this
concept accomplishes those goals?
Refer to: the design brief and the design criteria.
3. Make sure that your assumptions relating to each individual test
(value, execution, defensibility, and scalability) are as explicit as
possible.
Remember that these assumptions revolve around a set of educated guesses
you've made about the following:
Customers:
Why this concept will create superior value for them?
How much they will be willing to pay?
Whether there are enough of them to constitute a market of sufficient size?
Your organization:
How the organization will create and deliver the promised value and what capabilities it will
leverage?
Critical missing capabilities and whom you will partner with to obtain them?
Competitors:
Which competitors are likely to be affected and how they will react?
4. Determine which assumptions are most critical to the attractiveness
of your new concept.
Far more assumptions than you can feasibly test.
Highlight the handful that make or break your new idea?
Early stages of the innovation process: Value and Execution tests
Scaling and Defensibility come later
5. Identify the data needed to test the assumptions
Critical- What the data look like that would either confirm or disprove our
hypotheses?
Most managers have been taught to take the information they've got and
work with it.
Here we are identifying the information needed and then figuring out how to
get it.
Practice and patience—and a team with a range of perspectives.
“Designated doubters”- They are adept at looking for flaws in logic
How to get the doubters on your team & share their concerns in a productive
way?
6. Sort the data needed into one of the following three
categories:
what we know?
what we don't know and can’t?
what we don't know but could?
What we know?
• These are the facts related to each assumption that are already in
your possession.
• Beware of beliefs masquerading as facts.
• The doubters will help-
• Personal (and sometimes optimistic) interpretations Vs Realities
that need to be acknowledged.
Example- You know that your parent company has a great capability set
for doing financial transactions online and a lot of high-net-worth
customers.
What we don't know and can’t?
• It is the land of true uncertainty, the land of the unknowable.
• No amount of experiments—either thought or real—can resolve this
uncertainty.
• The only thing we can do here is predict.
• Example- Larger macro issues that matter to your new concept's
success—issues like housing starts and prime rate levels. You could
make some predictions here, but you aren't sure how to test these,
except against “expert” opinions
What you don't know but could?
• In any situation, there is a lot of stuff that is knowable, but
you haven't yet taken the time to go and get the data.
• Can be an expensive proposition, and you don't want to
chase data that you don't need.
• That is why it is critical to take a hypothesis-driven approach,
identifying only the really important data and then spending
the effort to get them.
7. Figure out how you could quickly get data in category 3 (what you
don't know but could) that lends itself to thought experiments.
Fun begins: Construct some data, which means not relying on what your
internal accounting systems or industry trade group chooses to give you
Talk to customers
Check the skill-sets available
8. Design your thought experiment, paying special attention to the data
that could prove you wrong.
How many times are we going to repeat this? No matter how often we do, it
won't be enough.
“There is nothing you can do to reduce the risk of growth that is more powerful
than paying close attention to signals that your assumptions may be wrong”
• Assumptions are about how customers (and partners in your value chain) will
behave
• So the more explicit – Better. “red flags” list
• Thought Experiments Key Assumptions Prototype Customers
= What Works
