How Muppets and Rockets help us understand Validated learning

Early Sesame Workshop

At the heart of managing innovation activity lies a key concept - Validated Learning. It is central to the goal of reducing uncertainty in any individual product being considered. More broadly, it is central to the goal of building a sustainable pipeline of new innovations in a business portfolio. Let's unpack the meaning of validated learning here and the role it plays in building a portfolio in the next post.

Validated learnings are narrowly targeted insights gathered from a specific audience. Those insights are intended to help a team creating something new understand the likelihood of success of whatever they are creating for that audience. Individual validated learnings help inform the success or failure of a specific experiment such as how much a customer segment might be willing to pay for access to our product. Accumulated validated learnings, over the short and long term, help inform periodic funding decisions. For example, is there a sustainable and repeatable business model emerging that would justify scaling?  

Gathering these validated learnings is a highly intentional act. Teams are not simply searching amongst available data sets hoping to find answers. Teams identify critical assumptions, craft experiments to tease out validation of those assumptions, and then analyze the emerging data. This is the Build-Measure-Learn loop in action. Hypotheses are the input. Experiments and data are the output. Validated Learning is the outcome. Teams operating in this way will repeat this loop many times over the course of their journey from early idea to revenue-generating scaled business. 

The context of the hypotheses changes over time and as such the nature of the experiments and the impact of the validated learning also changes. Early on, teams are hypothesizing about whether anyone will care about their vision, do the customers they have targeted really have a problem, and is it worth solving? Validated learnings at this stage give confidence that modest investment is justified and the direction the team is headed in is worthy of support. 

Later in the lifecycle, the hypotheses shift to focus on the execution of complex technical components at scale, or whether a new market is ready for the sales team to attack. Validated learnings in these later stages give the confidence to invest large amounts to support scaling the product or extending the experience in ways that were not even conceivable when the team started their journey.   

Eric Ries coined the term Validated Learning in his 2011 book The Lean Startup with examples being centered on building products, some purely digital others in the hardware space, but the approach is not limited to digital startups or high-risk commercial enterprises and has broad applicability.  

In the late 1960s, the creators of Sesame Street attempted to build a new TV show layered with many new ideas. One of the central innovations was that bite-size chunks of learning, modeled on 30-second tv commercials and adult sketch comedy shows, would help young kids learn because they would be short and funny enough to grab their attention. This was in addition to the radical notion that presenting a multicultural, highly diverse set of human and non-human muppet characters would fly in late 1960s era America. The first experiments presented the earliest of these segments to groups of kids. The researchers had a set of assumptions about what would be funny for preschool kids. Those assumptions informed the early scripts and they were validated; kids laughed. The researchers also had a series of assumptions about interaction between human actors and the muppets commissioned for the show. The assumed they would need to be shown separately on-screen or kids would be confused. These assumptions were not validated: kids were mesmerized by muppets but tuned out when humans were shown alone. The validated learnings gave the creators the confidence to move forward with the artistic vision and gave their funders confidence to support the journey past its initial research phase. 

For a more contemporary example, we can look to earlier in June when we saw the latest successful SpaceX mission including the landing of reusable rockets on a mid-ocean floating barge. The evolution of this highly innovative approach to making spaceflight economically viable comes after many years of deliberate accumulation of validated learning by the Space X team. Their critical assumption was that they could scale a business model reliant on reusable booster rockets. They hypothesized that if they could land a booster rocket on a barge rather than having it fall into the sea, they could dramatically reduce costs. The accumulation of validated learnings (amidst many many failures) about how to execute this maneuver, and the technology required to do it gave the company confidence to continue investing in this approach. It also had the effect of increasing the confidence of their future partners NASA that this was a company they could rely on. 

When we set out to build a new business or product in an environment of high uncertainty, we have two options that treat risk very differently. We can take the high-risk path and rely on our team’s expertise, our company’s market knowledge, and even analogs from other products we admire, and charge ahead with building. This treats risk as something to manage only once we finally present our finished product to the market. In other words, we hold our breath and hope we got it right. The managed risk approach leverages hypotheses, experiments, and validated learnings to build confidence as we go and increase our chances of delivering a sustainable repeatable business. This approach treats risk as a huge challenge that we can manage by repeated exposure of our ideas to the market with validated learnings guiding the way. In other words, we release the pressure and reduce our risk by repeated experiments.

Here's the challenge though, unless validated learning is presented in the context of a wider innovation ecosystem, teams will keep running into the buzzsaw of business as usual approaches to managing risk. We’ll look at how validated learning sits at the heart of building a sustainable pipeline of new innovations, next.