Human beings are innately talented learners when given a clear and objective assessment. (page 144)
…science is one of humanity’s most creative pursuits. I believe that applying it to entrepreneurship will unlock a vast storehouse of human potential. (page 283)
The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses made me mad, not because I thought author Eric Ries is wrong, but because he makes a lot of sense, and I wish I’d had these insights years ago. So I guess I’m mad in an inspired, actionable way.
In a nutshell: he makes the case for disruptive innovation, but only if it is rigorously measured, tested, and improved upon bit-by-bit as development happens. He calls for “small batch” design and development by cross-functional teams, borrowing from the principles of just-in-time manufacturing and applying them to new product development for software companies, new services offered by governments, or new projects by not-for-profit organizations. He prescribes a design cycle where assumptions are tested, improvements made, insights gained, and then the cycle starts again until, but-by-bit, a viable product with a proven market is built. He contends this iterative cycle applies not only to start-ups, but to established organizations as well.
“Teams that work this way are more productive as long as productivity is measured by their ability to create customer value and not just stay busy. True experiments are easy to classify as successes or failures because top-level metrics either move or they don’t. Either way, the team learns immediately whether its assumptions about how customers will behave are correct.” (Page 263)
As I read the book, I was sold on his insistence on testing and validating each assumption in the development of a new process or product. He calls it the build-measure-learn sequence. He makes the case for “building an adaptive organization, one that automatically adjusts its process and performance to current conditions.” (Page 227)
I am a fanatic about metrics. I am known at work as a bit of a pain in the ass, because whenever someone asks my department: “can you make me a poster?” or “Could I order some stickers?” or worse yet “Look at this web site we built, isn’t it cool? (and it usually is off-brand),” my response is: “We need a communication plan first; a way of measuring the success of the project.” As a strategic communicator, I went through a year-long accreditation process that included showing evidence that I have learned the most effective ways of measuring success of my communication strategies. Besides, I have a very limited budget. I can’t spend money on collateral or swag unless it shows some strategic value.
While reading The Lean Startup, I not only looked for ways the build-measure-learn sequence could help me in my work (in fact, I use A/B testing all the time in my newsletter already), but I applied it retroactively to the projects I have been involved with over my eighteen-year career.
I couldn’t help but wonder if disruptive innovation had been done the way Mr. Ries suggests, the amazing projects I’ve been part of could have done that much better, and some of the failures could have been turned around, if only for the want of testing assumptions and learning to build something better based on those experiments.
Here’s an example: I was once asked to help “raise awareness” of an online service offered by an organization I worked for (forgive my vague generalities in this example: I’m protecting the innocent). I asked my usual questions: “Who uses this service, who is it intended for?” The answer to that question was an incredibly broad subsection of the population, impossible to target meaningfully without the advertising budget of a large corporation. So I tried to drill down a little: “How do they use this service? Can they get a similar service somewhere else? What makes this service unique? Do you have some metrics? Can you show me some reports of who is using the service right now, so I can get a better idea of the target audience?”
The way the developers of this service made it sound: this was a service people could not get anywhere else, it was a great project, they “just needed to get the word out” and people would start using it. At first, it seemed like an exciting project. However, the answer to my last question pulled up short.
The developers didn’t have any reports on hand: none. Not one metric. They thought they had been collecting some data (the service had been built years ago by this point), but turns out their reporting mechanism hadn’t been properly working for many months, they couldn’t pin down exactly when, and they hadn’t even realized it until I asked. They pointed me to some web analytics, and I wasn’t surprised to see they had a very low number of unique visitors, who bounced out within a minute, at a rate of over 80 per cent, indicating people were leaving the page as soon as they landed on it.
My conclusion: “just getting the word out” wouldn’t get more people using the service. I couldn’t help them unless they themselves were interested in making the service better. Based on the web metrics they gave me, I had to conclude their service wasn’t solving some big hairy problem faced by even a small portion of their target market. They had no way of answering the most basic questions outlined in The Lean Startup:
1. Do consumers recognize that they have the problem you are trying to solve? 2. If there was a solution, would they buy it? 3. Would they buy it from us? 4. Can we build a solution for that problem? (p. 64)
This was an organization that prided itself on its commitment to disruptive innovation. They didn’t want the bureaucracy of a larger organization to cramp their style. They were pleased to be given the funding and freedom to explore cool new technologies and deliver them to an audience that henceforth would wonder what they ever did without such amazing new applications. Their passion, knowledge, and skill was never an issue. However, as Eric Ries found:
It may seem counterintuitive to think that something as disruptive, innovative, and chaotic as a startup can be managed or, to be accurate, must be managed. Most people think of process and management as boring and dull, whereas startups are dynamic and exciting. But what is actually exciting is to see startups succeed and change the world. (Page 10)
And that’s what made me mad about the above example. Maybe the developers did have an idea for a product their audience didn’t even know they needed yet. But they hadn’t learned, through a series of experiments, how that innovation might benefit their audience in the real world. Maybe more rigour, perhaps the Lean Startup method, would have brought their passion and commitment to fruition. And maybe we would now have a cool application that a lot of people use that I could point to as a great success.
On not wanting to iterate in the open, and needing to have something “perfect” before you launch, because otherwise someone else might steal your idea:
If a competitor can out-execute a startup once the idea is known, the startup is doomed anyway. The reason to build a new team to pursue an idea is that you believe you can accelerate through the Build-Measure-Learn feedback loop faster than anyone else can. If that’s true, it makes no difference what the competition knows. If it’s not true, a startup has much bigger problems, and secrecy won’t fix them. Sooner or later, a successful startup will face competition from fast followers. A head start is rarely large enough to matter, and time spent in stealth mode—away from customers—is unlikely to provide a head start. The only way to win is to learn faster than anyone else. (page 111)
On keeping the end in mind, and focusing efforts:
Remember that although we write the feedback loop as Build-Measure-Learn because the activities happen in that order, our planning really works in the reverse order: we figure out what we need to learn and then work backwards to see what product will work as an experiment to get that learning. Thus, it is not the customer, but rather our hypothesis about the customer, that pulls work from product development and other functions. Any other work is waste. (Page 201)
The Five Whys of just-in-time manufacturing (something I’ve put into practice right away in my own work):
The Five Whys ties the rate of progress to learning, not just execution. Startup teams should go through the Five Whys whenever they encounter any kind of failure, including technical faults, failures to achieve business results, or unexpected changes in customer behavior. Five Whys is a powerful organizational technique. Some of the engineers I have trained to use it believe that you can derive all the other Lean Startup techniques from the Five Whys. Coupled with working in small batches, it provides the foundation a company needs to respond quickly to problems as they appear, (page 233)
On “vanity metrics” and the traditional ways of getting support/funding from upper management:
There are no objective criteria by which a team can gauge whether it will win this coveted lottery. Teams have little confidence that they will receive any long-term ownership of their innovations. Thus, teams rarely are motivated to take real risks, instead focusing their energies on projects that are expected to win the approval of senior management …. : the use of vanity metrics instead of actionable metrics, an overly long cycle time, the use of large batch sizes, an unclear growth hypothesis, a weak experimental design, a lack of team ownership, and therefore very little learning. (Page 256-257)
On how to do “Lean Startup” even within a very large, established organization:
With an internal startup team, the sequence of accountability is the same: build an ideal model of the desired disruption that is based on customer archetypes, launch a minimum viable product to establish a baseline, and then attempt to tune the engine to get it closer to the ideal. Operating in this framework, internal teams essentially act as startups. As they demonstrate success, they need to become integrated into the company’s overall portfolio of products and services. (Page 264)
When products move from phase to phase, they are handed off between teams. Employees can choose to move with the product as part of the handoff or stay behind and begin work on something new. Neither choice is necessarily right or wrong; it depends on the temperament and skills of the person in question. (page 266)
On iterating quickly, with small, evidence-based improvements each time:
“…teams are almost guaranteed to improve as long as they get the constant feedback of small-batch development and actionable metrics and are held accountable to learning milestones. (page 268)