What is the difference between MVPs and experiments and why does it matter?

by Thor Ernstsson
October 6, 2014

We’ve talked about MVPs extensively in this blog, but an often overlooked point is that they are only one component of a much larger framework. Understanding the importance of an MVP is just as important as launching one.

It’s all about user feedback

The goal of an MVP launch is not to launch the first version of a product. It’s simply to start gathering feedback and observing users actually interacting with something. This distinction is critical.

Startups are typically focused exclusively on a product and users, and can therefore launch and iterate rapidly, sacrificing all other concerns. Larger organizations are more complex, so singularly focusing on iterating live products is very difficult.

The larger the organization, the more people have a say in what must be a part of the feature set, which results in a complicated, all-too-familiar mess. But by arming product owners with the structure of an experiment, they can push back against feature-creep and design by committee. Actual user data trumps Aeron-chair quarterbacking every time.

When colleagues warn that it’s not even worth launching a product in such bad shape, remind them that it’s not a launch. The goal is to get actionable data, not hyper growth right out of the gate. Continually and rapidly launching, learning, and iterating new product ‘experiments’ delivers more user insight than any 400-page specification.

So what’s the difference between an MVP and an experiment?

A full product launch generally takes considerable resources and planning. After the planning starts and a group is set on a course, shifting that course is often an immense undertaking. If the first time the product owner hears from an actual user is after months of planning and hundreds of thousands of dollars of design and development, making appropriate changes to the product strategy requires artfully managing team members, budgets, and expectations… often leading to tremendous waste when the same feedback can be gathered from a quick product experiment at a fraction of the cost and time.

However, getting the organization onboard with thinking of new products and features as experiments is often tricky as well. The normal process of specifications and endless meetings is upended by a drive for user insights, validating (or more commonly invalidating) assumptions, and starting to think of everything in terms of testable hypotheses.

Once that transformation happens, new product development in the organization can move orders of magnitude faster.

One more consideration: Limiting exposure

In most cases, an organization cannot risk negative exposure to its brand by releasing generally available product experiments. Fortunately there are many common tactics to limit exposure of half-baked products, such as geofencing, invite-only beta lists, and non-branded products. Of course those tactics wouldn’t be ideal for real MVPs that are supposed to be “Viable Products,” but are ideal for “Minimal” experiments that are testing a particular value proposition or a new user segment.

Thor Ernstsson

Thor is the Founder and CEO of Alpha, the on-demand insights platform enabling teams to make data-driven decisions about users, products, and new markets.