Nominate your colleague to be recognized for excellence at the intersection of product and research.

Take Survey

How do you create an experiment-driven culture?

by Anuraag Verma
September 16, 2015

We all grow up hearing the age-old saying, practice makes perfect. It’s instilled in us from childhood, and as we grow older, we feel compelled to pass this wisdom onto younger generations.

There’s enough evidence – both empirical and anecdotal – to suggest that practice does indeed lead to perfection. It should therefore be evident that perfection cannot easily be achieved without practice.

But, what if you’re not afforded the opportunity, time or resources to practice? How then can you achieve perfection? Before I begin answering these questions, let me diverge for a moment and talk about boxes. Yes, boxes.


One thing I’ve noticed over the years is that people have a basic need to put things into categories, or boxes. Are you a Republican or a Democrat? Are you a vegetarian? Do you live in a house or an apartment? Everywhere we look we see our lives put into boxes. We have a religion box, a race box, a sexuality box, a politics box, and a career box, just to name a few.

If you don’t fit neatly into a box, or if you fit into multiple boxes, then many people have a hard time dealing with it. (We all have a vegetarian friend that enjoys eating the occasional McChicken, and some of us have a hard time putting them into a box.)

I’ve now talked about boxes and I’ve talked about practice. What do boxes and practice have to do with each other? And, what does this have to do with creating an experiment-driven organization?

Boxed Organizations

Organizations are full of boxes. There are Business Unit boxes. Which box do you work in? There are Role boxes. What’s level box are you? There are Process boxes. Did you adhere to this box? There are Budget boxes. Whose box is this coming out of? There are Strategy boxes. Which box strategy shall we implement? There is a box for everything. Well, almost.

We’ve discussed how practice makes perfect, and extrapolating on that logic, an organization that practices is one steps closer to achieving perfection (or at least achieving their goals). Individuals practice through a process of repetition: they try something new, make a mistake, learn from it, and then try it again, ensuring not to make the same mistake the next time.

Organizations likewise practice through a process of experimentation. Product teams, for example, often run rapid, small-scale experiments to learn about their customers, and to learn what products or features they should invest in building next.

In most organizations, however, experimentation doesn’t fit neatly into a box. There is no clear ownership or accountability for experimentation. There’s no clear budgeting process for running experiments. There’s no incentive structure in place to reward learning through experimentation. Essentially, there’s no place for practice in organizations today.

How Can Organizations Practice?

Not all hope is lost. There are many things that organizations can do to make experimentation a core activity. Essentially, there are three options for driving change:

  1. Put experimentation into an existing box;
  2. Create a new box for experimentation; or
  3. Create a ‘box-less’ organizations.

For most organizations today, the third option is more of a goal than a viable short-to-medium term strategy. So, taking that into account, let’s focus on Options 1 and 2.

Option 1 is the simplest solution, but it’s also an imperfect one. For example, organizations can push accountability for experimentation to a specific team or business unit, such as Product Development, but that doesn’t solve the problem that every team can benefit from more experimentation or practice.

Another problem is that putting experimentation into an existing box means that it is subject to existing rules and norms. Experimentation requires new mindsets, skill-sets and tool-sets, and forcing a ‘square peg in a round hole’ is the best way to doom experimentation even before it’s begun. On the other hand, this option is better than inaction, as it begins to place value on experimentation.

Option 2 – creating new boxes – offers a more sustainable alternative, however, it requires acknowledging that the existing boxes aren’t good enough. Here are some new boxes that organizations can create:

  • New role boxes (or fundamentally reshaping existing role boxes);
  • New incentive boxes (financial and non-financial) to drive experimentation;
  • New process boxes to ensure teams and individuals can easily run experiments; and
  • New tool boxes to enable experimentation.

In some industries, experimentation is nothing new. For example, in the pharmaceutical and defense industries, roles, incentives, process and tools to promote experimentation already exist, and sit neatly in the Research & Development box. However, it should be noted, even in these industries, more need to be done to ensure experimentation is organization-wide, and not isolated to an R&D box.

Creating a new boxes occurs frequently in society. There comes a point where things aren’t simply black or white, and therefore we need to create boxes for every shade in between. For example, new music genres are seemingly created everyday. In the same vein, organizations need to start evolving and creating new boxes.

So, What Can I Do Tomorrow?

Creating new boxes in your organization will, like any change, face resistance. But, that shouldn’t be used as an excuse for inaction, as not every change needs to be a large-scale transformation costing millions of dollar and taking multiple years. Here are some simple strategies that can work effectively:

  • Provide teams will a small, discretionary budget for running experiments, which doesn’t require a robust “business case” to access;
  • Link weekly performance goals or OKRs to “new learnings from experimentation,” therefore driving teams and individuals to test and learn;
  • Invest and train teams in simple and easy-to-use tools to drive experimentation; and
  • Add new terminology into your corporate lexicon, lowering the emphasis on “ success” and increasing the focus on “learning” and “iteration”.

Treat “box creation” as an experiment in itself. Start with a small box, and if that’s easily to build, then start making it bigger or start making more boxes. It’s not magic.

Anuraag Verma

As VP of Client Partnerships at Alpha, Anuraag helps organizations embed experimentation into their everyday workflow. Prior to Alpha, he partnered with senior executives at Fortune 500 companies to identify learning and development needs as Enterprise Account Director at General Assembly.