See the results of our survey of 250+ product managers on best practices in 2019


Generate Actionable Insights with Alpha

In this guide, you’ll learn how to use Alpha to generate actionable insights so that you can make informed product decisions quickly. By the end of this guide, you’ll be able to turn any assumption into a test, and any resulting data into a shareable insight.



Structure Experiments


Everything in Alpha is organized into Experiments

Think of an Experiment as a place to store all product-specific information for you and your team to reference throughout testing. You can create a new Experiment at any time or access an existing one that you’ve already set up.


Defining the Objective of an Experiment

There are a variety of reasons that your team may want to create a new Experiment. Common examples include:

  • To improve a specific business metrics like ‘revenue per customer’

  • To understand a new market like ‘connected devices in the home’

  • To test a new product concept like a ‘data compression platform’

Begin by outlining the topics or objectives you want to learn about, and then adding a detailed description. Doing so serves two purposes: it keeps everyone on your team on the same page, and provides necessary context for the Tests that will soon be executed within your Experiment.

Define Your Audience

Set up the audiences you wish to test against in the “Audiences” section of your experiment. Enter a label (required) to identify the audience when creating a test, as well as a description (optional) if further details are necessary to communicate the type of user you’re looking for.



The Alpha platform is integrated with 400+ publishers and panel vendors, so that nearly any audience is addressable. If Alpha anticipates a problem reaching your audiences, you will be notified asap.

As best practice, Alpha recommends only putting conditions on your audience that are necessary and sufficient for the purpose of your tests. Each additional condition placed on your audience (i.e. income condition, location condition, job title condition) may slow down testing time.



Launch Tests


Think of a Test as a single question or assumption that your team needs to answer in order to make a decision. You can create new Tests at any time, and they will be added to your Testing Queue. Each Test takes between one to three days to execute on Alpha’s platform. While you can submit unlimited Tests, only a certain number will be executed at a time. Contact your administrator or customer success manager to understand how many Tests your workspace can run concurrently.

Understand the available types of Tests you can run in Alpha

There are six types of Tests that you can run on Alpha. Select the appropriate one based on the objective that you have when testing:

  1. User Discovery: To understand user current behaviors opinions, needs, and pain points
  2. Competitive Overview: To gauge user perceptions of competitor offerings to assess your position in the marketplace
  3. Concept: To gauge user reactions to a new high-level product concept or offering
  4. Comparison: To compare multiple items (e.g. features descriptions, messaging, screens, prototypes, etc.) to see which performs the best
  5. UX Refinement: To gauge reactions to individual page elements or designs to improve the user experience of a product
  6. Usability: To get data on how easy or difficult a product is to use and determine areas of confusion

As you create a Test, you can include any pertinent testing information (such as branding limitations, further information about your target audience, etc.)  in the ‘Additional info’ section. You can also attach any existing design assets that you already have so they can be used for testing.

Test Iteratively!

Alpha takes the heavy lifting of research out of your hands so that you can focus on prioritizing assumptions and driving smarter product decisions faster. You do not need to submit requests for intensive surveys into Alpha. In fact, we recommend that you follow a simple rule of thumb: submitting a new Test into Alpha should not take more than 90 seconds!

Remember, Tests on Alpha run within days, not months. You’ll have plenty of time to iterate or build onto past Tests, so don’t worry if it’s not perfect the first time around.



Review Results


After Alpha executes your Test, you will be notified that data and insights are ready for you to review.

Depending on the test, there may be a prototype created and used within the testing environment in order to illustrate your product more clearly to users (learn more about Alpha’s prototyping capabilities here!).

The Test results will include mixture of quantitative and/or qualitative data.  The quantitative data typically employs more breadth of user feedback by way of larger sample-size surveys, while the qualitative data employs more depth of user feedback by way of unmoderated user videos and/or moderated interviews (including full transcriptions!).

Alpha determines which type of data to use based on which will best answer your test question. For example,  if the question is more user discovery focused (i.e. ‘How do users feel about virtual reality gaming?’), a series of focused survey questions would effectively answer that question.  Alternatively, if the question is more usability focused (i.e. ‘How easy or difficult is the checkout flow on my website?’)  – unmoderated user video data would effectively answer that question (so you can actually see how users interact with your product!).

You can cycle through data in the full details of the Test, and highlight any individual chart or asset data point that you believe to be particularly insightful. Highlighting a data point (by clicking the ‘star’ icon) will make that data point appear in your shareable report.



Share Insights


Finally, share your findings with your team!

Once you’ve run your Tests, you can aggregate all noteworthy data points into a shareable report and present that information to your team.

If you’d like to create a report that includes multiple tests from an experiment, you can create a report from the Experiment view (by clicking ‘Reports’ and ‘Create New Report’). If you’d just like to share the results of one test, you can create a report from the Test view (by clicking ‘Share this Test’).

From the Experiment view:

From the Test view:

You will be able to see all the data from your test(s) within the report.  From the edit mode of the report, you can choose which data points to include (or not include) by clicking the checkbox in the upper right corner of the data asset.

When you are ready, click ‘Publish’ at the top right of the screen and share the link with your colleagues. Armed with insights from your target audience, you can begin crafting your product’s narrative and translating that information into brilliant product decisions!

Request more information

See how Alpha can enable your organization to be customer-centric at lightspeed.

Or run a free experiment