Introducing: Run tests in your sleep with Insight Trackers

by Claire Gerson
May 1, 2018

Great product teams understand that consumer preferences are rapidly changing. Building successful products means chasing a moving target: what was true yesterday may not be true tomorrow. There are a number of interconnected forces at play – some internal (e.g. positive media coverage) and some external (e.g. interest rates) to your company. Whether it’s cord cutting or telemedicine, teams in every industry need to track key shifts in user behaviors to stay relevant in real-time.

To that end, we’re very excited to announce the addition of ‘Insight Trackers’ to Alpha’s integrated experimentation platform. This new feature enables teams to stay in-the-know and be responsive when things change, even in their sleep. Simply specify a target and the platform will run tests over a recurring cadence to track insights related to that target. You can focus on your day-to-day job while keeping an eye out for actionable indicators.

For example, we’re tracking how consumer perceptions of self-driving cars are influenced by major news stories.

I annotated how two major events correlated with drops in respondents’ willingness to ride in self-driving cars. Of course, we also saw how quickly they’ve rebounded!

Jeff Bezos recently summed up why every organization needs this capability:

One thing I love about customers is that they are divinely discontent. Their expectations are never static — they go up. It’s human nature. We didn’t ascend from our hunter-gatherer days by being satisfied. People have a voracious appetite for a better way, and yesterday’s ‘wow’ quickly becomes today’s ‘ordinary’. I see that cycle of improvement happening at a faster rate than ever before. It may be because customers have such easy access to more information than ever before — in only a few seconds and with a couple taps on their phones, customers can read reviews, compare prices from multiple retailers, see whether something’s in stock, find out how fast it will ship or be available for pick-up, and more. These examples are from retail, but I sense that the same customer empowerment phenomenon is happening broadly across everything we do at Amazon and most other industries as well. You cannot rest on your laurels in this world. Customers won’t have it.

There are numerous possibilities when it comes to Insight Trackers, but five should be a part of any product team’s repertoire:

  • Usability Tracker: Generate insights about how usable your product is over time. Identify whether or not all the features and improvements you’re shipping are facilitating or inhibiting ease-of-use.
  • Perception of Value Tracker: Generate insights about how valuable your product is perceived to be over time. Identify the impact of major releases on user perceptions.
  • Competition Tracker: Generate insights about how your brand and product is perceived in comparison to the competition. Identify how users are reacting to your product updates compared to updates from competitors.
  • Pain Point Tracker: Generate insights about how consumers prioritize problems over time. Identify tipping points to determine when a commercial opportunity might present itself.
  • Emerging Technology Tracker: Generate insights about what new innovations are gaining the attention of your target audience. Identify which flashy new tools on Product Hunt may be your next competition or acquisition.

In just the last few months, we’ve already seen how powerful Insight Tracking is for experiment-driven organizations. A Fortune 100 financial services company was considering implementing facial recognition for customer logins. Recurring tests showed that consumers were continually suspect of such technology, so the company withheld investing in the capability. Everything changed in September 2017 when Apple announced that the new iPhone X would ship with facial recognition as the default unlocking mechanism. Basically overnight, consumer perceptions did a 180 – significantly more people positively viewed the ability to login with their face. Quick to identify this insight, the financial services company was one of the first to add the technology to their digital portfolio.

I suspect that Insight Tracking can enable organizations to time their investments in fast-paced environments, such as those related to consumer privacy, blockchain-enabled marketplaces, and AI-powered messengers. Ultimately, product management is about placing bets and making data-driven decisions about how to best deliver value to customers. Insight Tracking is a powerful weapon in your arsenal to accomplish both seamlessly.

Claire Gerson

Claire Gerson is a product manager at Alpha. She is responsible for the platform’s quantitative analytics solutions.