Getting Closer to Human Truths with Robust Analytics and Creative Application

This article was originally published on Luminate.

Companies are continually striving to gain competitive advantage by seeking to answer questions such as:

• What would be the impact of releasing a new product or service to market? Will people buy it? Will it cannibalise other products / services?
• What’s the impact of changing or optimising a product or service that is currently in-market? Is the change worth the cost? Will it draw in new customers while minimising any loss of existing ones?

The researcher’s ability to accurately predict or anticipate future consumer behaviour is critical in addressing these types of questions. However, making predictions about the future isn’t easy. Consumers don’t know what they want tomorrow, let alone know how to articulate it in a way that is reliable for marketers and product owners.

Research provides a window into tomorrow, but what’s critical is closing the gap between what emerges from research, and the real world. For research to be of value, this gap needs to be small and understandable.

The article looks at how operating at the intersection of analytical thinking / frameworks and creativity can help us close this gap with a view to making more accurate predictions about the future. This is ultimately an important topic because if done well, it helps derisk decision making.

Watch Outs with Predicting Behaviour

Predicting behaviour is far from simple. There will always be a disconnect between what one measures, and the outcome one is seeking to predict.

Primary market research approaches typically involve conducting an experiment to understand stated intentions towards some ‘future state’. What is missed in this approach are a number of critical factors that help get closer to actual real-world impact of this future state. Common pitfalls include:

• Questions are asked about intended behaviours in a vacuum and not a real-world context.  In other words, the actual environment where choices are made is not replicated in a research study.
• Primary research findings are often not calibrated to a robust starting point. This is critical to ensure that findings are realistic.
• Primary research findings often don’t adjust for broader market factors, such as distribution or barriers to exit.

Without designing research that considers and seeks to overcome these common pitfalls, poor decisions are made about the in-market potential of a product or service.  Or worse yet, considerable dollars are invested behind a product or service, only to have it fail in-market.

How to design an experiment that reflects reality

In designing an experiment, we need to bring the real-world context to life – the commercial, knowledge and choice context (as illustrated below).  Without taking the “context” into account, we’re testing the impact of a change in a superficial way and widening the gap between research and reality.

Once we fully understand all three elements of ‘context’ we can then design an appropriate experiment. This is where the ‘creative application’ comes to the fore, and behavioural economic principles are leveraged in the design (e.g. how information is framed, how consumers are primed, how choices are presented).

An applied example – understanding purchase intent 

To understand purchase intent, one classic direct question ‘market research’ approach may be to show respondents a pack and ask their likelihood to purchase (from ‘definitely would buy’ through to ‘definitely would not buy’). What is missed here is how decisions are actually made. To mirror the real-world context, we need to show respondents a shelf with all potential brands they could buy and ask purchase intent in this context.

In designing this shelf, we need to consider retailer planograms, how brands are positioned on shelf, and the share each brand and SKU has of shelf. Consumers also need to be able to have a zoomed-in view of the product so they can read information such as the flavour, claims or pack size.  It’s about creating a version of the real world as much as possible and asking questions in this context.

An applied example – understanding take-up of a ready-made meals offer 

Nature worked with a client that was expanding its offering to the ready-made meals category. To bring to life the ready-made meal offer we created a mock-up of the website, with imagery and copy to reflect the essence of the brand and the meals that would be sold.  Respondents were primed with this information before being asked a series of questions. This was critical to accurately bring to life the offer, rather than describing it through written words or a traditional concept test.

We also needed to determine the choice context and how we presented the offer.  Thinking about the category, it was clear we needed to test this independently (given no competitive marketplace exists), but with the awareness of competitor offers.  As such, respondents were primed with screenshots of competitor websites, to ensure some level of understanding before answering a series of questions.

Example of competitor websites 

Applying analytics to arrive at more robust predictions

Beyond collecting better data (by bringing the choice environment to life), we also need to apply robust analytics to make better estimates about the future.

One important step is calibrating primary data against a known and robust starting point. While this process needs to be tailored to the situation being predicted, there are a number of considerations we need to keep in mind to ensure rigorous and robust outcomes:

• Approaches should be top-down (rather than bottom-up) to minimise any errors in calculations.
• Inputs need to be as accurate as possible (e.g. cleaning the data to remove any outliers that could skew results).
• We need to avoid ‘black box’ calculations. All predictions require assumptions, and we shouldn’t hide them. By making these explicit, we can lean into how they impact our predictions, and ensure they are optimised. We can also then flex them if our future view of the world changes.
• Estimates about the future need to take into account broader market factors.  For example, not everyone who claims in a survey that they will switch service providers actually will.  In reality there are barriers to exit, such as the time it takes to switch or exit fees.

An applied example of calibrating primary data to a robust starting point   

Nature worked with a dairy company to understand the appetite for launching a new ricotta stir-through product. Scan data showed the opportunity in the stir through category to be $34 million. However, this wasn’t an accurate “truth”.  The new ricotta stir-through sauce would only address the needs of a portion of the shelf. Further refinement was needed to ensure an accurate starting point.  Analysis at the SKU level was used to develop an addressable competitive set, which equated to $14 million or 42% of the shelf – quite a different starting point once calibration was conducted.

In the primary research experiment we captured a number of data points including purchase intent, replacement and incremental purchase behaviour. Using data from the experiment combined with the robust starting point ($14 million) we estimated sales to be $7 million.  We’ve since been able to validate these predictions once the product was launched in market.  Actual sales were within 10% of our estimates.

In summary

Predicting human behaviour is not simple.  Every challenge is different and requires a bespoke solution to predict its likely outcome or impact.  Experimental research isn’t without its flaws. But through the fusion of robust analytics and creative application we’re able to get closer to the truth and provide clients with a window into tomorrow – and ultimately, help them make better, more informed decisions.

Emma Tommasini, Partner, Nature