What is incrementality testing?

Incrementality testing is a controlled experiment to measure the true impact a marketing campaign, channel, or other tactic, like a promotion, has on a key outcome, like sales or revenue. An incrementality test answers the question: did the marketing cause the outcome or would it have occurred anyway without it?

How Incrementality Testing Works

• Experiment Design - Incrementality testing is a controlled experiment, so there is a group that is exposed to ads, and a group that is the control which is not shown ads. The difference in outcomes between the two groups reveals the marginal contribution, also known as the incremental impact of the marketing efforts being measured. The most practical way to to create the two audiences is by creating geo-matched pairs, where individual geos (defined at DMA, ZIP code, or other level) are divided into two comparable groups based on the historic performance of each geo.

  
  

• Interpreting Results - A simple way to understand how to calculate incrementality is taking the difference in conversions between the test and control groups. For example, if there were 140 purchases in the test geos and 100 purchases in the control geos, the calculation would be: ((140 - 100) / 100) x 100 = 40% incremental lift. In practice, a more advanced equation is used, but the underlying principle is the same as the example.

  
  

• Applications - Incrementality testing can be done on any channel that allows for geography targeting, which is needed to create the exposed and control audience. It works for digital channels like Meta Ads, Google Ads, as well as traditional channels like TV and direct mail. Incrementality testing is not a fit for channels where you have less control of distribution like podcasts, organic social, or any other channel that doesn't support creating a control and exposed audience.

Conclusion

Incrementality tests are a needed part of any measurement solution as they measure the true impact of media. They ensure that advertising efforts are not just shifting demand from other channels and taking credit for it, but creating new demand that increases overall sales.