How do you conduct an incrementality test?

Conducting an incrementality test involves several key steps to accurately measure the true impact of a marketing campaign. Here’s a general outline of how to conduct an incrementality test:

1. Define Objectives and Metrics

   
   

• Set Clear Objectives: Determine what channel or tactic you want to measure incrementality. This is usually done at the channel level, like Meta Ads, or a subset tactic level, such as testing Performance Max.

• Select Key Metrics: Identify the metrics that will best reflect the impact of your campaign, such as incremental revenue, conversion rate, or return on ad spend (ROAS).\

  
  

2. Design the Experiment

   
   

• Choose the Testing Method: To run an incrementality test you need to divide your audience into two groups. The test group will be exposed to the campaign, while the control group will not. The most practical methodology to measure incrementality for ad channels is to create geo-matched pairs, where individual geos (defined at DMA, ZIP code, or other level) are divided into two comparable groups based on the historic performance of each geo.

• Set Test Length: a sufficient sample size is needed to remove the effect of randomness. At WorkMagic we general structure our tests to last 3-4 weeks.

  
  

3. Run the Test

   
   

• Implementation: Launch the campaign for the test group while ensuring the control group remains unexposed.

  
  

4. Analyze the Results and Apply Insights

   
   

• Evaluate Incremental Impact: Compare the performance of the test group against the control group to calculate the incremental lift.

• Optimize Campaigns: Optimizing based off incrementality tests alone is not always straightforward, as the results are only a snapshot in time for the channel tested. To solve this, we have created Incrementality-adjusted Attribution, a measurement methodology that combines incrementality testing with data-driven attribution to give marketers the most accurate cross-channel incrementality based measurement in real-time.

By following these steps, businesses can accurately assess the effectiveness of their marketing efforts and make better decisions to improve the return on investment (ROI) of their marketing.