How do you analyze results from an incrementality test?

Here are the steps for analyzing the results from an incrementality test:

Collect and Aggregate Data

• Data Collection: Gather data from both the test group (exposed to the campaign) and the control group (not exposed). This data should include the key performance indicators (KPIs) defined at the beginning of the test, such as sales, conversions, or website traffic.

Compare Test and Control Groups

• Identify Incremental Lift: Compare the outcomes between the test and control groups to determine the incremental lift. This involves calculating the difference in performance metrics, such as conversion rates or revenue, between the two groups.

• Statistical Significance: Ensure that the observed differences are statistically significant, indicating that the results are not due to random chance.

Calculate Key Metrics

• Incremental Revenue: Determine the additional revenue generated by the campaign by calculating the difference in revenue between the test and control groups.

• Return on Ad Spend (ROAS): Calculate the incremental ROAS by dividing the incremental revenue by the campaign's media spend. This helps in assessing the efficiency of the ad spend.

Interpret Results

• Positive, Neutral, or Negative Lift: Determine whether the campaign resulted in a positive, neutral, or negative incremental lift. A positive lift indicates successful impact, while a neutral or negative lift suggests the campaign had little or no effect.

• Identify Anomalies: Look for any anomalies or unexpected results that could indicate issues with the test setup or external factors influencing the results.

Take Action Based on Insights

• Optimize Campaigns: Use the insights gained to optimize future campaigns. This could involve adjusting budget allocations, refining targeting strategies, or changing messaging to improve effectiveness.

• Strategic Decisions: Apply the findings to make informed strategic decisions about channel selection and marketing investments to maximize return on investment.

  
  

At WorkMagic we automate this whole process and apply your incrementality test results to your brand specific incrementality based attribution model to give marketers the most accurate cross-channel incrementality based measurement in real-time.