How long should an incrementality test run for?

At WorkMagic we generally structure our tests to run for 3-4 weeks when conditions allow. However, the exact duration of an incrementality test is based on several factors:

• Business Size and Traffic Volume: For businesses with high traffic and conversion rates, a shorter test duration may be sufficient to achieve statistically significant results. Conversely, smaller businesses with lower traffic may require a longer testing period to gather enough data.

  
  

• Seasonality and External Factors: Avoiding peak seasons or periods with significant external influences is crucial, as these can skew results. The test should be long enough to account for typical fluctuations in consumer behavior, but not so long that it overlaps with major seasonal events.

  
  

• Statistical Significance: The test should run long enough to achieve the desired level of statistical significance. This involves collecting enough data to confidently determine whether observed effects are due to the marketing intervention rather than random chance.

  
  

• Pre- and Post-Treatment Periods: Including a cool-down period after the test can help in analyzing results more accurately by allowing any immediate effects of the intervention to stabilize.
  
  

  
  The exact duration of a test is determined based on a statistical analysis based on those factors. A data scientist or a platform like WorkMagic help determine the exact length.