In today’s rapidly changing advertising landscape, measuring ad effectiveness is critical.
Calculating the incremental lift and the resulting value that ads provide is a great way to accurately assess the impact of your advertising tactics, as well as improve performance and ROI.
Let’s take a closer look at what incrementality is, how incrementality testing works, and how it can benefit your marketing campaigns.
In digital marketing, incrementality is the measurement of events that would not have occurred without a specific marketing activity and resulted in a lift in the desired outcome (conversion, revenue, page visits, etc.)Incrementality can be used to measure ad effectiveness and identify the interaction that moves a user from passive to active.
Incrementality testing is an increasingly popular approach for businesses to measure incremental lift and understand the true impact of their digital marketing campaigns.Incrementality can be tested by randomly segmenting audiences into two separate groups and measuring which one outperforms the other.
One group consists of users that have viewed or interacted with a particular ad in a given campaign. This is referred to as the test group.
The other group is considered the control – a group of individuals who have not been exposed to these ads. The control group is kept aside for analysis.Effectively, both groups should have similar characteristics, however, they should not overlap.
When measuring incrementality, the results of each group can help you determine which conversions would not have been possible without advertising.
Consider this: You have segmented your audience into Group A (the test group exposed to ads) and Group B (the control group without ads). Because both segments show the same behavior, running campaigns only for Group A will show you the impact of those ads on your conversion rates (CVR) versus a control group.
The percentage difference of the test group from the control group is also referred to as incremental lift. This can be calculated by using the following formula:(Test CVR - Control CVR) / (Test CVR) = % Incremental Sales LiftTaking the example above, if Group A has 120 installs and Group B has 100, the incrementality would be 16,7% ((120-100) / (120) = 0,1666, or 16,7%).
Incrementality testing is the way for advertisers to truly understand the impact of their ads and make decisions that will help improve the efficiency of their digital marketing efforts. Some of the more prominent incrementality testing benefits include:
With that in mind, it’s worth noting that while there are certain similarities between incrementality testing and traditional optimization methods, there are also some differences to keep in mind.Traditional optimization is focused on attribution-based performance and actions that indicate a direct link to a conversion. Measuring incrementality by itself is not a replacement for traditional optimization methods. Instead, it works together with attribution to help brands better measure their performance.
Running incrementality tests can be a complex process, but if you are familiar with regular A/B testing, you should have a good starting point.Incrementality can be measured using a variety of different ways. Two of the more popular ones are sequential testing and Facebook brand-lift testing.
Sequential testing does not require a vendor, making it an easily accessible testing method.With sequential testing, you turn a channel on or off for a specific amount of time and measure the impact on the overall baseline.With sequential testing you can:
Fully turning off a channel, however, could lead to potential revenue loss. In addition, this tactic doesn’t remove the noise of external factors (e.g. seasonality), which might impact the end results.
Facebook Lift Studies are set up in the Facebook platform and can be used for both Facebook and Instagram campaigns. With this method, your audience is automatically split into two groups – one that is exposed to your ads and one that isn’t.Both groups are then shown a set of poll questions that you’ve selected before launching the campaigns.The test is focused on calculating the difference in performance between the test and control group, representing your Facebook advertising lift in relation to the conditions of your test.By running a Facebook brand-lift test you can:
Nevertheless, keep in mind that this testing method is only possible on Facebook and does not acknowledge the branding effect and the impact of other channels.
Along with sequential testing and brand-lift studies, digital marketers can explore several other alternative testing methodologies. Here’s a quick overview.
Custom audience split is easy to set and can help you get the most out of remarketing, however, the audience is limited to current customers and email lists.
While it’s hard to argue against the benefits of measuring incremental lift, it’s worth mentioning that there might be certain unintended consequences of incrementality testing, as well.Here are four of the more common incrementality testing pitfalls to watch out for.
When it comes to calculating an incremental lift, deciding when to start testing is vital, and seasonality is one of the most important factors to consider.Running holiday campaigns can really pay off, but when it comes to measuring incrementality, you should avoid testing during peak seasons (which may vary by brand) and try to keep the testing timeframe as equal as possible.
When launching incrementality tests, bear in mind that your control group should be unbiased. This is why you need to try and limit external factors, such as other digital channel partners (i.e. search, social) or offline partners (i.e. TV, print, radio).In addition, if you are testing a channel that is bottom of the funnel (e.g. Google), you should avoid making any major changes on top of funnel channels (e.g. Facebook) as this would have an impact on the bottom of the funnel.Essentially, the more awareness you’re bringing, the more your results will be skewed when you’re testing Google.Along with outliers, having overlapping audiences could alter your results, as well.For example, if you’re focusing on users interested in hip-hop music, make sure that you are not testing something else which is covering those people at the same time. The more your control group is exposed to other media that you’re running, the higher the risk of data contamination.
Deciding on the size of your segments and the timeframe of your testing can be challenging.Generally speaking, it’s recommended that you allow at least a week for your incrementality testing. The exact testing duration, however, will typically depend on your business.If you have a lot of traffic and conversions, you might be able to test a smaller number of people while still having statistically significant results and an audience that is randomized enough. On the other hand, if you have a smaller business and don’t have that many conversions, your testing period should be longer.
If you are running a test on Facebook, the platform will do the calculations for you and notify you when the percentage is significant enough.
When conducting incrementality testing, it’s important to be aware that, in the short term, you might experience some decrease in results.
That said, as it shows that your ads are working and you have an impact on the results, incrementality testing can be a very beneficial framework in the long run.
Once you’ve collected all the data from your incrementality testing, it’s time to compare and identify the incremental lift in the KPIs that match your business goals.Understanding the relationship between the test and control groups will help you determine why there was a positive, negative, or neutral incremental lift.Keep in mind, though, that if there is a big gap between the two groups, this might be a red flag that there’s something wrong in the configuration of your testing.Based on the results, you can take further action and apply the insight to your campaigns to maximize impact, define profitability, reduce cannibalization, and define the best time for re-engagement.
For most SaaS companies, the results from calculating incrementality should be compared to the customer lifetime value (LTV).In order to be profitable, the LTV needs to be higher than the number you get from dividing the cost per acquisition (CPA) by the incrementality. For instance: if your incrementality is 16,7% and your CPA is $2, your LTV should be more than $12 ($2 / 16,7% = 12).
Incrementality testing is arguably one of the best ways to reduce – and even eliminate – cannibalization.The test results can be used to scale up relevant channels to stop cannibalizing organic traffic, as well as ensure your growth targets are not rewarding cannibalization.
When it comes to re-engaging, the data gathered from the incrementality testing can help you identify the optimal post-install time to re-engage users and ensure high incremental lifts from your marketing efforts.
The results of each incrementality test can help marketers identify how much business value is driven by a specific tactic. This, in turn, plays a crucial role in determining which media investments are most effective, allowing you to shift budgets to improve future ad performance.
Measuring incrementality is a great opportunity to begin to understand the positive, negative, or neutral impact an ad has on your business. Executed well, incrementality testing is a powerful tool that can give you valuable insights and ensure that you’re making good use of your time and marketing budget.
If not done right, however, incrementality can hide risks and hinder your growth. Head to our blog and learn about the challenges of incrementality testing and how to tackle them.
I lead the PPC and Analytics teams. My professional goals are happy clients and colleagues. Also I love numbers, charts, and data-driven decisions.
