The Challenges of Incrementality Testing and Growth

As the buyer journey becomes more and more complex, the need to accurately measure the impact that your ads have on consumer behavior and ROI is becoming increasingly important.

Incrementality testing is one way to empower SaaS marketers to gain a better understanding of the real value of their advertising efforts.

Measuring ad performance by incrementality, however, doesn’t go without its challenges. Let’s take a closer look at the fallacy of incrementality and the unintended consequences of running incremental tests.

But First, How Does Incrementality Work?

In the context of digital marketing, incrementality is used to measure ad effectiveness in order to help optimize a brand’s advertising tactics, as well as improve (future) campaign performance and ROI.

To start running incrementality tests you will need to split an audience and only show your ads to one ‘test’ group. At the same time, another ‘control’ group will not be able to see the ads but will be exposed to all other marketing efforts instead.

What Is the Fallacy of Incrementality and Why It Matters

When managing paid search campaigns, it’s important to measure return on ad spend (ROAS) for each individual campaign. In addition, you need to calculate the customer lifetime value (the recurring revenue from the subscription service) and measure it against the customer acquisition costs per campaign.

You should not be paying for acquiring the individual users any more than what their predicted lifetime value (LTV) is. This means that you are prioritizing profitability on a per-user level, and above all else.

The fallacy of incrementality often throws marketers for a loop because it puts a stop to spending and growing for incremental users that – from LTV projected standpoint – are no longer profitable.

Generally speaking, companies in a very mature state where growth is no longer a priority, might be able to prioritize profitability. In all other cases, though, prioritizing profitability per user over growth is likely to compromise your chances of being a top industry player in the future.

Naturally, most SaaS companies are out to grow. Regardless of whether you’re creating a new category or simply competing for greater market share in your current one, growth is absolutely essential to your future evaluation and financing.

Keeping a Positive Incremental Lift Without Sacrificing Growth

For the majority of businesses in the SaaS industry, looking at user cohorts rather than trying to make predictions on a per-user basis is a tactic worth exploring.

As long as those cohorts as a whole remain profitable enough and you’re prioritizing growth, your business will be in a much better position to get a higher valuation on the next raise.A solid growth marketing strategy incorporates performance but also covers all funnel stages to scale your business. It further allows you to explore ways to expand your market up the funnel by engaging new audiences and nurturing them along the user journey.

What Are the Unintended Consequences of Incrementality Testing?

To keep things simple, let’s consider Facebook. The platform offers incrementality testing within its own Ad Manager. The goal is to help advertisers see the broader value of investing in Facebook Ads, where “last-click” conversions often happen downstream in other channels.

Although incrementality tests can pay off in the long run, it’s important to be on the lookout for unintended consequences across the Google-Facebook firewall.

At Hop Online, we’ve been running tests on the incremental impact of Facebook advertising to test the theory that you might be able to reduce your spending on Facebook without hurting the conversions reported on the platform.

What we’re seeing is that when you reduce the investment in Facebook, you inevitably see a decrease in branded search volume across other channels (measured by branded query impressions in Google Search Console). Consequently, this drags down paid search performance, too.

Essentially, you can cut down on the spend and still capture a lot of the conversions, but you need to be prepared that this might affect your campaigns across all channels.

That said, it’s worth noting that brand awareness via Facebook leads to Google searches, and Facebook ad spend is a must in order to keep the search pipeline "primed" with high volumes.

Moving Forward With Incrementality Testing

At the end of the day, incrementality testing can be a powerful tool for acquiring valuable insights and painting a more accurate picture of how each of your campaigns is performing and what value it’s driving.

Running incrementality tests in-house, however, can be complex. Handing over the marketing reins to a team of experts can make a world of difference and give you more time to spend on your brand’s core areas.

Discover the key questions for assessing a digital marketing agency and find a partner who delivers the right results for your business.

Delcho Stanimirov

Head of Paid Media

I lead the PPC and Analytics teams. My professional goals are happy clients and colleagues. Also I love numbers, charts, and data-driven decisions.