"What if we just turn off our ads?"

How to tell if your ads are actually working so you can find and cut your least efficient spend

A quick primer on incrementality testing

“Should we just turn off all our ads and see what happens?”

Every founder and CFO wonders "Does our ad spend actually bring new customers? Or are we paying to acquire customers who would have purchased anyway?" Good news… this has been tested.

In a recent study, Dropbox cut all mobile and paid search (SEM) ads in the USA, and compared the revenue impact to control markets (e.g. Canada). Results:

• They lost $8M in revenue, but saved $25M in marketing

• ~45% incrementality, meaning 55% of prospects would’ve signed up anyway

So yes, the old adage “I’m wasting half my ad budget but I don’t know which half” is pretty close – it’s actually 55% for a well-known brand like Dropbox.

Now how do you find out which half?

The problem with traditional attribution

Marketers constantly debate attribution methodologies, but the problem is that most measure correlation rather than causation, so they overstate performance. This is worse at the bottom of the funnel – trademark search and retargeting largely intercept people who already planned to buy.

The best way to establish causation is via incrementality tests, like Dropbox's holdout study, but they’re hard to run at a small scale with limited budgets and sample sizes.

What if you can’t run incrementality tests?

Don’t overthink it – if something is really working, you'll see it in your topline metrics. And if you can't see it? You don’t need more sensitive analytics, you need more impactful campaigns.

Just tell me what to do Matt

• Focus on one or two channels – if they work, you’ll see it in your topline metrics like signups and revenue. (My post on how to choose your best channel)

• Be careful with bottom-of-funnel spend – it looks efficient, but many of those prospects would've purchased anyway.

• Over-index on top-of-funnel spend. It will show lower ROAS, but you’re reaching a fresh audience who weren't already planning to buy.

• Test messages at the top of the funnel and shift spend to top performers.

• Echo winning messages in your landing pages, emails, and onboarding to boost conversion.

• Once you're spending meaningfully (e.g. 150 conversions/week), use Meta Conversion Lift to measure incrementality on paid social.

I hope this helps!