top of page
  • Writer's pictureMatthew Lerner

When marketing experiments fail

What do you do when your marketing campaign isn't delivering results, or the results are modest or inconclusive?

First of all, most growth experiments should fail! (If not, test bolder stuff). But, when an experiment fails, how do you know if you’re moving forward? Or wasting precious time? Or what to do next? Here’s the thing… even if the numbers are not going up – you need to be learning. It’s tempting to brainstorm the next marketing tactic – lower the price, try carousel ads, bigger headline… Don’t. Before you start brainstorming, take a few minutes to think through what went wrong. But how? In science or engineering, when something doesn’t work, you step back through the whole thing to isolate each and check variable. Once you rule out each moving part, you get to the root cause of the issue. Good news - you can do that in marketing too! And I made this video to explain how:


Video Transcript

Alright, you try out some marketing. A Facebook campaign, landing page, email blast, whatever. And it doesn't really work.

What do you do next?

Most people at this point will try some other random marketing tactics. That's a big mistake, because this is actually not just a marketing tactic that didn't work.

This is a fundamental moment in the history of your start up. I know that sounds a little dramatic bear with me.

So my name is Matt Lerner and I run Startup Core Strengths, and we've worked with over 100 different startups running growth experiments to help them figure out their playbook and unlock growth for the startup.

And with every startup, they run into this problem time and again where the marketing thing they thought would work didn't.

So let me explain how we coach them through this situation because it is actually super important.

Okay, back to the scenario.


You've got an idea, you run some Facebook ads, landing page, emails, blast, AdWords, whatever, AB test. It's not working. It's like a week later.

That like, hey, Matt, you know, the working.

Is this statistically significant?

How long do we leave it running?

And I'm like how many people have seen it?

Well, we're running for a week, 5000 people, but it's not statistically significant yet. According to our little Optimizely or whatever I said, don't worry about that.

You're looking for things that have a big impact. If you've been running for a week, you've shown it to 5000 people and you don't have any noticeable significant results.

You don't have, like, lots of new customers or new leads or something is not working.

So then what do you do next?

And this is the point where most people are like, okay, let's try carousel ads. Let's try using video in the ads. Let's make the button bigger. Let's try a bolder headline. Let's declutter the page. Let's offer a discount.

So what you get is people brainstorm a list of random marketing tactics.

This is a problem for two reasons.

First of all, it's a problem because any random marketing tactic is only going to make an incremental difference. Like switching from to video ads is not going to be the secret magic sauce that makes your startup a unicorn.

Guaranteed, that's not it.

And the second thing is, if a campaign didn't work, it didn't work for a reason. And until you understand that reason, you're not really going to be able to come up with campaigns that do work and guessing other random tactics isn't going to get you any closer to that reason.

So what do you do next?

Well, if you were a scientist or an engineer and you had some big thing you spent weeks doing and it didn't work, you break the problem into parts and you try to isolate each variable and work through them one at a time.

So let's do that for a marketing campaign.

So what are the variables?

First, the audience are you showing this to the right people who could even be a prospect for your business?

Second is attention.

Okay, are you actually literally even able to engage them? Get their attention?

Is the thumbs stopping scrolling through the feed or whatever?

Are they noticing your email in their inbox third is going to be comprehension, right?

People got their attention. Do they understand the message at all?

Next is resonance. Okay, they understand the message. Do they care if you show them something compelling and exciting enough to get them to take action?

And then the last piece is the action you want them to take. Is it a bridge too far?

They've got a little bit of motivation. So in the beginning, you can only ask for a certain amount of stuff from them, whether it's like register or install your app or what is the next step going to be low enough that the motivation they have from your ad is going to get them over anyways.

Those are the variables. Now let's talk about how to test these different things experimentally, how to isolate each of them and sort of take them out of the equation as a moving part.

So the audience for start that's the main reason things might not work is you're just not hitting the right audience.

So one way to do that is to take all to just go with an audience as guarantee to be right.

If you've already got a list and it's qualified list, you can try emailing that list.

You can try doing a lookalike audience off of that list. If you don't have a list, one thing you can do is just to buy some very high intent search traffic.

So if you're selling exam prep for accountancy board exams and you buy terms like Best Test prep for accountancy board exams, the traffic who clicks on those ads are going to be qualified.

No one's going to click that ad for fun or grins or entertainment value.

So it's going to be very expensive traffic. It's not going to be the basis for how you grow your business, but it will at least for the sake of testing, let you remove the audience quality variable from the experiment.

Isolate that.

So next thing, are you getting their attention?

So first of all, there are lots of designers and writers who know how to do good direct response, creative email, subject lines, headlines,

Facebook ads.

So work with people who have done this before. But even if you don't, what I like to do is just 3 seconds.

Test the ad with people. And by that I mean just kind of go up to people.

Can I get your help for a minute? Show them your phone count to three. Don't warn them and then take it away and say, hey, what did you just see and just catch?

Like, were they able to visually parse the ad of the landing page at all? Are they able to identify and read the headline to you? Which words did they actually read?

So then once you've got their attention, you need to make sure that you're actually your message is going to resonate with those people.

So what I like to do, there is a five second test. So I take the message just on a piece of paper or on a blank slide.

Just the words, the headline, whatever headline and sub head, go it to someone same thing,

show to them for 5 seconds, take it away and then say, okay, do you see what that?What does that say?

And then they'll repeat it back to you. And then the next thing is, okay.

What do you think that means? Are they able to then explain their own words, what it means, just literally, do they even understand what the headline is?

We're not testing resonance yet, or they care, just comprehension.

So once you isolate each of these variables, resonant is actually super important.

This is like fundamentally the thing you're offering.

You've got the right audience you're talking about in the right way.

Do they actually care about it? Is this enticing for them or not?

Now, I don't know a simple way to isolate that in advertising, but that is fundamentally like the core of your product team's job.

So have you done some mom test interviews? Have you validated the need?

Have you validated the proposition?

You know which pieces people are looking for when they're out trying to solve this problem, which features or benefits or challenges you talk about first.

So really need to lock down that motivation. That's a much harder one.

And then what's the next step?

That motivation is going to get them moving forward a little bit, but not enough to, like, install your product, pay a $1,000 a month and go to a training course on how to use it.

So what's the right next step?

And it could be as simple as like, if all your ad traffic is on mobile, does your landing page work on mobile?

You're asking for too many things.

Can you just get an email upfront?

So I usually try if you think there's too much friction, just try a super simple form or a really simple next step, lower the friction as much as you can.

Just take that variable out of the equation.

So as I said, this is this is how you would think through and test and a campaign that didn't work.

But this is much more than that.

Right in the beginning, I said, this is fundamental to your business, and the reason is that each of those variables I just walk through.

Those aren't just the variables in an ad campaign. Those are the fundamental assumptions you have to get right to build a successful business.

So literally, like, there has to be an audience.

You got to figure out who they are, what they're struggling to do, where they're looking for solutions what they imagine the solution looks like since they haven't ever heard of your startup yet, they think they're looking for something else they're using.

Certain words are going certain places that's going to help you define your message, your channel, who you partner with, what your SEO strategy is, all that stuff.

And then, of course, once you've got the right words, you then need to validate.

You've got a compelling proposition, and then that onboarding journey has to get them to the "Aha Moment" quickly.

So what is the right next step?

So each of these things you're bottoming out isn't just make our ad campaign work.

These are the things you've got to figure out to build a successful business.

Which is why every time you do a marketing test, you're not just trying to optimize your return on ad spend or lower your CPA, or get a few customers to make the lines look right for your next fundraise.

You're actually fundamentally learning.

How are we going to grow this business?

And that learns thing is super important. Which is why every time you run an experiment that doesn't work, take the time to figure out why it didn't work.

Because experiments failing is no fun.

It feels like you're not making any progress. But if each experiment you're learning, you're getting a little bit smarter than you are making progress and the fundamental journey of your startup.

Anyways, I hope this is helpful.

If you did find it helpful, I publish a two minute email every Tuesday morning and you can read all the past issues, and you can sign up as well.

Subscribe at

Thank you very much.

1,395 views0 comments

Recent Posts

See All

Upwork is 1,000 AI startups waiting to be built.

(Steal this idea, if it works, give me 15%) Thanks to AI, the cost and speed of spinning up a SaaS startup has collapsed by 90%, but you still have to find a validated market need. If only there were

Sort your 2024 priorities with these 2 questions

Imagine if everyone in your startup consolidated their 2024 goals into a single list. How long would it be? If it’s longer than 3 items, keep reading. There’s a pattern in the early days of any great

Advanced B2B churn rate benchmarks for nerds 🤓

If I told you a B2B business had 90% customer churn, would you invest? What if I told you they gross $28B and have a 10-year CAGR of 16%? So how can such a strong business have 90% customer churn? Her


bottom of page