In my last article I talked about the problem of using averages in email marketing. But there is of course a more pressing measurement issue with email – attribution.

Attribution is the process of identifying and giving credit to the various touchpoints or interactions that contribute to a desired outcome, such as a sale or conversion. The goal of attribution is to understand the customer journey and determine which marketing channels, campaigns, or interactions played a role in influencing a user's decision. 

I would argue that most email marketers aren’t using accurate attribution models for email. That is not to say it’s their fault, it’s simply no perfect models seem to exist that meet the needs of email. Now don’t get your hopes up – this article doesn’t solve that dilemma, but we hope by talking around attribution we can change the way we think about measuring success.

The problem of email attribution

The 2 challenges with email attribution:

Firstly, where the attribution model under reports on the value email is delivering. Typically, this is where you are using Google Analytics or some last click model on your main campaign sends. We’ve talked many times about how an email influences conversions even without a click. In fact, we know just seeing the brand name in the inbox without even opening the email will increase conversions.

Clearly the downside here is email doesn’t get the credit it deserves, and then perhaps not the investment needed. It also means you might kill a strategy which is working incredibly successfully, it’s just not showing in your reports as such.

Of course, it works the other way where you can over report success of email. I see this a lot with triggered messaging such as cart & browse abandonment, next purchase emails and so on. These emails are sent to customers who have the highest propensity to buy in the next few days. Even if you don’t send the email many will buy.

It gets exaggerated even more if then you are using your ESPs attribution model based upon sales from customers who opened an email within a few days. Those most likely to buy anyway will also be the ones most likely to open your emails.

To give you an idea of over reporting on these emails we know of a control test on a cart abandonment programme where 50% got the emails, 50% didn’t. Those that got the emails returned at around 15.5%. Those that didn’t returned at 14.2%. The vendor was reporting on the full 15.5% when in reality the 1.3% incremental uplift didn’t cover the vendor costs.

You might wonder what the downside is here. Well, if you are over reporting you are probably spending too much time optimising something that doesn’t really deliver anything. Indeed, you can accidentally optimise it to not improve real results, but to aid the biases of the attribution model. For example, in the cart abandonment model based upon open attribution adding a 2^nd email a few days after the first will probably not create any incremental revenue uplift – but it will increase the number of opens, and if some of those go onto buy a few days after that it will look like it’s increased sales.

But if some reports are up, and some are down doesn’t it balance itself out? Probably not. But the important thing is you can’t reliably drill down into what is and isn’t working.

What are the main attribution choices for email marketers?

I know there are loads of different attribution models but here is what is commonly used by email marketers.

Google Analytics

The new GA4 attribution uses machine learning to work to weight each marketing activities contribution to a sale. Certainly, this should be an improvement over the old last click model of the old Google Analytics where revenue could only be attributed to the last marketing activity the customer interacted with.

It also benefits that all marketing channels will never add up to more than 100% of the total revenue in a period. I’ve been in trading meetings in the past where all the channels were using their own reporting tools and the total sales reported were nearly double that of what was taken as a result!

But its flaws are it still relies on clicks, but also you know why Google provide an analytics tool for free right? As much as they talk about machine learning their models are always likely to be biased towards their own paid media channels.

ESP Last Click

Most ESPs will offer a conversion & revenue tracking solution. The problem with these are it only takes into account activity from email. Often there are some generous cookie periods set. This all leads to an over reporting on the whole as 100% of sales are taken where other channels contributed.

But also, it still relies on just clicks.

ESP Open Attribution

OK only one ESP uses this. They can remain nameless. But let’s just say you can’t go anywhere on LinkedIn without seeing their optimistic attribution quote some crazy revenue figures in a case study.

I’ve touched upon the problems with it before (without even mentioning the iOS 15+ problem inflating the issue) so all I’ll say is there is a reason only one ESP uses this – because it is total utter nonsense way of measuring.

What other options do we have?

Hold out & control tests

This is my go-to strategy for nearly everything we run for clients – it’s the only way we can accurately determine incremental uplift our efforts have generated - if any! We simply keep back a percentage of randomly selected customers from the email, and afterwards compare how many of those converted versus those that receive the email. The difference between the 2 is the incremental uplift.

We might also do this to compare one email against another (or even a whole series of emails).

Going down this road isn’t as easy though it takes effort and data. You can’t run this sort of analysis on a few thousand people – you simply won’t get reliable enough numbers. And the numbers aren’t as comparable to other reporting methods as it is a one-off exercise rather than ongoing real-time numbers. And you certainly don’t want to keep people off a send just so you can measure it as that will mean you miss out on revenue from that control group.

But if you want the confidence that what you are doing is a winner, and by how much then nothing beats this approach.

Redemptions

If your campaign uses coupon codes or some mechanic that can be matched back to those sent the email, then this is an option.

I’ve always found this reports more than Google Analytics would for the same campaign, but it still has the potential challenge of customers using the codes who would have bought anyway, and customers who also were influenced by other channels.

What all attribution misses

Not all sales are equal.

I don’t mean the revenue they generate, but how hard certain sales are compared to others. For example, you could argue a first-time purchase is more valuable than a loyal customers 50^th purchase because it took more effort, and because it leads onto the potential for further spend without having to do any marketing activity to drive those sales.

Equally, selling through end of season stock or that late availability deal on a holiday is more important to a brand than the easy sales at peak times of year.

No attribution model I know takes this into account – nor do I think it ever will. But email can play a major role in delivering these key strategic wins such as converting first time shoppers to repeats, clearing key stock lines or perhaps launching new products. Therefore, email marketers need to think about how they can communicate these extra wins they can deliver for a business.

So, what on earth should email marketers do?

How do you solve a problem like attribution? You don’t!

Attribution is always going to be a challenge and email has to join in with other channels with their competing priorities. Email is the odd one out being the retention channel amongst all the acquisition channels so isn’t going to win the battle on an optimised attribution method.

Instead, I’d look to focus on what can be controlled:

• Run incrementality tests with control and hold out cells. Find out how much your various campaigns are really generating, and use that evidence to demonstrate the importance of email, but also to focus your time more wisely on areas that make the biggest difference
  
  
• Be the critical voice – while other channels are using last click or something biased, be the one that challenges whether the activity is incremental or not. Is PPC delivering you new customers or just catching your loyal customers who were going to buy anyway? If you educate others then they are more likely to come around to your way of thinking and stop the ridiculous behaviour of optimising campaigns for the attribution model
  
  
• Talk about strategic wins that only email can provide – don’t just focus on top line revenue but talk about the business-critical issues email solves. That is what will help you when it comes to justifying budgets and resource allocation.
  
  
• Fill in those team reports where all channels drop in their sales figures – but take every opportunity to talk about specific campaigns and the incremental uplift they generated. Only the email channel can prove incrementality – and that certainty will be valued over fantasy revenue numbers delivered by optimistic attribution reports.

So, while the challenges of attribution aren’t going away anytime soon getting your own measurement in top order is still in your own hands.

Photo by Will on Unsplash