Jeanne Jennings: This week’s discussion is centered around last week’s case study on email performance, lists and staffing. If you haven’t had a chance to read it yet you can find it here: Interactive Case Study: Email Performance, Lists and Staffing.
We raised a lot of questions about next steps in the post itself, and we’ll get to all of those over the course of the week, but to start with I wanted to see what ‘jumped out’ at each of you as you read the case study? Were there data points or qualitative details that had light bulbs (or sirens!) going off in your head? Did one particular piece of ‘low hanging fruit’ catch your eye?
Inquiring minds want to know…
Looking forward to our week discussing this!
The thing that stuck out to me was the last option the exec team was considering for staffing:
"Decentralizing the email process and eliminating the email marketing specialist position. In this scenario, the marketing managers would take turns getting the weekly email out and split up other responsibilities."
The fact that this is even an option is concerning. Considering their other concerns about deliverability, lack of A/B testing, etc, I'm not sure how they think this would lead to a more efficient and effective email program. Maybe it's worked at some companies. I just don't see how it's a sustainable option. It sounds like they think email marketing is a part-time gig.
Jeanne Jennings:
Yes! The people at the top are trying to make some decisions about email but they don’t really understand what it takes from a staffing perspective. They also don’t understand how this option would hurt, not help, meeting the other needs they have expressed for the channel.
The more customized email that they're so proud of generated about 11% less revenue than the less customized email.
Jeanne Jennings:
For those following along, Bob is referring to the customized version of the weekly newsletter where they include toys and games targeted specifically for the ages and genders of the children each subscriber shops for.
It takes a lot more time to do this… and when you look at the average revenue for each email sent (RPE) this version is generating just $0.52, versus $0.58 for the non-customized version.
I love case studies, and this one is really well written. Thank you Jeanne for taking the time to pull this together.
That said, I completely (across two reads) missed the part Bob is calling out. Weird. I will say that as a parent who has just been through all of this, I have consistently found toy guides by age range to be among the most frustrating and least useful guides across toy sites. I'd scrap it and pick another way to slice the data - customization works when it is done well, and age range is a miserable way to start trying to figure out what toy might be appropriate for a kid. Of course, beyond the numbers Bob noted, I only have anecdotal data. I would love to hear more.
Jeanne Jennings:
Thanks Gretchen!
So we have one vote from Gretchen to scratch the customized version of the newsletter and just send everyone the general version which includes (from the case study):
“a curated assortment of toys and games; the content team makes sure that toys and games for all the major age groups are included in each issue.”
Does anyone else agree? Does anyone disagree?
And, by the way… did anyone else miss these RPE numbers when they read the case study? If so, perhaps take a minute or two now to read it again. There are other ‘Easter eggs’ like this in there – low hanging fruit that should be easy fodder for discussion.
Gretchen Scheiman:
I wouldn't scratch customized, just that version of customized. I still think there is a win, just not in the version of customized that they are pursuing now.
Oooh - easter eggs. I'll take another look when I have a minute.
Bob Frady:
I think there's a bit of a danger in making department-altering decisions based on a single data point. So - while that revenue per email is a telling statistic - I'd test and learn as many possibilities as possible.
That said, if you're scrambling for resources then it might be wise to lessen your high-needs efforts (like the customized email.)
I'd also introduce a second email. Maybe "new item of the week" or "Short supply" or something different than the standard. I would almost guarantee that it would increase revenues far more than tweaking the current creative. If the brand promise of this toy company is developing and selling cool toys, then develop an email that emphasizes that fact. If you don't like the idea of a second commerce-based email, then use the email to draw your customers closer to the brand.
So looking at this overall and addressing the global challenges
- There is a fundamental business problem if new sign ups is falling vs growing in Q4 for a toy company. Site traffic must be down significantly or the sign up is not firing from all locations on the site.
- Open Rate should start dropping in the run up to Halloween and through the Holiday season with significantly greater sends with deeper into the list and more frequently. if this is not happening there is a fundamental problem with the email programs and whoever is directing it.
- In a business like this there are 4 buyer types - they should be identified, segmented out and different strategies tested to them. part of this can be determined by the 4 record types existing and the rest can be through buying patterns and future surveys
- Parents - this would be the most valuable buyer type since they would repeat regularly and purchase for multiple children.
- Older kids - it is not clear what type of toys they sell and to what audience but kids aged 12 and older can be marketed to if they are purchasing on their own with a parents credit card
- Grandparents - Repeat multi buyers for multiple grand kids
- One time gift buyers - least valuable
- The full list should be mailed no less than 10 times per year otherwise there could ultimately be delivery challenges.
- Not looking at delivery for 2 months is cause for firing - a simple review of open rate trends by ISP by segment for like mailings will show if there is a problem. This takes less than an hour and can identify a problem that is reducing revenue by more than the salary of the person running the email department
- yes they should create a buyer profile page along with a reminder section baaed on dates as well as age and gender of the child. there is 3rd party apps that make this easy and inexpensive
- How many cart abandons are there - there should be a minimum of 3 - if not they need to add 2 more. Product recommendations are tricky here and may not add value since there are multiple end recipients for each buyer. I may buy a doll for my daughter and a toy truck for my son - although it’s worth testing
- Yes add search, product, category and default browse - at least one each
- send 5 x per week in season - and 2x per day black friday and cyber monday. sending once a week is suicide when everyone else vying for those $ is in the inbox daily
- Frequency testing can identify how often to send but this needs to be done by audience category. there is no set answer to how many times to send until you test
- The 2 most valuable tactics to list growth are missing
- Refer a Friend
- Social media
- Hiring the right Agency should cost less than internal resources not more. A full service agency has division of labor and can bring down the overall cost. The right agency will also be testing as a part of the overall program and deliver significant growth. (we even offer a model where we only get paid on beating the current performance trend ).
- Some agencies (like SellUP) offer a free analysis no strings attached - so there is no reason not to get the analysis
- But with or without an agency they need to TEST TEST TEST - everything discussed above can be tested into and this business would grow by 50% at least in a year.
they can also run a contest although the quality of names will be poor - mostly kids will sign up
My quick thoughts on the below:
Here’s the short list of the tactics they are considering:
- Third-party list rental: add money to the budget to rent third-party lists on a monthly basis (twice-a-month in key time periods) for promotion. – For a variety of reasons, I would avoid at all costs. 99+% of third party list rentals fail to generate returns equal to the cost.
- List append: gather all the names and postal addresses in the database that don’t have an email address and have the missing data appended to the file (this was done a few years ago and grew the list significantly, but the number of records missing an email address is now much smaller). – This is your low-hanging fruit, enabling you to capture guaranteed deliverable addresses for your subscribers/customers at a fraction of the cost of acquiring new customers/subscribers.
- Partner Share: approach retail partners and offer a swap; we’ll give them email addresses they don’t have from our house file if they give us the same number of email addresses that we don’t have from their house files. – The email addresses should be better than those available through a third-party list rental, assuming your partner provides you with their working house file, not some dormant email addresses they’ve given up on, but best to avoid anyway as response will be minimal. Given the volume of emails people get these days, marketing emails that people didn’t sign up for simply get ignored or, worse yet, result in spam complaints, which will hurt the rest of your email program.
- Incentive: offer website visitors a $5- or $10-off coupon for signing up for email. – If the numbers work, absolutely do this. Don’t forget to perform the analysis on the cost of the incentives across all new subscribers relative to how many subscribers you would have gotten without the incentives. For example, if you could sign up 10,000 new subscribers/customers without any incentives and it cost you $60,000 ($5 in cashed incentives per person) to sign up a total of 12,000 new subscribers/customers, then it’s really costing you $60,000 for a net gain of only 2,000 new signups.
- Children: set up a system to comply with the Children’s Online Privacy and Protection Act (COPPA) and begin asking children to sign-up for email (currently sign-ups are asked for their year of birth and emails are only collected from those who are 14 or older).” – Seems fraught with risks.
Allan Levy:
The consulting firm they hired focused on the home page, adding a hero banner at the top, lengthening the page to showcase more products and incorporating blocks to hold third-party display advertising (an additional revenue stream for OST). They also moved some of the previously above-the-fold items to the bottom, just above the footer. This included the money-back guarantee, the ‘sign-up for email’ box and the social media links. Here’s what the email sign-up looks like today:
but that would not explain the drop off in November and December when they should have had a 300% growth in site traffic and orders. They may be down over the prior year but would not be down over July. I noted before that it’s possible in the redesign there is also a technical issue an they are not properly getting all members from all sign up locations. This happens often with site re-designs.
fix the above - get it above the fold - add an interstitial ( delayed pop up) for email capture - as well as a dynamic header to capture emails. Offer whatever value proposition the user clicked through in the offer - example if they came through a Free Shipping offer in search shop that
As far as social media, search and other sources these are not - OR - choices - these are all AND choices. Marketeers today are moving towards CDP’s in order to better identify how and when to communicate with individual members. Facebook retargeting and custom audiences should be combined with email….
I guess the TEST TEST TEST comes back to establishing a test plan with projected results based on historical metrics. Here you need an agency or a consultant to identify the subject lien testing for example can deliver a revenue lift of between 10% and 30% - take the range and apply it to the offers that would see value.
If 10 content newsletters a year drive $15,000 in total annual revenue that’s only $1,500 - $4,500, if 40 promotional emails a year deliver $4 million a year in total annual revenue that’s $400,000 - $1,200,000 - that’s what you test
Start with the big stuff like subject lines that are easy to execute and have the most impact and only test them on high value campaigns then work your way down the list.
This is an interesting topic not limited to email as the martech landscape changes.
The more critical shift is to a culture of experimentation continually and be more product led.
Email teams led by data and insights when establishing experiments and testing tasks are how it should work today – and not a future thing. Already email agencies and CRO agencies are disappearing as optimisation and personalisation in web and digital are becoming customer centric and not channel or specific parts of a stack.
Most recently we (thelumery.com) did a study on over 200 experiments (big and small) across an airline customers and found that the most successful 80% were data driven and unemotional and less than 5% were ad hoc or opinion based. Again, not email specifically but all things customer experience.
So, when asked how much to spend on this operationally my answer would be to move the entire team to a continual data driven cadence and track the return.
Within this you can determine the time, cost and amount of tests you can both achieve and sustain. The timing on transforming is longer than people want but you can do it at the pace you like.
Hi Jeanne
To your point - subject line tests [don't] move the needle dramatically they do and I think Persado might jump in on this too as their business is built on that premise :-)
The problem is most people only A/B test, that is they test only 2 Subject Lines at a time and that isn't enough.
Research I published in a post on Only Influencers into the effect of testing on subject line performance using data contained within Touchstone - Machine Testing for Machine Learning that allows marketers to try out as many Subject Lines as they like without burning out their database - shows marketers who do not test Subject Lines at all face a 1 in 3 chance the one they use will under perform their average by at least 20% and a 1 in 10 chance that it will under perform 42% below average.
A/B testing does improve the chances of success but there is still a nearly 10% chance both will be 20% or more below average. Subject Line “A” is bad, “B” is worse – bad beats worse, but they are both crap!
More proof comes in the form of this case study by Persado Their Client Air Canada went from testing two variants to 32. When it comes to testing more is better whether you use a human copywriter or AI driven tech like Persado or even Phrassee whose recent case study with Dominos Pizza shows that they now test more than 10 subject lines at a time.
Now I accept that not everyone has the resources or enough people on their list to write and test 32 different Subject Lines every time they send an email, but we do recommend everyone test at four subject lines at a time. A four Subject Line test is likely to result in a lift of 33%, by comparison a classic A/B test is only likely to give a lift of 3%. Even if you run a 3 Subject Line test your likely open rate lift will be seven times higher. For more on this read our White Paper Zen & The Art of Subject Line Testing
But help is at hand
Email marketers and copy writers who are serious about improving the performance of their email campaigns can only do so by increasing the number of Subject Lines they test. Touchstone - using AI to identify winning subject lines without burning your list - makes the testing part easy. If your problem is writing 10 subject lines at a time and tools like Persado are out of reach for reasons of scale or budget, here is a handy guide to subject line copy writing.
Here is a link to PDF containing a HUGE collection of articles on Subject Lines and Testing written by me Team Alchemy Worx and Touchstone http://bit.ly/2B4RY2O
Every email you send opened or not delivers a brand impression.
In my view, as with TV, Press, Radio, Billboards, you name it. Therefore if a consumer or business is subscribed to the email program of 2 brands in the same space – all things being equal if brand A send 1 email a week and brand B sends 3 emails a week, you would expect brand B to wipe the floor with brand A.
The email channel is unusual if not unique among digital channels in that it ascribes zero - indeed to many people negative value (everyone hates email right?) to email impressions. This is largely because if use open and or click rates as your proxy for engagement. In fact I will go further, any KPI that includes the word rate even revenue/email - which is a rate will go down if you up your frequency (again all things being equal). You will therefore get the impression (geddit?) that when it comes to email less is more.
However if you use TOTALS (opens, clicks or $£) your KPI and you or your team do their job in terms of deliverability best practice, do everything you can using all the latest tools to delight your subscribers, every additional email you send will improve your KPI’s. Even if you don’t take the fact that unopened emails drive results in other channels into account. What we call the halo effect.
Measuring the impact of your email campaigns in other channels boils down to your attribution model and THAT is another whole kettle of fish. I am in the UK at the moment so it is 6pm Friday and I am in the office – need to go home. So if you will forgive me I will point you to stuff I have already written about the halo effect and attribution.
The Email Halo Effect: Finding Value Beyond Open and Clicks
Attribution: Calculating the Effect of Email on other Channels
Using Last-Click Attribution? You’re Hurting Email: Part 1
Using Last-Click Attribution? You’re Hurting Email: Part 2
Using Last-Click Attribution? You’re Hurting Email: Part 3
There were a number of studies done in the past (by Epsilon, SilverPop and maybe one or two others - if I remember correctly) that show a consistent churn of ~30%/year or ~2.5%/month on the average email file every year. This would included things like address abandonment, unsubscribe and complaints. With inbox sizes now being in the 10+GB abandoned accounts will not bounce as full for sometime, thus being delivered successfully but having no activity for a long period of time.The results seem to be fairy consistent over the years as I recall this being used in the mid 2000's and in the mid 2010's. In some cases it was higher, like .edu as the schools that didn't give life time access to an email account rolled out the school's email address file every semester/year (varied by school).
Holding on to these long dormant account can have long term delivery impacts, from experience I've seen many delivery issues get resolved by tightening up the targeting and excluding or greatly reducing the number of messages that these dormant accounts get with a marginal winback success of accounts that are quiet over 10-12 months. The older the account, and the longer period of inactivity, the less likely any type of winback campaign will work.
Last thought, from a legal perspective CASL requires implied consents be upgraded to express or removed at 24 months... This time period limitation actually help naturally several Canadian clients with delivery issues as they were basically forced to remove a chunk of inactive and abandoned account from their list each month. The first time was likely the most painful for many brands but the monthly roll off of 1-2.5% was more easily digested afterwards.
Bill Kaplan:
Great case study!!! And I didn’t have to pay HBS a dime to read this. Haha.
Given your low-hanging fruit below, I felt compelled to jump in. From our experience, the leading factor resulting in dormant subscribers is simply that companies are emailing these subscribers at email accounts they no longer read. 20% to 30+% of an email list churns on an annual basis. To avoid wasting money on reactivation programs that generate minimal returns, a successful reactivation program needs to incorporate an Email Change of Address (ECOA) service to ensure that your re-engagement emails reach people at their preferred email addresses, rather than landing in dead, unread email boxes.
Accordingly, I would recommend OST make one last stab at their dormant subscribers with a compelling subject line and an attractive offer. If that doesn't generate a response, OST should look to update their inactive and bouncing email addresses with an ECOA (Email Change of Address) service before removing them from their list. Doing so will enable them to then leverage their marketing spend, reconnect with lost customers, and drive the ROI and revenues they’re seeking.
As for the “the three schools of thought” below on how best to handle these dormant subscribers, each is completely flawed as follows:
- Stop sending to them altogether – OST made considerable marketing investments to acquire these subscribers/customers. Why throw away this asset when you can reconnect with them at a fraction of the cost of acquiring new subscribers/customers?
- Continue to send to them but cut back on frequency – only send during the key periods of the year (pre-Easter, pre-Summer, pre-Christmas) – Reducing frequency will only waste money on messaging costs without generating much new activity, if any. The low sending frequency will most likely result in these emails being relegated to junk folders, assuming they even get delivered at all. Finally, a basic marketing adage is that you need to send 5-7 marketing messages before you can get somebody’s attention – minimizing frequency goes against this basic marketing principle.
- Continue to send to them at the same frequency they receive email now – Given the long-standing dormant history due to sending to dead email accounts or whatever, continued sends will not achieve OST’s re-engagement goals. Furthermore, this will be a waste of money and the lack of engagement could damage OST’s sending reputation, thereby reducing the effectiveness of their overall email marketing program.
Good luck with everything! I’m sure the pointers above coupled with the recommendations of the rest of your OI Marketing Team will help OST turn its email marketing program around.
Dela Quist:
When it comes to Inactive or dormant subscribers the most important challenge for email marketers is how or when you can be sure address really is inactive rather than what I call “Unemotionally Subscribed” (more on that later). Given how often the question of what to do with “inactives comes up it is arguably THE most important question for a very simple reason: Any sane marketer would immediately stop mailing any address or person on their list who they knew for certain was unused, a spam trap, would never transact again and/or actively disliked receiving their emails.
The real and more important question is not IF you should take an inactive person off the list, but HOW do you tell or know they ARE inactive. That question however is very rarely asked! Yet almost all “best practice” documents seem to advocate indiscriminately purging your list of every subscriber who has not opened an email for 3, 6 or 12 months (note how cozily round those numbers are). Despite the consistency of this best practice advice the vast majority of email marketers continue to send to long term non-openers, despite the fact pretty much every ESP and most their deliverability experts have been saying the same thing for years.
So why do they ignore such "sensible" advice? Because it is arbitrary and wrong.
Starting with the causes here are 5 reasons why 30% - 65% of your list have been “inactive” for 12 months or more:
- They want your email, but haven’t needed your product, service or information in that time.
- False negatives – your email is optimized to be read with image blocking on and the subscriber is opening your email without you knowing (the reverse is also true because some subscribers who you think are opening are not)
- They don’t want your email, but do not care to unsubscribe, they are so disengaged or mistrustful of your brand that they prefer to delete your email or mark it as spam
- Email address churn – they no longer use or rarely check that email address
- They don’t see your email because it always goes to the junk folder
By far the largest group is those who want your email, but haven’t needed your product, service or information in that time. I call these people “unemotionally subscribed”. They are happy to ignore your messages until they are ready to buy, because it is easier than unsubscribing and having to remember your URL or Google you at a later date.
What most people never try to establish however, is which people on the list fit into each of those categories, which makes it very difficult to decide who to remove.
We have gathered plenty of evidence of this phenomenon, from our deep dives into client data and here are some examples of how we know how many subscribers are unemotionally subscribed:
- $120,000 generated by subscribers who had not opened (downloaded images) or clicked on the previous 25 to 40 emails – it was a great offer.
- 14 percent of year 2 revenue and 7 percent of year 3 revenue generated by subscribers who did not open or click at all in year 1
- The most common or modal open, click or purchase frequency across every email audit we have ever conducted is 1.
The takeaway is simple. While some of those inactive addresses may be people who really won’t transact or hate you, the vast majority are unemotionally subscribed. They need you – but not yet!
So don’t beat yourself up over the fact that they don’t feel compelled to read every email you send.
As far as I can see the main (only?) reason email marketers are advised to remove inactive addresses from any list is because they might be honey traps monitored by ISP’s. And as we have been told emailing a honey trap email address will damage a brands reputation with the ISP’s and messages will be more likely to bounce or be delivered to the junk folder.
However, as most practicing email marketers know, things are never that simple!
In the real world:
- The people on your list are either customers or have opted to receive email from you.
- You cannot assume that everyone who has not opened for any given time period definitely wants to be removed from the list.
- It is almost impossible to actually prove empirically the scale of any negative financial impact caused by sending email to dormant (as opposed to truly dead or purchased) addresses.
- The cost of continuing to send email to these people is very low.
- The revenue generated by addresses that become active after a long period of inactivity is very significant.
Faced with a choice between hard numbers such as the low cost of mailing inactives and how much revenue is generated by their email programs and the mostly theoretical cost of a bad reputation, it is hardly surprising that so many email marketers working for well-known and reputable brands ignore “best practice” advice.
Most marketers know that while you CAN say every dead address, honey trap or person who does not want your email will not open it; you CANNOT say a person who does not open your email is a dead address, honey trap or does not want to be on your list.
No one disagrees that it is a good idea to remove certifiably dead email addresses so telling them that achieves nothing. What email marketers really need help with is how to ensure that they are not culling apparently dead people who are happy to be on the list and plan to re-engage in future – the unemotionally subscribed.
The best way to approach the problem of inactives or unresponsive people on your list is to try and answer the following questions before you start a wholesale purge:
- Do you have reputation or deliverability issues?
- Are you trying to cut your costs?
- Are you trying to create the illusion of improved open and click rates by reducing send numbers?
- Are you doing it because everyone says you should?
- Have you explored other options such as sending an exceptional offer, prize draw or voucher?
- Have you followed the money?
Many of your “inactives” may be transacting via other channels – driven by the subject lines of the unopened emails you sent (See the chart below) so start by following the money. Compare revenues or site visit numbers of the inactives v never mailed.
We provide clients with these kind of insights, where we overlay the timing of emails sent with the timing of sales from other channels e.g. in-store, the website, call center, PPC and affiliates. You would be amazed at how people make purchases within hours of receiving an email they did not open in their inbox.
The answers to those questions will make a huge difference to the value you place on the unresponsive people on your list and make it easier to decide whether to remove them or not.
Bill Kaplan:
Nice overview of the inactive/dormant issue all marketers face, Dela!
I think everyone agrees there’s potential “gold in them thar hills.” The question is how to find it most cost-effectively without killing the “goose that laid the golden egg” (i.e. the rest of your email marketing program).
While the out-of-pocket costs of continually emailing dormant subscribers might be relatively small, the risks of hitting a spamtrap or exceeding ISP’s stringent spam complaint thresholds from messaging dead or recycled addresses is real, whether or not these costs can be definitively calculated.
The bottom line remains: Don’t just throw out your dormant subscribers as there are significant revenues to be generated therefrom. Rather, take another stab at them with an aggressive offer to see if you can awaken these sleeping bears. For the ones that continue to be dormant, try something different, like re-engaging them at their current, preferred email addresses through an Email Change of Address service. Per the quote most often attributed to Albert Einstein, “Insanity is doing the same thing over and over again and expecting a different result.”
Regarding your questions
I wanted to circle back to the case study and discuss dormant subscribers. Two questions:
1 – any idea what caused this 30% of the list to be dormant (there’s a possible answer in the case study)?
- have you checked your inbox deliverability?
- have you checked the mobile responsiveness of your outbound emails?
- what were the last emails they opened and the piece of content that they did engage with you?
2 – what, if anything, would you do with these dormant subscribers?
- remove those nonengaged or non-openers to a new list
- create a re-engagement series of emails
- reach out via other forms of media - send direct mail re-engagement
- consider social media re-engagement
- set up a custom audience with facebook and google, load those emails and contacts and run paid re-engagement advertising to them
- take a percentage of the non-openers list and have a third party conduct a short phone survey and ask the question - How can we serve you better with the emails we send you?
+1 on that.
I’ve never found a list that didn’t have 30% to 60% so called inactives. It’s normal and not a problem per se.
Only consider removing if you have an inbox issue and then only after trying some re-activation.
Just as Dela notes, inactive doesn’t mean zero value. The email addiction consumer research report showed this:
I also split a segment of ‘never opened’ vs ‘opened at least once’. If the ‘never opened’ are not relatively new, I’ve never seen value in them.
Removed the never opened, try win-back or otherwise just carry on with the inactive if there is not inbox issue. Some of the inactive will come back in their own time.
Tim brings up an excellent point with the never opened segment. That segment has the highest chance of belonging to people who aren’t your customers and who really don’t want your mail. This, really, is the one of the segment most of the engagement based list purging is trying to deal with. Those addresses that ended up on your list because someone other than the address owner typed them into a form. These are the people that you’re spamming, and continuing to spam them isn’t necessarily going to convince them to buy your stuff.
For those addresses that actually signed up for your mail, send as many emails as you want, as frequently as you want. But other than engagement, and in the face of companies refusing to do any sort of subscriber validation, I’ve not found any way other than engagement to remove the spam victims from lists.
I’d be interested to see a question on the next marketing survey along the lines of:
What is your reaction when you get mail from a brand you’ve heard of but never gave your email address to?