How Subject Line Tests Can Drive Long-Term Engagement & Loyalty

In my last contribution to the OI Blog I wrote about Alchemy Worx’ Audience Management methodology. Which requires a move away from just pulling segments, which is a reactive process, to a more proactive approach which takes behavior from all channels into account. In it I showed how subscriber engagement does not remain constant. Over a 2- or 3-year period it fluctuates between full on and no engagement on a regular basis, depending on the length of your purchase cycle  and where your customers are in the purchase cycle.

What does this mean for Subject Line testing?

The subject line is without doubt the key driver of open rates and, in my opinion, click rates. In other words, engagement. Not just with your email program, but in other channels as well. As a result, most people run subject line tests pre-rollout on a regular basis.  At its most basic, or for senders with smaller lists, a subject line test is run on a representative sample of the entire list. More sophisticated testers who have large enough lists to do so, will run tests against each segment.

The problem with this approach is it assumes that everyone within a given segment will behave in the same way and that premise is false. No matter how many segments you have, at any given time, there will always be a group of people about to move into another segment. With Audience Management where your goal is to increase overall engagement by influencing movement between segments in a proactive way, you need to dig a little deeper into your test results.  

Here is an example based on a recent subject line test on a client’s VIP Cohort. As you can see from the chart below the open rate for subject line A generated an open rate of 20.6% while the open rate for subject line B was 18.76% -- respectable, but subject line A was the clear winner.

If your focus is todays open rate and the near-term, you would roll out subject line A to the rest of the file and go home happy. However, if you looked at the test from an Audience Management perspective you would need to get an understanding of which customers opened each subject line tested.  Using numbers from a recent Alchemy Worx case study we were able to calculate the incremental value of increasing how long a subscriber stayed engaged or reducing the time they stayed in the unengaged file as being worth $6.42/subscriber/week.

By diving into the data and looking at the results at a subscriber level, you can get a very different picture.  

For simplicity’s sake I am focussing on a subject line test conducted on the VIP cohort – the most valuable customers, where the only way out of the segment is to fall into a less valuable segment. Other segments will have subscribers moving in more than one direction.

Above you will see the same test results from a subscriber level. We defined at risk as people who were within 2 weeks of falling out of the VIP cohort based on time of last engagement (in any channel). The results are clear the “losing” subject line influenced 75% more at risk subscribers to re-engage.

To maximize engagement, you need to rollout subject line A to the engaged and subject line B to those At Risk of dropping out.

If you are lucky enough to use a platform that makes this easy to do, great. if not, you might want to consider sacrificing short term open rates for long term engagement. I am not advocating you do this every time, but it is probably worth picking the “loser” every so often, if it proves to work for those at risk of becoming less engaged. Your data will tell you how often to do this.

In summary, there are three variables that impact the size of your engaged subscriber file, the most valuable in terms of loyalty and sales.

• Acquisition rate: Subscriber engagement is usually at its highest following acquisition and falls over time
• Loyalty rate: How long on average they remain engaged
• Win-back rate: How long on average they remain disengaged.

Knowing those numbers is the key to Audience Management so it’s well worth the investment it will take to get you to that point. The goal of Audience Management is to increase post-acquisition loyalty rates and improve win-back rates. The focus is on more than next week’s numbers, you also have to take engagement over time into account. Loyalty builds incrementally with each additional engagement cycle, so the more time customers are engaged and the less time they spend unengaged the more loyal they become and with that loyalty comes advocacy and a higher lifetime value.