Back in the 2000s everyone used to ask ESPs what their email delivery rate was. It was a meaningless number but one ESP would quote 98% delivery rates, the next would just claim 99%. Now you get the same questions around 'is your platform AI driven?' and ESPs have added AI functionality of varying quality and usefulness.
I want those with limited experience with AI to understand exactly what it is, and for you to be able to critically evaluate different AI options and technology, and which ones are right for your business.
Let's start with where we seem to be with AI today.
This is an (extreme) analogy of where I think AI for email is today in many marketers heads.
Fans of ‘The Office’ might remember an episode where Michael Scott first gets to try satellite navigation in his rental car, which of course he misunderstands and ends up driving straight into a lake. Despite the lake appearing right in front of him he refuses to believe the machine is wrong. Everyone gets wet.
This is where I think we are right now, a lack of critical thinking of AI - “The machine knows”.
AI has developed a mythical reputation for being this magical force that knows best. It can make better decisions than you, it can predict things a human can't do. Based upon your purchase behaviour it can work out if you are pregnant.
With AI being new to most email marketers they are not in a position of experience or confidence in critiquing AI, and in working out which AI solutions are right for them.
It has the potential to do many of these things but let's first look at what AI is.
What is Artificial Intelligence?
One of the challenges of defining Artificial Intelligence is the sheer number of different definitions. In reality AI is a broad umbrella term for any technique that enables mimicking of human behaviour by machines.
Within this you also have terms like ‘Machine Learning’ and ‘Deep Learning’. These aren’t different to AI, they are forms of AI.
https://www.linkedin.com/pulse/artificial-intelligence-vs-machine-learning-deep-mateis-ph-d-/
If for one moment we focus on the definition of AI in the above diagram we can see it can be as simple as if-then rules or decision trees. It doesn’t have to be as mythical as you might see in a Hollywood movie about robots that have developed their own thoughts and feelings.
And this is where the majority of AI is today. Us humans have told the computer how to work out the answers to problems. The benefit isn’t that us humans can’t get to the answer, but the computer can run near infinite calculations, on huge data sets at lightning speed and predictable accuracy.
This is the really crucial point for marketers to understand about the limiting factors of AI in marketing applications.
It’s only as good as:
- The breadth and volume of data it uses
- The quality of the algorithm written by the humans
What about Machine Learning?
The difference between machine learning and the very basic AI is that basic AI will never improve the accuracy, whereas machine learning uses outcomes that have occurred to continually improve the accuracy of results. For example, this could be a prediction tool to work out the best time to send an email. As the model gets more data over time it uses real-results to get its predictions closer to the actual real-life results.
This is clearly superior to basic AI, but still has the problem of being limited to the data supplied to it, and most machine learning models still need an element of human involvement to adjust and tweak algorithms to ensure accuracy.
And Deep Learning?
I don’t know of any current email marketing applications for deep learning. But this is how Alexa, self driving cars and more operate. I won’t go into too much detail here as it's outside the scope of this article but it’s worth touching upon as it’s a buzzword that gets thrown around. In summary, it takes machine learning onto another level where it is capable of learning on its own, mimicking human thought processes. Instead of working with structured data (such as lists of products or order data, lists of people opening the email) it uses unstructured data (where there is no defined framework such as images, text from articles or audio).
Here is an example of how that might work in email marketing.
Let’s say you have a list of items that customers have bought with information about brand, price, category and so on. That could be used by basic AI or machine learning to predict what customers will buy next as it is possible to programme rules to predict this from structured data.
Let’s consider if instead of that product information you used the image of each product instead. The image itself is unstructured data, and deep learning can extract attributes from that product, decide what is important in inferring a customers product taste, and then recommend other products. This activity cannot be done by a human.
What are the most popular AI applications in email?
Product Recommendations
Recommending products, holidays or news stories that are thought to be of interest to the customer
Churn/Unsubscribe Prediction
Predicting customers at risk of churning or unsubscribing, perhaps allowing you to adjust your email frequency or segment into special campaigns.
Send Time Optimisation
Working out the best time to send an email for each individual recipient to achieve maximum response.
Affinity Analysis - Product & Offer
Predicting the likelihood of a customer being interested in a specific product, category or offer. This allows you to reach those likely to be interested even if they haven’t purchased this before that overcomes the limitation of a lot of segmentation approaches that only use past behaviour and therefore restrict the audience sizes.
Subject Line writing
Taking the content of your email and automatically suggesting subject lines that contain language that is more likely to achieve an open.
How to try and evaluate if a specific AI solution is right for you
All of these things sound pretty cool right?
But do they work that way in practice? As marketers evaluating whether we want to add this ‘magic’ to our email programmes we need to apply some critical thinking here.
The sales people will say things like “We will find the best time to send the email to ensure they open it” or “We will send them products we know they are interested in buying”.
Those are amazing outcomes they are promising but of course this is misleading. What they really mean is “We will make a prediction based upon the limited data sources available to us, using a generic algorithm that is the same for all clients.”
It’s hard to be a critical thinker when there is so much mystery to these solutions. So I’ve put some common problems with AI solutions below to enable you to drill into the reality.
Generic AI Algorithms
My main beef with a lot of the AI available to email marketers is where the same data sources and algorithm is in place for all clients using that functionality. Therefore you might have an airline and an online discount store using the same model. If we know the quality of an AI model is the breadth and depth of data used immediately you know that a generic algorithm like this must be making crude assumptions around product category, brand and other important product data attributes. Additionally, how can you expect the algorithm to be tailored to be accurate for both an airline and an online discount store?
It’s not just product recommendations but also think about some of those optimisation models around send time optimisation or churn prediction. Again the algorithms can be generic and you have no control over the data used.
Does it actually work?
The sales literature will make claims but how real are they in real-life? Not all AI functionality is equal. Some is simply added to a product quickly using an off the shelf algorithm from Amazon Web Services - a Data Scientist isn’t involved at all, just so the AI buzzword can be ticked. Others will have been crafted up from the ground up and be continually refined by an expert team to improve results for customers.
For those choosing new vendors, really drill into their case studies to see if the results make sense. Don’t look for a single case study but vendors with a pattern of excellence in this area.
For those thinking of adding front their existing platform try a simple trial, split testing it against what you normally do.
Simple ranking dressed up as personalisation
I’ve seen phrases like “Choose from any number of AI personalisation algorithms” when they then begin to list them as ‘best sellers’ or ‘trending’. All this is taking your product list and ordering the results they are displayed in. It’s barely justifiable as AI, and they only badge it with AI to associate it with the buzz.
Statistical significance
We all know about statistical significance when it comes to A/B testing - you need a decent volume of data to ensure the result isn’t down to randomness. The same is true for AI - if there isn’t enough data how can you be sure the predictions are reliable.
This can be a key reason why some AI isn’t recommended for some brands, they just don’t have enough of the right data for the models to work effectively. One client was being sold the AI dream by a vendor that they can predict what the ultimate offer is to convert a customer to 2nd purchase, despite them having only a few hundred of these customer events each month - nowhere near enough for reliable predictions.
It’s also one of my key worries with iOS 15 potentially killing open data is we will have to move onto clicks and conversions as data points. There are so few of these events in comparison it pushes some types of AI out of reach to only those with the biggest lists.
Predicting Rare Events
It’s not just the volume of source data but the amount of times what you are predicting will come true. Something like an unsubscribe very rarely happens. It happens maybe at a rate of 0.15% each time. Very irregular events are very hard to predict to the extent that any prediction is meaningless, as observed in ‘Monkey Logic’ by respected Data Scientist Mike Swinn.
Does it support your business priorities?
An AI model which predicts the most likely product your customer will purchase next sounds great but is that exactly what you need? This on its own falls short for a few reasons. Firstly, simplistic AI like this often doesn’t take into account seasonality. It might predict a strong interest in Christmas jumpers but that’s no use in July.
Then there is the issue with the conflict between the priorities of the business and what the AI model recommends. For example, the business might really need to push new season stock, or clear out slow moving items but the AI model can’t take the context of the campaign as inputs so it just ends up pushing items you are going to sell through anyway, or can’t adjust the recommended items to match the overall campaign theme.
And finally there is also the conflict between what attracts interest, and what they’ll end up buying. A story I always tell is the ‘shop window’ effect of how visual merchandisers in retail stores create displays with the most eye-catching items to get customers attention, even though they know they’ll end up buying something more conservative in-store. The same is for email, and if your AI model is purely focussed on what they’ll end up buying, not what is best to catch attention then it won’t be as effective.
Creative Messaging
One of the reasons why AI isn’t replacing the marketer anytime soon is the need for creativity and convey emotion into campaigns. Making a campaign resonate with the customer is something that will always need humans. AI can help arrive at what products a customer might be most interested in, or even the right words to use in your subject line. But it can also hinder this if not used correctly, but the marketer still needs to apply their creativity skills.
Measurement & Incrementality
A lot of AI solutions are used within campaigns such as welcome or repeat purchase. These naturally have higher engagement and revenue performance, with or without AI involvement. As always the key to drilling into a vendor case study about the success of their AI functionality is what was the incremental uplift generated versus not using the AI in the exact same audience. Beware figures quoted that show higher revenue per email than the average email.
Practicality of AI
So the whizzy feature promises amazing results - but how practical is it in real-life? For product recommendations will it simply show the same products each time creating fatigue after the first use? How useful is the send time optimisation when you are promoting a flash sale of less than 24 hours? What good is knowing what offers or products customers are most interested in if suppliers, partners and advertisers pay for their exposure? It’s these annoying everyday practicalities that vendors aren’t aware of but you need to consider to evaluate how much benefit you can really get with AI.
What makes good AI?
It’s all very well being sceptical and finding fault but clearly there is strong reasons for AI applications in email marketing. Here is what I think is key for making it a success:
Clear objectives & Plan
Starting with the solution and working backwards to the problem is how a lot of AI is bought. Getting caught up with the buzz of the tech and then needing to find a justification for using it means you have a very narrow field of solutions. Starting with the problem, and identifying all the means of addressing it gives you the best chance of improving, and finding the optimum solutions.
Customisation & Tuning
Going back to how AI algorithms work, good AI allows you to control the volume and breadth of the data, and tune the algorithm to suit your specific scenario.
People & expertise
Think of AI like being a tool such as a paintbrush. The paintbrush in the wrong hands will lead to a mess, while in the hands of a skilled artist can lead to a masterpiece. For AI you need people that understand how it works, how to get the most out of it and how to apply it to your business. It’s not going to get rid of skilled email marketers anytime soon.
Summing up
The tone of this article might come across as a little negative or sceptical. Please don't get me wrong. I'm not an AI sceptic and don't want to put people off AI. AI will play a huge role in all of our lives going forward. Despite the obvious fact that there has been no good music since 1998 I'm not someone clinging onto the old ways of doing things - I even run a business that has AI at the heart of it.
But just like there are good and bad restaurants, and good restaurants that simply aren't your taste, there are good and bad AI technology and models, and those that just aren't a fit for you.
While even average, generic AI solutions might give you a bit of a positive ROI, the chances are you’ll have a large opportunity cost from not evaluating AI properly before you start. While it can be bewildering to understand AI to the novice, have the confidence to cast a critical eye on your proposed AI, don't just believe all of the hype.