24 Ideas How to Conduct an Efficient Email A/B Testing
Categorized as : Ecommerce
A well-thought-out email strategy helps businesses to notably increase conversion rates, especially during the holidays when the demand for particular products is considerably higher than usually.
But how can you know what emails can produce the desired effect on customers and encourage them to make a purchase? Only by experimenting with different variants of letters! In this article, we will give you some tips on how to organize an effective email campaign through A/B testing (or split testing) of emails.
If you are running a Magento 2 web store and need to test different versions of emails, a smart choice here is to use the recently updated Follow Up Email Extension 2.0. The solution provides the A/B Testing Mode, which can be enabled on the Edit Email page in the Magento 2 Admin Panel.
First, prepare the email (version A) and its alternative variant (version B) required for your A/B test. These emails should differ only by one parameter, in other words, you can create two variants (or more if you need to conduct several A/B tests) of the same email that will differ in, say, headings, images, or texts. Let’s check what Follow Up Email 2.0 is ready to offer.
Using the A/B Testing Mode of Follow Up Email 2.0, you can experiment with the next parameters: Sender Name, Sender Email, Subject, Content, Header and Footer. In addition, you can take advantage of the provided variables and preview mode. With the extension, you can create messages with various layouts, as the solution supports HTML and CSS.
Having created the letters, next, you need to get access to the statistics displaying the number of sent emails, their openings, clicks, open rates, and click through rates.
And, Follow Up Email 2.0 provides the statistics on all the above key points.
To find the answer to this question, view the statistics of previous email campaigns. If the majority of clicks occurred during the first 24 hours, it is possible to limit the duration of testing to this time period.
Note that the time for your email split tests should be extended in case if you have a large customer base and did not manage to collect all the desired results in 24 hours.
Emails contain many components that need to be tested. Let’s consider each of them individually.
Having received your letter, first, the recipient pays attention to the name of the person who sent it. So, test this parameter by changing the male and female names for different versions or by choosing the company’s name for this field.
When composing the subject of the letter, pay attention to its wording and length. The wording of a subject reflects the style of the following text. By reading the subject, a subscriber makes the decision whether to read the letter or move it to trash.
To find out the subjects that work better, try to use different wordings making them either personalized or general, writing them in the form of a statement, a question, a call to action, or in some other forms. Also, pay attention to the number of symbols used to write the subject, as too long email subjects might seem boring to customers.
This is the text that appears after the subject of a letter. In the example below, the preheader is highlighted with the red block.
Pay a close attention to this parameter, as the text of the preheader specifies the subject of the email and urges the subscriber to open or trash it. The experiments with preheader styles are much similar to the ones with subjects.
A/ B tests can help you define the fonts to be used for email texts. In fact, there are no universal rules for using fonts in emails, so here, you are limited only by your imagination. However, it’s not recommended to use more than three fonts in one email, as the letter with too many fonts will look messy. For example, you can use different fonts for the header, main text, and product description of your message.
With Follow Up Email 2.0, you can experiment with fonts as you like. For this purpose, just edit the content on the Edit Email page.
And here how it looks in the Preview Mode.
Let’s consider the fonts that are commonly used for product descriptions:
Below, you can see the example of an email made by Follow Up Email 2.0 with the use of all the above-described fonts.
Here, we implemented various styles for each font. For the header, we chose the bold Arial font of the 3px size. For the main text, Helvetica was used in its default size, and for the last two sentences of the letter, Verdana in italics was chosen, still, we kept the original size of the font. Compared to the example of the same letter we brought earlier, this one might seem more appealing to customers.
Different text layouts work differently as well. You can send emails with plain texts, lists of facts and features, or texts written in two or even three columns with some pictures attached and background colors. Below, you can see three examples of the same email demonstrating different layouts.
In the latter two versions, the texts look well-structured compared to the first variant. The emails organized as lists or divided by columns are pleasant to read and easy to remember.
What is the best option for your business: one main reference or a few links attached to the email? Try to test it and make your choice.
What works better: text or graphic links? One more question to consider during your email A/B tests.
To test this parameter, use official tone for the first variant of the email and more informal one for the second. Test both variants.
Cryptic messages are often used to create a sense of intrigue and make customers interested. So, try to test various teaser options, add thrilling images if you like to see how this tactic might influence the purchasing decision.
"Instant sale!", "Only for … hours ...", "Have the time to buy, there are only ... days left". It is believed that the sense of urgency better impacts sales. So, test various types of offers by creating, for example, a usual newsletter and an urgent newsletter to find out which one will be more effective in terms of sales.
Which announcement looks and sounds better to you: "10% discount" or "Save $30"? Consider the variants of announcements to be tested.
Each color produces a particular effect on customers. The yellow color, for example, attracts attention, the red color increases the heartbeat by creating the feeling of urgency. The blue color causes a sense of peace and comfort. The black color is the symbol of power and authority. In advertising, this color often symbolizes luxury. So, learn more about the psychology of colors to improve your mailings.
Additional items such as arrows, icons, etc. serve to highlight most important fragments of emails and attract attention of readers. Try to add them to your email to see what effect this move will produce compared to the letter with no additional items.
Consider how to better visualize prices to attract more attention of the audience and increase the CTR. Try to highlight the prices of your products by using color blocks, bold fonts, and other visual effects to find out which variant seems to be more appealing to customers.
Often, the CTA (Call to Action) buttons are placed at the bottom of the contents. Try to test the variant where these buttons appear at the top of the email. Carefully consider the number of CTAs to be included, as multiple CTAs can distract readers’ attention. Compose, for example, the version with two CTAs and the one with three CTAs and compare the results.
When testing this parameter, make a series of photos of your product that will differ in style or camera angle to see the photos customers like more.
Sometimes, it is very difficult to decide how many products to advertise in a single email. So, test the variant with one product image and the option containing the whole line of product images and then compare the results.
The larger is the size of the product image, the more details subscribers will see on it and will likely better know it. So, try different picture sizes in your emails.
If you are not sure whether to use video content in your mailings or not, test it and make sure. Using videos you can present your products, add video guides, or simply congratulate clients on their holidays.
Your video can be easily embedded in the HTML code of the email in the Content field on the Edit Email page of Follow Up Email 2.0.
A GIF picture with dynamic animation attached to emails always catches the eye since many readers are already tired of static images. GIF animation is often used for promotions, as flashing frames evoke reader's desire to find out details of the proposal. You can test different versions of your GIF images or even combine them with your video clips.
This parameter should be discussed in more detail.
When you see positive effect from your mailings, including subscribers interest and engagement, sales growth, etc., there will be a temptation to send emails more often. Most companies start with weekly mailing, but then, they increase the number of emails to 3-4 per week or even send them every day, including weekends.
This increase in sending usually produces only short-term positive effect, but eventually, you risk to scare off clients with your assertiveness.
The optimal frequency can be determined in practice, again, by conducting A/B email testing. First, select two random segments of the audience in your database. Then, choose one of the segments and send it a series of emails with a certain frequency.
For the second group, you should send the same emails with a different frequency. Then, compare the indicators of each email for each segment. The key indicators for this test are the number of openings, CTRs, target actions, and the number of “unsubscribes”.
Note that if any of the tracked indicators dropped by 10-20%, there is no need to worry about such an insignificant decrease. But, if you see a notable growth of the parameters, then you can keep sending emails with the optimal frequency without ceasing to closely monitor all the above-mentioned KPIs.
Also, try to adjust the frequency of emails depending on the stage of the life cycle of a subscriber. New subscribers are ready to receive letters more often, for example, several times a week, as they are still finding out about your brand and want to learn more. Those who are in your client base for a long time (at least, six months) would probably prefer to receive emails from you 1-4 times a month.
Any marketer will say that the best days for sending emails are Tuesday, Wednesday, and Thursday. The best days for useful and entertaining content are Friday and even weekends. Such mailings should be sent in the morning or in the afternoon when internet users are the most active. So, try to test various days and time periods for sending your emails.
In this case, you need to determine which offers are more preferred by your subscribers: gifts, bonuses, discounts, etc., then you can send emails with different variants of incentives and then choose the one that produced a better effect.
For many people, it is very important to try a product or a service before making a purchase to find out whether it is suitable for them or not. Therefore, you can create two A/B testing options: the first one, for example, will offer a free trial subscription for two weeks and the second - one month. Thus, you can determine the time your customers need to evaluate your products without rush.
A/B testing is a reliable tool for optimizing email campaigns. Since there is no one sure way to create efficient emails, you need to compare different variations to find out the one that works better for your audience and therefore, for your business. And the Magento Follow Up Email 2.0 Extension by Aheadworks offers great opportunities for A/B email testing, thanks to which you can apply practically all the above-described ideas.
But how can you know what emails can produce the desired effect on customers and encourage them to make a purchase? Only by experimenting with different variants of letters! In this article, we will give you some tips on how to organize an effective email campaign through A/B testing (or split testing) of emails.
Email A/B Testing
If you are running a Magento 2 web store and need to test different versions of emails, a smart choice here is to use the recently updated Follow Up Email Extension 2.0. The solution provides the A/B Testing Mode, which can be enabled on the Edit Email page in the Magento 2 Admin Panel.
First, prepare the email (version A) and its alternative variant (version B) required for your A/B test. These emails should differ only by one parameter, in other words, you can create two variants (or more if you need to conduct several A/B tests) of the same email that will differ in, say, headings, images, or texts. Let’s check what Follow Up Email 2.0 is ready to offer.
The Letter Settings Panel
Using the A/B Testing Mode of Follow Up Email 2.0, you can experiment with the next parameters: Sender Name, Sender Email, Subject, Content, Header and Footer. In addition, you can take advantage of the provided variables and preview mode. With the extension, you can create messages with various layouts, as the solution supports HTML and CSS.
Having created the letters, next, you need to get access to the statistics displaying the number of sent emails, their openings, clicks, open rates, and click through rates.
A/B Testing Statistics
And, Follow Up Email 2.0 provides the statistics on all the above key points.
Duration of Tests
To find the answer to this question, view the statistics of previous email campaigns. If the majority of clicks occurred during the first 24 hours, it is possible to limit the duration of testing to this time period.
Note that the time for your email split tests should be extended in case if you have a large customer base and did not manage to collect all the desired results in 24 hours.
Testing Parameters
Emails contain many components that need to be tested. Let’s consider each of them individually.
1. Sender Names
Having received your letter, first, the recipient pays attention to the name of the person who sent it. So, test this parameter by changing the male and female names for different versions or by choosing the company’s name for this field.
2. Email Subjects
When composing the subject of the letter, pay attention to its wording and length. The wording of a subject reflects the style of the following text. By reading the subject, a subscriber makes the decision whether to read the letter or move it to trash.
To find out the subjects that work better, try to use different wordings making them either personalized or general, writing them in the form of a statement, a question, a call to action, or in some other forms. Also, pay attention to the number of symbols used to write the subject, as too long email subjects might seem boring to customers.
3. Preheader Texts
This is the text that appears after the subject of a letter. In the example below, the preheader is highlighted with the red block.
Email Preheader
Pay a close attention to this parameter, as the text of the preheader specifies the subject of the email and urges the subscriber to open or trash it. The experiments with preheader styles are much similar to the ones with subjects.
4. Fonts
A/ B tests can help you define the fonts to be used for email texts. In fact, there are no universal rules for using fonts in emails, so here, you are limited only by your imagination. However, it’s not recommended to use more than three fonts in one email, as the letter with too many fonts will look messy. For example, you can use different fonts for the header, main text, and product description of your message.
With Follow Up Email 2.0, you can experiment with fonts as you like. For this purpose, just edit the content on the Edit Email page.
Email Content
And here how it looks in the Preview Mode.
Email Preview Made by Follow Up Email 2.0
Let’s consider the fonts that are commonly used for product descriptions:
- Arial, Helvetica – very simple and well-perceived visually. These fonts are displayed correctly by any email service. Apple Mail and Gmail use these fonts by default.
- Verdana – favorably influence the ease of reading due to its inter-letter space.
Below, you can see the example of an email made by Follow Up Email 2.0 with the use of all the above-described fonts.
The Email Using the Arial, Helvetica, and Verdana Fonts
Here, we implemented various styles for each font. For the header, we chose the bold Arial font of the 3px size. For the main text, Helvetica was used in its default size, and for the last two sentences of the letter, Verdana in italics was chosen, still, we kept the original size of the font. Compared to the example of the same letter we brought earlier, this one might seem more appealing to customers.
5. Presentation of Information
Different text layouts work differently as well. You can send emails with plain texts, lists of facts and features, or texts written in two or even three columns with some pictures attached and background colors. Below, you can see three examples of the same email demonstrating different layouts.
Plain Text
List of facts
Information Presented in Columns (Using Pictures and Background Colors)
In the latter two versions, the texts look well-structured compared to the first variant. The emails organized as lists or divided by columns are pleasant to read and easy to remember.
6. Number of References
What is the best option for your business: one main reference or a few links attached to the email? Try to test it and make your choice.
7. Link Types
What works better: text or graphic links? One more question to consider during your email A/B tests.
8. Tone of the Offer
To test this parameter, use official tone for the first variant of the email and more informal one for the second. Test both variants.
9. Teasers
Cryptic messages are often used to create a sense of intrigue and make customers interested. So, try to test various teaser options, add thrilling images if you like to see how this tactic might influence the purchasing decision.
10. Limited Offers
"Instant sale!", "Only for … hours ...", "Have the time to buy, there are only ... days left". It is believed that the sense of urgency better impacts sales. So, test various types of offers by creating, for example, a usual newsletter and an urgent newsletter to find out which one will be more effective in terms of sales.
11. Announcements
Which announcement looks and sounds better to you: "10% discount" or "Save $30"? Consider the variants of announcements to be tested.
12. Color Schemes
Each color produces a particular effect on customers. The yellow color, for example, attracts attention, the red color increases the heartbeat by creating the feeling of urgency. The blue color causes a sense of peace and comfort. The black color is the symbol of power and authority. In advertising, this color often symbolizes luxury. So, learn more about the psychology of colors to improve your mailings.
13. Navigation or Attention Marks
Additional items such as arrows, icons, etc. serve to highlight most important fragments of emails and attract attention of readers. Try to add them to your email to see what effect this move will produce compared to the letter with no additional items.
14. Price Visualization
Consider how to better visualize prices to attract more attention of the audience and increase the CTR. Try to highlight the prices of your products by using color blocks, bold fonts, and other visual effects to find out which variant seems to be more appealing to customers.
15. Location and Number of Calls to Action
Often, the CTA (Call to Action) buttons are placed at the bottom of the contents. Try to test the variant where these buttons appear at the top of the email. Carefully consider the number of CTAs to be included, as multiple CTAs can distract readers’ attention. Compose, for example, the version with two CTAs and the one with three CTAs and compare the results.
16. Images of Products
When testing this parameter, make a series of photos of your product that will differ in style or camera angle to see the photos customers like more.
17. Number of Product Images
Sometimes, it is very difficult to decide how many products to advertise in a single email. So, test the variant with one product image and the option containing the whole line of product images and then compare the results.
18. The Size of Product Images
The larger is the size of the product image, the more details subscribers will see on it and will likely better know it. So, try different picture sizes in your emails.
19. Video Content
If you are not sure whether to use video content in your mailings or not, test it and make sure. Using videos you can present your products, add video guides, or simply congratulate clients on their holidays.
Your video can be easily embedded in the HTML code of the email in the Content field on the Edit Email page of Follow Up Email 2.0.
20. GIF Animation
A GIF picture with dynamic animation attached to emails always catches the eye since many readers are already tired of static images. GIF animation is often used for promotions, as flashing frames evoke reader's desire to find out details of the proposal. You can test different versions of your GIF images or even combine them with your video clips.
21. Frequency of Emails
This parameter should be discussed in more detail.
When you see positive effect from your mailings, including subscribers interest and engagement, sales growth, etc., there will be a temptation to send emails more often. Most companies start with weekly mailing, but then, they increase the number of emails to 3-4 per week or even send them every day, including weekends.
This increase in sending usually produces only short-term positive effect, but eventually, you risk to scare off clients with your assertiveness.
The optimal frequency can be determined in practice, again, by conducting A/B email testing. First, select two random segments of the audience in your database. Then, choose one of the segments and send it a series of emails with a certain frequency.
For the second group, you should send the same emails with a different frequency. Then, compare the indicators of each email for each segment. The key indicators for this test are the number of openings, CTRs, target actions, and the number of “unsubscribes”.
Note that if any of the tracked indicators dropped by 10-20%, there is no need to worry about such an insignificant decrease. But, if you see a notable growth of the parameters, then you can keep sending emails with the optimal frequency without ceasing to closely monitor all the above-mentioned KPIs.
Also, try to adjust the frequency of emails depending on the stage of the life cycle of a subscriber. New subscribers are ready to receive letters more often, for example, several times a week, as they are still finding out about your brand and want to learn more. Those who are in your client base for a long time (at least, six months) would probably prefer to receive emails from you 1-4 times a month.
22. Day and Time of Sending
Any marketer will say that the best days for sending emails are Tuesday, Wednesday, and Thursday. The best days for useful and entertaining content are Friday and even weekends. Such mailings should be sent in the morning or in the afternoon when internet users are the most active. So, try to test various days and time periods for sending your emails.
23. Bonus Offers
In this case, you need to determine which offers are more preferred by your subscribers: gifts, bonuses, discounts, etc., then you can send emails with different variants of incentives and then choose the one that produced a better effect.
24. Trial Period
For many people, it is very important to try a product or a service before making a purchase to find out whether it is suitable for them or not. Therefore, you can create two A/B testing options: the first one, for example, will offer a free trial subscription for two weeks and the second - one month. Thus, you can determine the time your customers need to evaluate your products without rush.
Conclusion
A/B testing is a reliable tool for optimizing email campaigns. Since there is no one sure way to create efficient emails, you need to compare different variations to find out the one that works better for your audience and therefore, for your business. And the Magento Follow Up Email 2.0 Extension by Aheadworks offers great opportunities for A/B email testing, thanks to which you can apply practically all the above-described ideas.
Related Posts
