Email testing framework

Why is email testing important?

The main reason why email testing is important is because you want to keep building and enhancing your email marketing performance and in turn improve your results. Email testing can help you make changes to how you develop your emails in the short term and help you make long-term gains.

Email testing enables you to use the data that you gather in an efficient way and not just guess why your audience is responding in a certain way. Therefore in a nutshell, you want to do email testing because you could be missing out on opportunities to increase the return on investment from your campaigns.

The Asgardian testing framework

We like to have fun, and still get things done. While email testing and following an email testing framework might seem boring, we’ve found a great resource by Pure 360 that brings a bit of fun to the process.

Ever heard of Asgardians? Well the framework we’re going to use is called the Asgardian framework and is a great framework to follow for any marketing testing that you may need to do.

Ever watched the Marvel movies Thor, or Captain America? Well the Asgardians are based on Norse mythology and are said to be a race of extraterrestrials similar in appearance to humans with advanced technology that resembles magic.

While there’s nothing magical about the email testing framework we’re going to introduce to you now, adding an element of fun will make this framework easy to follow and remember.

Let’s unpack this is more detail

The ASGARDIAN framework is made up of the following:

• AIM
• SUCCESS
• GRASP
• ASSUME
• RECIPIENTS
• DATA
• INTERVAL
• AMBITION
• NEW
• SUCCESS

When you start using this framework, to make life easier, we’ve compiled a downloadable PDF for you that you can fill out. All you’ll need to do is fill out each section and monitor your progress as you move through the steps.

Download the Asgardian framework for yourself today.

Why should you use this framework?

• It has a logical flow and steps to follow
• It enables you to document the whole process
• You have clear records of what you’ve tested and what the outcomes were

The Asgardian framework unpacked:

AIM

The first step to completing the Asgardian testing framework is to define what your aim is. Basically, what is the goal you are trying to achieve through the test you are going to run? You can keep it super simple or make it as detailed as you want. An example could be:

We’ve seen a big dip in our open rates over the last 3 email campaigns that we have sent out.

SUCCESS

This is exactly what it says, now what does success look like for you after running a test? An example could be:

We would like our click-through rate to return to 20%, where it has been at 15% for the last 3 campaigns we sent out.

GRASP

Grasp refers to what your current situation is. Where are you now? An example of this could be:

Our current average click-through rate is 15%.

ASSUME

These are the nuts and bolts of your test. In essence, this is the change or test you want to run based on what you assume is going to help you succeed. An example of this could be:

We assume that we can improve our average click-through rates by changing our CTA buttons from orange to green.

RECIPIENTS

The next step to completing the testing framework is to decide who you are going to send the test to. You might want to run the test on a certain segment of your subscriber base or send it to everyone. This is up to you. An example of what this looks like is shown below, but you could also identify a certain list you want to send to if you have multiple lists.

We will be running this test with all our subscribers that have subscribed in the last year.

DATA

Once you have decided who you are going to send to, you will need to figure out what a statistically significant number is to you. You could decide to do a 50:50 split or figure out a formula that makes sense for your unique use case.

INTERVAL

How long are you going to run your test for? You need to consider how frequently you send out emails generally to the selected recipients to ensure that you don’t skew your test results. If you generally contact these subscribers twice a week, you should still contact them twice a week but decide for how long you will implement your change.

Our test will be running for the next 4 weeks starting on X Day.

The last three steps in the framework are essentially just a summary of why you want to be running these tests:

AMBITION

Set the scene in terms of why you want to run these tests. Be very specific here. You can speak to your challenges and why you want to overcome them.

NEW

Clearly outline the exact specifications of the test. Use as much detail as possible here and also include what your control group will be.

SUCCESS

Reaffirm what success looks like and tie this to your business objectives. How will running this test improve your performance and help you achieve your goals?

What elements of your email can you test?

The possibilities are really endless when it comes to what you can test in your emails. A more important place to start would be what metrics you are trying to improve. Below we will look at elements that are commonly tested and what metrics they can influence, but this is by no means an exhaustive list. You need to understand that each email audience is unique and have their own needs and motivations when it comes to what they want to receive.

Subject lines:

Subject line testing can be extremely useful to help you increase your email open rates. Finding the right subject lines that resonate with your audience can go a long way to helping you get those open rates up. If you’re unsure how you can spice up your subject lines, we’d recommend starting out by trying to use some subject line formulas.

Another test to run is to also increase or decrease the amount of characters that you are using in your subject line. Subject line testing should be run over a long time because you need to account for the influence of offers and creative elements.

You could also look at your Google Analytics account and see what people use to search for you as a guide for what to include in your subject lines. Other ideas:

• Look at personalised vs non-personalised subject lines
• Including offers vs not including offers
• Using your brand vs not using your brand

Time of send:

You may think that sending your email when it’s convenient for you is just the way it is, but sending at specific times can greatly improve your email performance.

We recommend running time of sending tests over a month period, especially if your emails include offers. This applies to when you test the day of send too. Time of send can influence your email open rates.

Header images:

If you want to test which header images in your email campaigns perform better, you’ll want to ensure that you have a control test and a test group where nothing but the header image changes. Try and ensure that no other parameters are different when you are testing this kind of element. Header images can influence whether people scroll through or click through to other elements in your email, therefore they can impact your click-through rates.

Calls to action:

Calls to action are extremely important when it comes to achieving your email goals as you want people to know what to do next. For example click through to a product on your website. Calls to action tests can be done in terms of the actual design of the call to action button or simply the text that is on the button. Do not change the text and colour at the same time, as you won’t know which change had the biggest impact.

The offer in your email:

It can be really useful to test the offer that you provide in an email to your subscriber i.e. does a 15% discount offer more value to a subscriber than a 20% discount offer. While this may seem obvious, different audiences perceive offers differently and may be more likely to actually engage with a smaller discount. You could also test what threshold of minimum spend influences the uptake of a discount i.e. 15% discount with a minimum spend of R500 vs 15% discount on a minimum spend of R750. Again it’s really important to only change the offer when running these tests and to keep all other elements the same to see what works best.

Tagline or headline testing:

If you commonly use taglines or big headlines that lead into your content, these can be a great element to test. You can set up two different headlines or taglines to see how they perform.

SUMMARY:

Email testing is extremely important to help you enhance your email performance but can seem daunting to do. We hope this email testing framework will enable you to set up a structured email testing process so you can reach your marketing goals. How often do you run email tests?