Email A/B Testing: 4 Tests That Actually Improve Results

Most email programmes plateau not because the strategy is wrong, but because no one is testing anything. Email A/B testing is how you find out what actually works for your specific audience, not what works in general. The two are rarely the same.

The principle is straightforward: you send two versions of an email, each with one variable changed, to two segments of your list. Whichever version performs better tells you something real about what your audience responds to. Then you use that knowledge in every email that follows. What makes A/B testing genuinely valuable is not the individual result. It is the compounding. A CTA that gets 10% more clicks improves every email that uses that format going forward. A sender name that lifts open rates by 5% does the same. Small improvements, applied consistently, add up to a programme that performs noticeably better over time. Here are four tests worth running.

A note before you start Mail Blaze does not currently include a built-in A/B testing tool. To run these tests, split your list manually, send each version as a separate campaign, and compare results in your reporting. It takes a little more setup, but the method is exactly the same and the insights are just as valid. Mail Blaze's comparative filters are built for exactly this: once your test is done, bring both campaigns in and measure them against each other directly.

1. Your subject line

Subject lines are the most commonly tested element in email, and for good reason. They are the first thing your subscriber sees and they determine whether the email gets opened at all. The important thing with subject line testing is to test one variable at a time. Do not test a short subject line with an emoji against a long one without. You will not know which variable made the difference. Test length in one experiment. Question versus statement in another. Personalisation in another. Each test produces a clear, usable answer. Some findings from subject line testing are counterintuitive. Negative framing ("Don't make this mistake") sometimes outperforms positive framing ("How to improve your results"). The Obama campaign's "Hey" subject line, which appeared too casual to anyone who saw it, raised $2.5 million more than its nearest competitor. Your audience will surprise you. That is the point. One important note: since Apple's Mail Privacy Protection launched in 2021, open rates are less reliable as an absolute measure. Apple Mail pre-fetches email content, including tracking pixels, which inflates open rates for Apple Mail users. Open rates are still valid for comparing two versions of a subject line in the same test, since both versions are affected equally. Just do not use them as your primary engagement measure outside of testing.

2. Your call to action

CTAs are one of the highest-value elements you can test, because a CTA improvement compounds across every future send that uses the same format. There are several variables worth testing: button versus text link, button copy, button colour, and button position in the email. These are four separate tests. Run them one at a time. On copy specifically: the research consistently shows that first-person phrasing outperforms second-person. "Start my free trial" gets more clicks than "Start your free trial." It is a small change, but across thousands of sends, it is worth knowing. Before you test, make sure both versions are already following good practice: the copy is specific and action-oriented, the button contrasts clearly with the background, and the language tells the reader exactly what will happen when they click. Testing a weak CTA against a slightly less weak one does not teach you much. Test two strong versions against each other.

3. Your send time

Send time is underrated as a test variable. The same email sent at 6am versus 2pm can produce meaningfully different results, and the right answer varies by audience. The problem is that most email programmes pick a send time and stick with it, often based on a general recommendation rather than their own data. Testing morning versus afternoon for your own list takes one experiment to answer a question that will then inform every campaign you send. If you are running automated flows, send time testing is even more valuable there, because any improvement applies to every future send in the flow automatically.

4. Your sender name

Sender name is one of the highest-leverage tests you can run, because it appears on every email you send. Even a small improvement in open rate compounds significantly over time. The formats worth testing: personal name (Pauline), brand name (Mail Blaze), or a combined format (Pauline at Mail Blaze). In some industries and audiences, a personal name builds more trust. In others, a recognisable brand name performs better. You will not know which applies to your audience until you test it.

The one rule that makes all of this work

Every A/B test you run needs to follow one rule: change one variable at a time. If you change the subject line, the sender name, and the CTA in the same test, you will not know which change produced the result. The test is meaningless.

It also means waiting for enough data before you decide. Most marketers check results a few hours after sending, see one version ahead, and call it. The problem is that this is often just random variation, not a real result. For open rate tests, two hours of data gives you around 80% accuracy. For revenue-based tests, let the experiment run for a full day. Only about 1 in 7 A/B tests produces a statistically significant winner. That is not failure. It means most of your current practices are already reasonable, and meaningful improvements are found at the margins. Run enough tests and you will find them.

When a test is inconclusive, that is useful too. It tells you that particular variable does not matter much to your audience. Move your testing energy to something that does. The best email programmes treat every send as an opportunity to learn something. Document your results. Share them across the team. And keep testing.

Ready to see what your audience actually responds to? Mail Blaze's reporting gives you the comparison data you need to make testing decisions with confidence. Start your free 14-day trial and run your first test with no credit card required.

P.S. If you have a question about setting up your first A/B test, our team is available seven days a week. Just reach out.