Email Marketing: What to Test and How to Test

kid scientist


Posted: July 25, 2013 | Email Marketing

One of the most powerful aspects of email marketing is that it can be tested (and thereby optimized). Following generic “best practices” or fixed “rules” is not a path to success, because what works for one audience won’t necessarily work for yours. Testing is your biggest ally. You should always ‘test for the best’ in your email campaigns.

What to Test

Historically, the subject line has been the most popular marketing email element to test, followed closely by the content of the message itself. But as MarketingSherpa highlights in this chart, there are many other elements of your emails that are just as testable.

email test

Here are 25 elements you can consider testing:

  1. Subject line
  2. From name
  3. Day of the week
  4. Time of day
  5. Frequency
  6. Mostly-images vs. mostly-text
  7. Short copy vs. long copy
  8. Links vs. buttons
  9. Number of links
  10. Unsubscribe at the top
  11. First name personalization — in the subject line
  12. First name personalization — in the email body
  13. Animated gifs
  14. Font colors
  15. Font styles
  16. Opt-down
  17. Social sharing icons
  18. Social connecting icons
  19. Delivery by time zone
  20. Call to action — number
  21. Call to action — placement
  22. Post-click landing page
  23. Social proof
  24. Tone — human vs. corporate
  25. Copy length

Here are some results of real email tests we’ve run at Marketo.

How to Test: Eight Pro Tips

  1. Start simple. Test subject lines and headers first. It doesn’t take a lot of time or creative work to come up with a few simple variants, and the return can be significant.
  2. Test one element at a time. If you test more than one element, you won’t be able to tell which variant drove the success.
  3. Control for time of day and day of the week. If you’re testing other variants, then send on the same day and at the same time to eliminate the timing variant.
  4. Keep a log of all your tests. Record your findings so you can refer back to the specific variables tested and, more importantly, learn from them.
  5. Make sure testing is part of your day-to-day processes. Testing doesn’t have to be daunting, and it shouldn’t be something you put off due to a lack of resources — it should be part of your daily routine.
  6. Run tests on groups that are small, but large enough to determine a clear winner. The winning variables should then be incorporated into your larger mailing.
  7. Don’t forget that small differences can be significant. This is especially true if your sample sizes are large. Visit Marketo’s The Ultimate Guide To Test Statistics to learn all about this.  And for those of you who self-proclaim, “I’m not good at math”, we’ve got a handy A/B testing calculator you can use. (This calculator was originally designed for landing-page conversions, but it works great for email numbers.  Just replace “page views” with “emails sent,” and “conversion rate” with “click-through rate” or “open rate”.)
  8. Listen to what your tests tell you! All the testing in the world won’t matter if you’re not making decisions and modifications to your campaigns based on what you’ve learned. Unused data is sad data, indeed.

Enter the WhichTestWon Email Testing Awards

Speaking of testing, Marketo is a proud sponsor of the 2013 WhichTestWon Email Testing Awards.  Entries are due by 5 pm PST on Tuesday, July 30.  Categories include:

  • List gathering: opt-in pages, forms, offers or campaigns
  • Landing pages: specifically for email-driven clicks
  • Newsletter template
  • “Blast” template – offer campaign
  • Autoresponder or triggered campaigns
  • Transactional notices
  • Subject line or “from” name test
  • Mobile test
  • Social test

Go ahead and enter your test to win fame, links, and a shiny blue ribbon!

email test awards

Jon (@jonmiller) is a VP and co-founder at Marketo. He is the author of multiple Definitive Guides including Marketing Automation, Engaging Email Marketing, and Marketing Metrics & Analytics. In 2010, The CMO Institute named Jon a Top 10 CMO for companies under $250 million revenue. Jon holds a bachelor’s degree in physics from Harvard College and has an MBA from the Stanford Graduate School of Business.

Read Jon's Blogs

Learn the top 25 elements you should be testing in your #emailmarketing @jonmiller

Follow Us

Most Shared

true colors feat

What Brand Colors Say About Your Business – Marketo

purple 4 hands

The 4 Things Digital Marketers are Missing

measuring social

6 Ways to Make Social Measurable

Influencers to follow 2017

31 Influencers to Follow in 2017

You Don’t Know Jack About Online Marketing – More than Just the Coolest Marketing Game Ever