Modern B2B Marketing
Modern B2B Blogs
November 30, 2006
3. Test everything – but don’t over test
Posted by Jon Miller
The best B2B marketers test everything, and almost everything can be tested: offers, copy, headlines, forms, bids, colors, designs, lists, and more. Testing removes any debate about what works and what doesn’t; testing lets your customers vote with their actions. Many marketers are surprised to find out that the winner of the test is not the one they would have picked before the test. Landing pages are one of the most valuable things to test. A division of Siemens USA found that testing on the home page improved conversions by 2.3% but testing on the landing page increased conversions by 115%!
Be careful not to overtest. This is especially a problem in the “high value, low volume” world of B2B marketing. A good rule of thumb is that you need at least 15-20 positive actions within a two week period for each item you want to test. (Some may argue you need more for complete statistical confidence, but 15-20 should be enough to make most B2B marketers comfortable.) For example, let’s say you have an ad group with a set of keywords that collectively are searched 20,000 times a week. Assuming you get a 2% click-thru rate and a 5% conversion rate, you will get 20,000 x 2% x 2 weeks = 800 clicks and 800 x 5% = 40 conversions in a two week period. With this volume, you can safely test up to 800/20 = 40 different versions of your ad copy, and two versions of your landing page.
What this means is that B2B marketers can and should test many versions of their search ad copy and headlines, constantly experimenting to see what works best. Small changes in wording and call to action can make a big difference. At the same time, marketers who work with the lower volumes common to B2B must focus landing page tests on the highest impact pieces, typically the call to action and the form. This also means that high-end testing solutions may be overkill for many B2B marketers.
Marketing & Strategy Innovation Blog said on February 24, 2007 at 3:03 am
3. Test everything – but don’t over test
by: Jon MillerThe best B2B marketers test everything, and almost everything can be tested: offers, copy, headlines, forms, bids, colors, designs, lists, and more. Testing removes any debate about what works and what doesnt; testing lets your cus…
Jon Miller said on January 4, 2007 at 12:22 pm
As a follow-up to my landing page example above, assume you show the two landing pages evenly over the two weeks. This means Page A gets 400 impressions and Page B gets 400 impressions. If A gets 22 conversions (a conversion rate of 5.5%) and B gets 18 conversions (4.5%), then you can be 74.2% confident that A is better than B. Although not statistically significant at the 95%-level, this may be considered directionally correct. (Note: This confidence is calculated using a Simple Two-sample Z-test which assumes a normal distribution. Using a Pearson Chi-Squared test, we find there is a 51.6% probability that chance alone created the difference in response rates — meaning we’re halfway to being confident that the difference is real. Again, not significant but directionally correct.)