The market is likely close to hitting the bottom — with a fairly quick ascension anticipated in many verticals over the next couple quarters — and most communities are in the middle of a phased return to the workplace. While we may never see a return to business as usual, we are on the path to a “new normal” which is already serving as a signal to business executives to ramp up their marketing.
With the noise out there competing for the attention of your prospects already, the volume of which is anticipated only to increase over the coming two months, how do you ensure that your message will actually break through? What’s more, how do you ensure your marketing messages are appropriate — and not tone deaf — given all that’s going on in the world? The answer to both questions is “split testing” on a grand scale … a marketer’s secret weapon.
Leveraging Split Testing
Split testing, also referred to as A/B testing or multivariate testing, is a method of conducting a series of controlled, randomized experiments with the goal of selecting the strongest campaign and fully optimizing its effectiveness. The objective for a traditional split test is to wait for a statistically significant difference in consumer behavior to occur as a result of seeing one ad versus another — with the only difference between the two being a single variable such as the image used or headline. You then continue to test additional variables within that ad over time until it’s fully optimized. The key to an effective split test is ensuring that only a single variable changes each time. For example, you might test the same ad design with 5 different headlines. Or the same design with 5 different calls to action. The most commonly tested variables are: images and other design elements, color palette, font, copy, headlines, and calls to action.
The fundamentals of this approach to testing new advertising campaigns align well with a business philosophy of author Jim Collins — outlined in his book “Great by Choice” — where he shares the benefits of a bullets-then-cannonballs (figuratively speaking) approach to innovation testing and recalibration. Please forgive the reference to ammunition in light of today’s troubled world, but it’s an essential comparison to make the point.
Bullets Then Cannonballs?
Here’s an analogy, from Collins, that will help cement the concept in your mind. “Picture yourself at sea, a hostile ship bearing down on you. You have a limited amount of gunpowder. You take all your gunpowder and use it to fire a big cannonball. The cannonball flies out over the ocean…and misses the target, off by 40 degrees. You turn to your stockpile and discover that you’re out of gunpowder. You die. But suppose instead that when you see the ship bearing down, you take a little bit of gunpowder and fire a bullet. It misses by 40 degrees. You make another bullet and fire. It misses by 30 degrees. You make a third bullet and fire, missing by only 10 degrees. The next bullet hits — ping! —the hull of the oncoming ship. Now, you take all the remaining gunpowder and fire a big cannonball along the same line of sight, which sinks the enemy ship. You live.”
The idea of firing bullets and then cannonballs in business-innovation testing is inspired by this analogy. First, you “fire bullets (low-cost, low-risk, low-distraction experiments) to figure out what will work — calibrating your line of sight by taking small shots. Then, once you have empirical validation, you fire a cannonball (concentrating resources into a big bet) on the calibrated line of sight. Calibrated cannonballs correlate with outsized results; uncalibrated cannonballs correlate with disaster. The ability to turn small, proven ideas (bullets) into huge hits (cannonballs) counts more than the sheer amount of pure innovation.”
Simple A/B Split Testing Vs. Aggressive Multivariate Testing
So let’s talk specifically and practically about how to implement a bullets-then-cannonballs approach to your return-to-market advertising campaigns to ensure they are appropriate and breaking through.
You may elect to simply run one test at a time and continue to iterate your ad campaign until further refinements offer diminishing returns. Note that you’ll need to test your campaign separately through each channel, as certain images will perform better on social media versus email, for example.
While testing just one variable at a time can be a painstaking and lengthy process, there is another way — multivariate testing — where you are testing a variety of elements all at once. This is the fastest path to clarity around messages that are appropriate for our times and most likely to elicit the desired behavior (e.g., clicking or buying).
Here’s a sample map of what a multivariate test of a Facebook ad campaign could look like if you’re testing three images, three ad titles, and five precise prospect interests for fine targeting. These prospect interests, by the way, are determined by Facebook based on the pages the audience likes, the posts they engage with, etc. As you can see from the graph that follows, this results in testing 45 different ad scenarios.
