A/B Test Everything in eCommerce: From Social Media to Email Marketing

Ruthie Careyby Ruthie Carey

Home | Blog | A/B Test Everything in eCommerce: From Social Media to Email Marketing

Share this article

No matter if merchants are aiming to generate more leads, grow their email list, acquire more click-throughs, increase open rates, drive more sales or achieve a myriad of other goals, A/B testing is a critical component to success.

Moreover, when discussing why you should A/B test, it is important to note that this strategy, which is usually associated with email marketing, extends to other essential marketing channels such as social media.

The fact of the matter is that A/B testing can be used to optimize the process of turning social media traffic and email subscribers into full-fledged customers.

However, in order for merchants to successfully harness the power of A/B testing for social media or email marketing efforts, it is essential to realize that there is a panoply of variables that must be considered.

From copy to creative and everything in between, there are a multitude of elements that come together to create an overall experience that will either entice consumers into action or not.

By employing various tips for A/B testing, merchants can uncover the best arrangement and combination of components to encourage consumers to act on their interests.

By adopting A/B testing for email marketing and social media marketing tactics, sellers can effectively elevate their online performance, generating more leads, followers and subscribers, website traffic and (most importantly) sales.

To help retailers bring such ambitions to fruition, today, we will explore why you should A/B test email and social media campaigns, as well as take a look at a variety of tips for A/B testing to optimize performance.

Prior to digging into the juicy details, let’s take just a moment to lay the groundwork for the conversation.

What Is A/B Testing?

When many folks think about A/B testing, they often think of it in terms of making minor adjustments to CTAs, headlines or the color of buttons.

In reality, A/B testing goes much deeper.

To put things plainly, A/B testing is a methodology used to determine which design practices, content, arrangements or functionality is most successful for driving visitors to action.

A/B testing enables merchants to test a variety of elements that could impact user behavior to achieve optimal outcomes.

For example, A/B testing can involve:

  • Testing different messaging to see which produces more of the desired action
  • Comparing different images to determine if one resonates with consumers more than another
  • Showing people divergent offers to find out if one is more desirable than the other

These are just a tiny sampling of the things that might be evaluated using A/B testing.

However, it is vital for sellers to understand that A/B testing isn’t a one-time thing. Effective A/B testing demands repeatedly testing improvements until sellers arrive at the best possible version of a post, ad, email or whatever else is being optimized.

In short, A/B testing is an iterative process that builds upon the results of previous tests.

It is all about incremental improvement.

Why You Should A/B Test

Online retailers have tons of tasks already on their plates.

Is it really worth it to tack on something like A/B testing, thereby adding to the laundry list of other eCommerce necessities?

In short, yes.

The fact of the matter is that A/B testing allows retailers to improve the results experienced from their marketing efforts significantly, be they email social media or paid search advertising.

Netflix is a prime example of this principle at play.

Over the years, Netflix has transformed itself from an obscure mail order DVD website to one of the largest corporations in the world. Much of that success is attributable to the company’s dedication and insistence on perpetual testing.

As the Netflix Tech Blog reveals about its tests:

“Every product change Netflix considers goes through a rigorous A/B testing process before becoming the default user experience. Major redesigns… greatly improve our service by allowing members to find the content they want to watch faster. However, they are too risky to roll out without extensive A/B testing, which enables us to prove that the new experience is preferred over the old. And if you ever wonder whether we really set out to test everything possible, consider that even the images associated with many titles are A/B tested, sometimes resulting in 20 percent to 30 percent more viewing for that title!”

Experimentation (in the case of this discussion, A/B testing) is a critical practice for delivering a product to customers that resonates, engages and ultimately boosts customer satisfaction.

After all, if consumers continually receive a string of irrelevant emails or uninteresting social media posts, it is likely that their satisfaction with a brand will plummet.

The reality of the situation is that these kinds of companies are constantly testing everything and anything they can because they want to provide consumers with a compelling reason to continue using that service.

Therefore, A/B testing is merely a tool to create more engaging experiences and to earn brand loyalty from customers.

A/B Testing Examples

As it stands, there are tons of different A/B testing case studies online which show the effectiveness of comparing various permutations of email, social media posts, adverts and the like.

To better understand why you should A/B test various marketing efforts, let’s take a quick look at a few examples:

Blenders Eyewear

In a Klaviyo: BOS roundup post from 2018,  the company highlights the importance of A/B testing by highlighting one of the platform’s clients: Blenders Eyewear.

In the example given, Blenders Eyewear utilizes two drastically different visual styles in a birthday email while maintaining the same messaging, CTA and the like.

Of the two examples, the visual style with the larger image and color profile ultimately increased clicks by 33 percent and boosted email revenue by 48 percent.

This kind of example perfectly illustrates how altering a single component can have a drastic impact on campaign performance.

Strategyzer

In an example of Facebook Ads split testing provided by AdEspresso, the company shows how a brand that it worked with called Strategyzer aimed to increase ticket sales for an upcoming workshop.

In the initial, straight-to-the-punch advert, the company A/B tested images and audiences. In the end, it cost the brand over $4,400 and generated only a single sale.

From there, the company worked with advertising specialists to test out new ad copy that had a more story-driven approach and provided the audience with more information on what they would learn from the workshop.

By altering the tone and copy of the advert, the company generated 92 purchases at an average CPA of $123.45, thereby decreasing costs by nearly 97 percent.

BlaBlaCar

Turning now to an email marketing A/B testing case study from Braze, a client called BlaBlaCar sought to maximize the open rates for its email campaign.

To achieve this aim, the company began by forming a hypothesis indicating that email recipients would be more likely to open emails coming from a person’s name as opposed to a company name and that different types of names would perform better in different countries.

Ultimately, the brand tested its hypothesis in its French and Russian markets, altering the name of the sender from “BlaBlaCar” to an actual person’s name.

In the end, the company’s hypothesis proved to be accurate, as emails with the first name of a person generated 24 percent higher open rates than messages that were sent from “BlaBlaCar.”

With these short examples clearly illustrating why you should A/B test various marketing campaign components, let’s take a look at some tips for A/B testing for both email marketing and social media marketing.

Tips for A/B Testing: Email Marketing

When it comes to A/B testing email campaigns, there is a panoply of different elements that merchants can test to boost email marketing performance.

Understanding that, when testing email campaigns, retailers will want to experiment with:

Subject Lines

As far as A/B testing emails is concerned, the subject line is one of the most critical elements, as this is the first thing recipients will see.

Therefore, some of the ways that sellers might opt to A/B test subject lines include:

  • Questions versus statements
  • Including the recipient’s name versus a more generic subject line
  • Featuring general offers compared to more personalized ones
  • Presenting a clear subject line versus a clever one
  • Short or long subject lines
  • Word order

Email Length

Email lengths are a hotly debated topic within the industry.

While some contest that longer, more informative emails are better, others would argue that in the age of mobile dominance, shorter emails are the way to go.

The fact of the matter is that merchants should A/B test different email lengths to see what resonates most with their audience.

That said, a HubSpot piece on the ideal length of emails based on 40 million messages found that:

“Data suggests the ideal length of an email is between 50 and 125 words. Emails this length had a response rate above 50%. A similar study found emails with approximately 20 lines of text, or about 200 words, had the highest click-through rates. When in doubt, keep emails short and under 200 words.”

Again, retailers should test different lengths to see what works best for them, but this is a good starting point.

Visual Elements

Human beings are, by nature, visual creatures.

When speaking to why people are wired to respond to visuals, it is worth noting that the human brain is capable of processing images 60,000 times faster than text.

What this means is that visual elements can be an incredibly potent means of conveying an email’s message.

However, when speaking to tips for A/B testing, there is plenty to examine when using images.

For instance, retailers might try out different image sizes or formats. Alternatively, it is wise to experiment with various color palettes to see if one is more effective than another. If one is to believe in the use of color psychology in marketing, there is a good chance that disparate colors will produce variances in results.

Additionally, merchants might opt to test out different types of images, such as product shots versus action or lifestyle images.

Moreover, while images are undoubtedly useful, some sellers might want to test out how valuable images are in driving results, which leads us to our next point.

Images Versus No Images

While images can certainly have a positive impact on particular email marketing messages, it might not always be the best course of action, depending on the email’s design.

For instance, when testing out images in newsletters, some organizations have found that certain visuals actually distracted recipients from the core message of the email, thereby reducing conversion rates.

The fact is that while images are likely to be a net positive for most email campaigns, this is not a steadfast rule. Therefore, tips for A/B testing would dictate that sellers analyze the performance of elements that are typically viewed as an email staple, such as images.

Structure

The structure of an email is a critical component for grabbing a recipient’s attention and engaging them.

Generally speaking, the structure of an email should successfully guide readers to each element of the email, ultimately leading them to take the desired action.

Achieving this aim can take many forms. However, there are several email structures that are commonly used in the pursuit of click-throughs and conversions.

First, a single-column email design is quite popular with many retailers as the type of layout is not only easy to optimize for mobile devices, but it also provides a sequential flow to the email, culminating in a CTA at the end.

Another advantageous email layout is the inverted pyramid design. Using this structure, the email naturally guides the reader’s eye down to a point at the bottom of the email, which is where sellers will want to place their CTA.

This kind of structure enables retailers to highlight various pieces of content or offers, thereby potentially leading to increased traffic and sales.

Finally, the zig-zag email layout is a design that allows sellers to create distinct sections of the email in a visually appealing way. Moreover, this structure easily accommodates multiple CTAs without cluttering up the overall design or confusing recipients.

However, to know which email structure will work best for sellers, it is necessary to A/B test different layouts.

Try building out the same email in two different ways to establish which is more effective in producing results.

Copy

Messaging is (naturally) one of the most crucial components of an effective email.

After all, if a recipient opens an email, finds it visually appealing, yet finds the copy uninspiring, they are likely to click away.

Retailers can discover the type of messaging that works best with their audience by A/B testing different copies. There are a number of features that can be played with to optimize campaign outcomes.

As mentioned earlier, copy length is something to which merchants should pay considerable mind. In addition to that, retailers can potentially use personalization to increase email impact.

The tone is another element that sellers will want to test.

The tone of an email’s copy can have a meaningful effect on the number of click-throughs a campaign generates.

For instance, instead of using straightforward messaging, sellers might opt to inject some humor or positivity into their copy to see how recipients respond.

The only way to find out which tone resonates with subscribers is to A/B test different styles to see what performs best.

Calls-to-Action

Naturally, CTAs are a vital element to email marketing campaigns as they can help to increase (or not) click-through rates by making clear to consumers the next step they should take.

CTAs should be prominent and bold, standing in contrast to the rest of the email. If the color palette of an email blends too well with a call-to-action, readers are liable to miss the button and click away from the email.

Given how critical of an element CTAs are to effective email campaigns, it is wise to test the color, placement and size of a CTA.

Additionally, merchants can also experiment with CTAs that take the form of buttons and those that appear as links.

That said, no matter where retailers land regarding these elements, it is also critical to consider the copy of a CTA as this can have a dramatic impact on campaign results.

While many merchants opt to employ rather generic CTAs like “See More” or “Buy Now,” using more unique, actionable copy like “Get Yours Before They’re Gone!” can serve to motivate a more significant portion of a seller’s audience to click-through and convert.

However, just like all of the other elements listed above, this is not a hard and fast rule. Therefore, retailers will want to A/B test CTA copy to see what email recipients enjoy most.

Send Times

It is essential to understand that tips for A/B testing don’t just apply to copy, colors and CTAs.

What is as vital (if not more) as all of these elements is the time that sellers choose to send an email. After all, it matters not how well an email is optimized if it never gets opened.

Therefore, retailers should extensively A/B test various send times to determine what time and days of the week receive the most engagement from recipients.

Of course, there is a ton of data on this floating around online. For instance, OptinMonster’s study on the best time to send emails reflects that Tuesdays and Thursdays are the best days to send marketing emails. As for times, the company identifies some key periods, including:

  • 8 am
  • 1 pm
  • 4 pm
  • 6 pm

However, as the piece goes on to state:

“There’s no consensus on the exact day or time to send emails. In reality, these slots don’t always work. Every email list is made up of a different group of people with different habits. So, your best send time may or may not be the same as other businesses.”

Therefore, the only thing that sellers can do is use this information as a benchmark and A/B test their emails to understand their own audience’s habits and behaviors regarding opening marketing emails.

With these elements outlined, let’s take a quick look at some tips for A/B testing email campaigns.

How to A/B Test Emails

Understanding some tips for A/B testing, such as which elements to test and how to actually run an A/B test, are two different things.

Therefore, when running such an experiment, merchants will want to adhere to the following steps:

Create a Hypothesis

Before creating any email permutations, sellers should start by creating a premise to be tested.

For instance, going back to the BlaBlaCar case study, the company created a hypothesis wherein they expected emails sent from a person with an actual name to perform better than those sent from the business name.

By starting out with a supposition, merchants will understand which elements to test as well as how to interpret the outcome of the test, which leads us to the next point.

Determine What Success Means

Prior to sending out different versions of an email, sellers will need to decide what they will be testing for and what will be considered a success.

Therefore, merchants will first need to select a metric to measure and improve, such as email deliverability, open rates, click-throughs, conversions, revenue, etc.

Next, take a look at the company’s historical rate of success for the chosen metric and determine what success would look like based on current standings and what the brand ultimately wants to achieve.

Select a Single Element

To ensure clear results, retailers will need to choose a single element to modify and test.

If sellers select more than one component, then the results could get muddied. However, it is possible to change more than one aspect if retailers set up a multivariate test, which VWO defines as:

“A form of A/B testing wherein a combination of multiple page elements are modified and tested against the original version (called the control) to determine which permutation leaves the highest impact on the business metrics you’re tracking.”

Therefore, if merchants are doing an A/B test and not a multivariate test, they will want to keep all elements exactly the same, excluding the one being tested.

Make the Test Statistically Significant

To ensure that tests provide decent results, merchants are advised to include at least 5,000 recipients, if at all possible.

If sellers do not have that many email subscribers, then just include as many as possible.

Analyze the Results

After running the email campaign A/B test using the different versions, it is time to take a look at the results.

There are several different metrics that sellers will want to examine, including:

  • Open rates
  • Click-through rates
  • Conversion rates after landing on the website

No matter which metrics sellers have chosen to measure during the test, it is vital to analyze all of these performance indicators.

Rinse and Repeat

After determining the outcome of the test, apply the winning element and proceed to conduct another A/B test—this time testing a different campaign component.

It is worth noting that if the outcome of the campaign came out unclear or if retailers are feeling a bit too overwhelmed by the process, it would be wise to partner with email marketing experts who can effectively structure and test email campaigns to produce optimal outcomes.

With that information covered, let’s go ahead and turn our attention to tips for A/B testing various elements of social media efforts.

Tips for A/B Testing: Social Media Marketing

Where A/B testing for social media is concerned, there is a significant number of things that can be examined and analyzed.

For instance, merchants might test how a specific social media post with an image compares to one without an image. Similarly, retailers might opt to test out various paid social media advertising elements such as:

  • Copy
  • Images
  • Offers
  • CTAs

Moreover, merchants can do this for their posts and ads on any social media platform.

That said, some of the social media tips for A/B testing elements include:

Post Text

Much like with email marketing campaigns, the style and language deployed in social media posts are undoubtedly worthy of close examination.

When experimenting with a post’s text, sellers can test:

  • Post length
  • Use of emojis
  • Use of punctuation
  • Tone (casual, passive, authoritative, etc.)
  • Post style (statements, questions, quotes, etc.)

The fact is that the way in which merchants structure their copy and its essence is likely to have a considerable impact on engagement rates.

Content Previews

When linking to content on social media, the headline and description are vital to test as these components will play a considerable role in people’s decision to click-through or keep scrolling.

Given that it is possible to edit the headline in the link preview, it is important to A/B test this element. Moreover, the headline does not need to match the one on the website exactly. However, retailers should not deviate so much in their testing that the headline becomes deceiving.

Calls-to-Action

Similar to email marketing, the CTAs used on social media are incredibly important for driving consumers to action.

Getting this bit right is vital as it can help to convert social media traffic into customers. Therefore, A/B testing calls-to-action for various posts and paid ads is well worth a retailer’s time and energy.

Use of Images and/or Videos

As mentioned a moment ago, testing the use of images versus video is something that merchants should strongly consider.

While most would think that a video post would vastly outperform an image post given how popular the former has become, this is not always going to be the case.

For instance, when referencing data on video vs. images in Facebook Ads, Databox reveals:

“Next to offers/promotions, images are the top content type to influence people to click on a link on social media. Images are most likely to influence 25 percent of people to click through to a website from social media. Videos are third, at 16 percent. Videos are still a successful content type, but images are more successful at driving clicks.”

However, there is also a multitude of studies that show video to be most effective in generating clicks and conversions.

Therefore, all merchants can really take from this is that there is no conclusive answer as to which is more effective.

This is exactly why you should A/B test video and image posts to establish which works best for your company. When testing images and video, retailers can examine:

  • Text alone versus posts with images or videos
  • Static images versus GIFs
  • Static images versus video content
  • Length of video content

There are a slew of different test variations that retailers could implement to determine which elements will create the most effective post or advertisement.

Hashtags

Hashtags are an essential tool for helping to extend the reach of social media posts. However, depending on a seller’s hashtag strategy, there is a chance that they could be annoying followers and reducing engagement.

The only way for retailers to be sure is to A/B test different posts and ads utilizing other tagging tactics. Some of the ways merchants can test hashtags include:

  • A single hashtag versus multiple hashtags
  • Different industry hashtags to see which boost engagement rates
  • Hashtag placement (at the beginning, in the body or at the end)
  • Branded hashtags versus non-branded tags

While hashtags may not be the most critical element to social media posts, they are instrumental in increasing a post’s reach. Therefore, it is wise to spend some time figuring out which hashtags are most fruitful.

Posting Time

Much like finding the ideal time to send emails to subscribers, there are also sweet spots of when to post on social media to receive the most reach and engagement.

Naturally, posting times will not only vary from brand to brand but from social platform to social platform as well.

Much like with email send times, there is existing data on the best time to post social media updates. Per Social Media Today, here are the best times to post on various social media websites:

  • Facebook: 1 pm – 4 pm, 6 pm – 10 pm (Tue, Thu, Fri, Sat, Sun)
  • Twitter: 8 am – 10 am, 11 am – 1 pm, 5 pm – 6 pm (Mon – Fri)
  • Instagram: 11 am – 1 pm, 7 pm – 9 pm (Mon, Wed, Thu)
  • Pinterest: 12 pm – 2 pm, 7 pm – 10 pm (Sat, Sun)
  • LinkedIn: 8 am – 10 am, 5 pm – 6 pm (Tue, Wed, Thu)
  • Reddit: 6 am – 9 am (Mon, Sat, Sun)

Of course, this information also varies depending on the source. Therefore, it is critical for sellers to A/B test this by posting at different times of the day to see which days and times produce the best results for each merchant’s business.

Ad Formats

For those who are doing YouTube advertising, TikTok advertising or running paid ads on any other social media platform, it is critical to test out different ad formats to see which are most effective for the offers and contents being pushed.

For instance, those who are trying to reach target shoppers on Facebook should possibly try out carousel ads for product announcements. At the same time, retailers can run the same ad campaign using dynamic product ads to determine which is more effective for generating eCommerce sales.

By A/B testing various ad formats, merchants can pinpoint those that resonate most with an audience, thus producing the best outcomes.

However, no matter which ad formats retailers elect to try out, just be sure to protect your ad account from hackers.

Target Audience

While merchants can certainly A/B test their target audience, this tactic works a bit differently.

Instead of showcasing advertising variants or post variants to similar groups of consumers, merchants will show the same advertisement to two different audiences to see which one responds better.

For instance, retailers might A/B test Messenger Ads with different audiences, with the results showing that one group enjoys these kinds of ads while the other finds them invasive.

If it weren’t already clear enough why you should be A/B testing, the only way to find out how an ad will perform with different audiences is to conduct a split test.

How to A/B Test Social Media Posts and Adverts

The basic process of A/B testing varies very little from split testing email marketing campaigns.

However, it is worth noting that Facebook has a built-in split testing feature. That said, many other platforms do not. Therefore, merchants might opt to utilize one of many different A/B testing tools, such as:

That said, the basic structure of A/B testing social media ads or posts is as follows:

Form a Hypothesis

As with testing out email campaign elements, sellers will want to produce an assumption as to what they think will happen by altering a specific aspect of their post or ad.

Establish Metrics to Measure

With an assumption in hand, retailers must determine which metrics must be measured and at what point the test will be considered a success.

Retailers might opt to measure metrics such as:

  • Comments
  • Shares
  • Click-throughs
  • Cost-per-action
  • Conversions
  • ROAS
  • ROI

Pick an Element to Alter

Per A/B testing guidelines, sellers will now select a single element to test.

Again, be sure only to test one component. Otherwise, retailers will need to create a multivariate test, which is different from a basic A/B test.

From there, create the two variations and run the experiment

Analyze the Results

After the test is complete, retailers will select the winning permutation and implement the optimized element in future campaigns.

Rinse and Repeat

With the initial A/B test complete, retailers will now conduct another A/B test, this time targeting a different component.

Repeat this entire process as many times as desired to produce an optimized social media post or advertisement.

Final Thoughts

A/B testing is a vital component of eCommerce success. However, understanding why you should A/B test is not enough.

Given how incredibly competitive the industry has become in recent years, it is imperative that merchants implement tips for A/B testing to every aspect of their marketing efforts.

A/B testing enables retailers to take a good concept and turn it into something extraordinary. As a result, merchants can elevate their brand awareness, online visibility, sales revenue and market share.

That said, building out a practical A/B test can sometimes be a complicated, tedious endeavor.

If your brand would like to ensure that its marketing campaigns are optimized to the fullest, then reach out to Visiture’s paid advertising experts.

Our team of eCommerce advertising professionals can help to set the framework needed to test every element of your search and social advertising campaigns, thereby producing the most profitable outcomes possible.

 

Join 150+ Leading eCommerce Brands

And see how Visiture can grow your revenue online through award-winning transactional focused marketing services.

Stop Waiting!

Receive a Free eCommerce Marketing Audit Today!

Audit Emails

  • Submit Email

Popular Articles.

Data-Driven Marketing + Creative Commerce = Results.

Let’s Bring Our Teams Together and Connect You to Your Ideal Customer.