The Small Business A/B Testing Guide
Or, a more modern iteration for our small business owners: Data, data, data!
Without necessarily even trying, if you run a website for your biz, you’re probably stacked with data.
Data on visits, consumer behavior, product or service sales…and the list goes on.
Having all that information at your fingertips isn’t a terrible thing, but if you don’t know how to act on it, you might as well be drowning in it.
A/B testing is a low-stakes method to make lots of business data actually work for you.
Want to keep growing those month-over-month sales numbers you’re seeing? A/B test different iterations of your product/service page, pricing, and even features.
Feeling a little down about your homepage’s call to action clicks? Benchmark the current data, make your tweaks, and run your A/B test to see how next month’s numbers compare.
A/B testing like this is an approachable way for most busy business owners to turn data into big improvements, quickly and affordably.
And you don’t even have to be an expert going into it — because this guide will make you one. Starting now.
What Is A/B Testing?
A/B testing —which you may see referred to by many other names like split testing, bucket testing, and the very similar but slightly unique multivariate testing approach — is simply a way to compare two different versions of something: version A vs. version B.
To A/B test something such as a marketing email, you would send the original copy and design (usually version A) to about half of your audience, then a slightly-tweaked variation (that’d be version B) to the rest of your recipients.
Other times, A/B testing might look like running two different versions of something at the same time like two product pages, two social media ads, etc., and serving them to two different segments of your audience.
Later, we’ll talk about some tools that’ll help you pull this off.
No matter your approach, the goal is the same: seeing which option gets better results! You can A/B test anything from your marketing to your website functionality to your products or service offerings. Typically, A/B testing is something you would do on an ongoing basis as and when you have time, making incremental changes and improvements.
And here’s a pro tip many a business owner has probably had to learn the hard way (we get it, patience is hard) — for the best outcome, it’s important to change just one, measurable element of version B each time you run a test. This way, you know exactly what made an impact and can more readily duplicate your success going forward.
Pretty cool concept, right? We hoped you’d think so, because we can’t wait to dive into the deets.
The Power of Split Testing for Small Business Websites
Sometimes it seems like the most celebrated business and marketing strategies on the internet aren’t really for smaller and earlier-stage operations.
However, A/B testing has some unique benefits that make it a great fit for our more time and budget-conscious readers out there.
Make the Most of Your Budget With Proven Optimizations
Instead of spending (and potentially losing) big money on a complete and misguided website or product overhaul, small businesses can make DIY, incremental improvements through A/B testing.
A/B experiments help make sure that only the changes that have been proven to lead to positive results are implemented.
Beat Competitors Thanks to Deeper Customer Insights
Regular testing across the customer journey is an affordable way to gather insights into customer behavior, which is difficult to come by in any other way.
Over time, this is sure to boost your understanding of your customer base in a way larger and less-invested competitors can’t. With that information, even the smallest business can tailor their offerings and experience to their target audience and outpace the competition.
Find All-Important Product-Market Fit
Product-market fit is crucial for small businesses. After all, if there isn’t an audience for your offering, you won’t be able to survive for long!
A/B testing plays a crucial role in finding that product-market fit, as it helps businesses incrementally fine-tune their offerings based on the real-life user feedback they’ve observed through experimentation.
Easy Access Means A/B Testing Benefits for All
One of the greatest advantages of A/B testing is that it’s within your reach.
It’s an easy concept to understand, and most, if not all of the strategies can be rolled out yourself if you have the time. The tooling to pull off the more advanced tactics is usually affordable and accessible, and the impact that can come from doing it right can be high reward —all with a relatively low risk of loss!
What Do A/B Testing Analytics Look Like in Action?
HubSpot shares how they ran A/B testing on the homepage for HubSpot Academy to help visitors better understand and engage with the offering.
Even with a whopping 55,000 page views, a minuscule 0.9% of people were watching the homepage video. Additionally, chats indicated visitors were generally struggling to understand the messaging.
So HubSpot ran more of an A/B/C experiment with one control and two variants. (Yep, you can do that, too!)
The first option featured more imagery in a creative layout and an animated headline. The second one was all about boosting the color and movement, but also incorporated animations.
And the results were interesting. While variant B outperformed the control, C actually underperformed it. The team was able to extrapolate from their experiment that option B would help them add 375 more sign-ups to their offering every single month!
This is just one example of how A/B testing can be surprising, approachable, and truly impactful. Inspiring, no?
Where Can I Run A/B Experiments? 5 Core Areas
Not to overwhelm you, but just about anything that you can change — you can test.
For the smaller business owner and website operator, here are the most relevant areas to consider A/B testing:
Your Offering
Whatever your product, service, or other offering type, chances are it’s rich for comparison testing. You can tweak the pricing, shipping details, checkout page flow, product descriptions, pictures, and even the features of the offering itself to see what best boosts sales.
Your Website
Now this area is really ripe for A/B testing!
Each page likely consists of web copy, images, buttons, and the layout in which they’re all arranged. Then there are the more universal elements, like the design that unites all the pages, the menu, and the footer that helps folks navigate your site, font, speed, how responsive it is on mobile devices, etc. Any of these elements can be improved upon — incrementally — to improve your key website performance metrics whether they’re clicks, conversions, bounce rate, etc.
Blog Content
If blogging is part of your content marketing strategy, you know how time-consuming (or cash-intensive if you hire it out) it can be. That’s why we recommend playing around with A/B testing on your blogs to make the most of them.
The first thing most marketers will tweak is the headline, but there are lots of pieces to consider. What parts of a blog post are easy to adjust but can make a big impact?
Think subheaders, imagery, links, buttons, calls to action, and length. If you really want an overhaul? Even the content itself.
Email Marketing
As we already mentioned, marketing emails have a variety of features that make them perfect for little tweaks and tests. Email subject lines, copy, images, and buttons are all great areas to switch up to measure metrics like open rate, click-through rate, etc.
Advertisements
Running ads on social media, search engines, and other websites has always been a practice in experimentation. So why not introduce A/B testing to the mix?
Similar to emails, ads can be tweaked and compared based on their features. They have headlines, content, usually some kind of imagery, and a button or some other call to action. The best way to find which combo of these elements is most clickable to viewers is through ongoing testing.
Facebook and Google, for example, have built-in A/B testing into their ad creation interfaces.
If you were to go Googling around for A/B testing tools, some of the top options that would probably come up would include AB Tasty and Optimizely. These are big hitters in the field, and have the customer list, complex functionality, and custom pricing to match.
In this section, we instead want to highlight the software that’s better suited for smaller businesses, based on features, focus, and pricing.
1. Zoho PageSense: Testing, Optimization, and Personalization in One
PageSense is one of the many offerings in the Zoho universe. Like many of the platform’s tools, it’s a good fit for small to medium-sized businesses because it’s easy to use yet robust.
Features include website analytics tracking, analyzing and optimizing via A/B testing, and even personalization based on real visitor data and user behavior.
Yearly, Zoho PageSense users can get started for just $12/month, for up to 10,000 monthly visitors.
2. Hotjar: Tons of Tools for All Things Website Testing
Hotjar promises to go beyond website analytics by providing a variety of tools for testing and delivering high-converting web experiences.
For A/B testing in particular, Hotjar offers heatmaps and recordings of visitor sessions that help you identify the impact of your tweaks.
You can start using these features for free, and Hotjar’s pricing is reasonable as your website visitor sessions increase and you add on more features.
3. Apptimize: Cross Platform-Focused Testing
In our multi-channel world, users engage with your business across lots of platforms. And Apptimize gets that.
With Apptimize, you can set up seamless experiments, track impact by user and by channel, and optimize the digital experience across mobile, web, and beyond.
Unfortunately, Apptimize is not transparent about their cost, so we do have to assume they’re on the higher side. It appears you can set up a free account or get a demo to see if the offerings and price are right for you.
4. VWO: Find Opportunities for Front and Backend Enhancements
VWO is probably the most complex offering on our list, offering end-to-end support for optimizing the user experience, but that’s why we wanted to include them.
For our website owners who are more tech-savvy —or have a tech-savvy support team —VWO can even dive into your website code and use it to conduct server-side experiments. This will help you identify opportunities for iteration in speed, security, and other areas that lighter-weight software can’t touch.
Because of its advanced nature, working with VWO is more pricey. You can get started with their testing feature for free. The next plan for web testing starts at $219 per month, annually, for up to 10,000 monthly tracked users. When charged yearly, adding the server-side testing will be an additional $553/month.
5. Lucky Orange: Easily-Integrated Testing Tooling
Lucky Orange wants to help busy owners get back to business. Same here!
To fulfill this goal, they offer plenty of features that fit right into your flow to test out the changes you make on your website. These include surveys, heatmaps, session recordings, and live chat features so customer support can engage with visitors in the moment.
If you’re big on seeing how your actual consumers react to A/B test variants in real time, Lucky Orange will be a helpful tool to keep in your kit.
Lucky Orange is free for up to 100 monthly sessions. After that, pricing begins at $32/month annually, for up to 5,000 monthly sessions.
6. CartFlows: A/B Testing for E-Commerce WordPress Sites
CartFlows is a unique WordPress plugin that focuses on increasing sales through smart upsells, dynamic discounts, cart abandonment recovery, and…you guessed it! A/B testing to perfectly optimize your offerings.
CartFlows makes it easy to spin up multiple variants, run your tests, and implement your changes based on your findings in just one click. Pricing is also super simple, and the pro package that includes A/B testing starts at $299 per year.
6 Simple Steps for Running Your Multivariate Test
Realistically, this process is going to look different based on things like how much time and money you want to spend, how your business and website function, your skill set, the tools you choose to use during this process, and perhaps a billion other variables.
So, we’re going to keep it high-level but directional, and talk through the basics of what the A/B testing flow might look like. Feel free to add any unique touches you need!
1. Set Your A/B Testing Goal(s)
All smart business undertakings start by clearly identifying what you want to achieve.
Is your goal to increase signups for an email marketing newsletter, boost sales, reduce bounce rates, or get more clicks on a call to action button?
Defining one or more (but not too many) specific, measurable, and ultimately achievable goals is critical because it helps you hone in on what matters, what tools will get you there, and how to measure success.
2. Choose What You’ll Test
Next, it’s time to decide which element of your website you want to test to hit your goal(s).
It could be something small like a blog headline, or something bigger like the anchoring color of your overall website design.
This is where it’s important to remember to change and test only one element at a time so that you can accurately determine what exactly impacts your results.
3. Build Out Your Variations
Now for the fun part. Creating two versions of the element you’ll test.
Version A should be your original (control) version to keep it simple, and version B should be your test variation.
So, if you’re testing out a new headline for the call to action section of your homepage, option A will stay the same. Maybe that’s “Sign up for our Newsletter!”
And option B may be tweaked to read “Join our Exclusive Mailing List Now.”
4. Conduct Your A/B Testing
If you want to make changes to your website and measure your metrics manually, you can use website analytics (like the Google Analytics tool) as well as tracking sales numbers over time.
If you’d like to save some time, choose one of the A/B testing platforms we recommended above. This option is more costly, but it could offer some automation opportunities and insights. Not to mention, a lot of saved manual effort.
You want to let your test run for sufficient time to gather meaningful data. We’ll talk about this more later, but at least two weeks is ideal. Why? To make sure you’ve compared enough instances to see the true difference between versions A and B.
5. Implement the Best Version
Now that you know which version is more effective in moving toward your starting goal, it’s time to implement it live on your site, email marketing platform, or wherever.
For future A/B testing projects, this is now your variable A!
6. Keep Iterating
A/B testing should not be a one-time task but an ongoing process of optimization that you revisit every time you have the bandwidth.
Once you’ve tested and implemented a successful change in one place, you should have the confidence and know-how to experiment and improve upon other areas and elements of your website.
By continuously repeating this entire process, you can ensure your user experience is always evolving, getting better at meeting your goals, and boosting every business’s key success data point — revenue!
Mistakes To Avoid With Your A/B Test Results
Naturally, A/B testing is not infallible, and first-timers should be aware of some of the most common mistakes beginners make.
Taking It Too Seriously
Of course, sample size is critical to the success of any experiment. Some smaller businesses simply don’t have the volume of users it takes to come up with statistically significant results.
Keep in mind that ideal sample size is relative to the study at hand. But generally (and ironically), with a smaller population, the ratio is higher. So, if your total audience is around 800 people, a sample of 30% or 240 people is exemplary. For a larger group of 5,000, a smaller ratio of about 10% or 500 folks can be enough.
Now, if you don’t have those kinds of numbers, there’s nothing wrong with still running your A/B experiment. But remember to take those results with a grain of salt, and don’t sink time or money you don’t have into testing or tweaking indefinitely.
Reacting Too Soon
When you’re strapped for time, or just excited, it’s easy to stop an A/B test too early and act on the immediate results. The problem is that anomalies can happen, so the more testing time you allow for, the better.
But when is it enough?
Like many things, there isn’t a strict rule here. Oftentimes, at least a few weeks is recommended. The higher the impact of the change, the longer you may want to run your test. For smaller sample sizes and/or more complex tests, longer is also better. If you notice you’re getting similar results over and over again, you’ve probably maxed out your test’s potential.
Many A/B testing tools will help guide you through the process, so don’t get ahead of yourself and worry about this too much.
Catching Metric Madness
Trying to measure way too many metrics in one A/B test can easily lead to confusion and overwhelm. Plus, monitoring tons of data points increases the chances of seeing random patterns that don’t actually mean anything.
Instead, it’s better to pick a few very important, impactful metrics that you can stick to measuring throughout the course of an A/B test, such as those that measure marketing success, engagement, and conversions.
What Other Website Metrics Can We Help You Boost?
We hope we’ve illustrated what makes A/B testing a simple yet powerful method for boosting your small business website metrics.
It’s one of the most approachable and data-driven approaches that empowers businesses to make informed decisions, grow conversion rate and customer satisfaction, and eventually improve sales.
But of course, that doesn’t make it effort-free!
If you don’t have the time or desire to go after A/B testing, or some other dream project you have in mind on your own, we can help. Our Pro Services team at DreamHost helps website owners with everything from website building to design, management, and marketing.
As your partner in website improvements, we handle the technical elements so that you can concentrate on your business.
This page contains affiliate links. This means we may earn a commission if you purchase services through our link without any extra cost to you.
