A quick guide to A/B testing tools and tips

Dhanya Arumughom -


My last article 'A/B testing Process explained from the scratch' was all about A/B testing and the complete process involved. This article aims in taking you to the next step by listing out the most recommended tools for A/B testing and also few tips or best practices that you need to keep in mind while A/B testing.

A/B Testing Tools

A/B testing, as you know plays a very important role in the entire Conversion Rate Optimization (CRO) process. There are a number of A/B testing tools available in the market now and it is natural to get lost in deciding which one is best for you. All these tools are loaded with a ton of features and come at different pricing.
All A/B testing tools essentially have features like split traffic among the variants, calculate conversion rates, measure the statistical likelihood that one variant outperforms other, etc. Some sophisticated tools might go a step further by allowing you to run targeted or segmented tests by displaying different versions to customers based on demographics, device type, referral source, etc.
Another major criterion that you might have to look for is whether you want a server-side tool or a client-side tool. Server-side tools render code on the server level and send a randomized version of your web page to the user without making any modification on the visitor’s browser. On the other hand, client-side tools send the same page but JavaScript on the viewer’s browser manipulate the appearance of both the original and variation. Client-side tools are easier to use without much development efforts needed and saves a lot of time. Server-side tools require more coding but can often be more robust. VWO, Optimizely, Adobe Target, etc are some of the client side tools while Google Analytics Experiments is a server-side tool. Some tools like Conductrics comes with capabilities of both too.
You should choose the tool which suits you best based on your testing needs, company size, budget and resources available. With too many tools available, here we have come up with a list of tools to help you select the tool that suits your organization.

To 10 Tools for A/B testing

1. VWO

VWO is considered to be one of the easiest A/B testing tools. It allows you to change various elements of your web page easy to create multiple variants. It lets you track different conversion goals like revenue, signups, clicks, etc and gives you all the required statistical data and results and can generate wonderful reports which can make analysis easy.
Website: https://vwo.com/

2. Optimizely

Optimizely is a very popular, simple and widely used testing tool for A/B testing and multivariate testing. It helps in analyzing data and generates clear reports. The tool integrates well with other tools. It also supports visitor segmentation.
Website: https://www.optimizely.com/products/experimentation/

3. Adobe Target

Adobe target is yet another easy to use tool which allows for A/B testing and multivariate testing. It also has AI powered automation capability. It can be synced with Adobe analytics for better targeting and more precise test filtering. The tool also allows you to perform segmented tests based on visitor location, target content to specific audience, etc.
Website: https://www.adobe.com/in/marketing-cloud/target.html

4. Oracle Maxymiser

Oracle Maxymiser is part of the Oracle Marketing cloud and supports A/B and multivariate testing. It helps you gather data from any marketing platform. It also helps launch A/B or multivariate tests and segment traffic.
Website: https://www.oracle.com/marketingcloud/maxymiser.html

5. Google Analytics Experiments

A tool that can be used to create A/B tests from withing Google Analytics.This tool comes for free but demands some coding abilities as it is a server side tool unlike other tools like Optimizely which is completely WYSIWYG tool.
Website: https://developers.google.com/analytics/solutions/experiments

6. AB Tasty

AB tasty is an all in one conversion optimization platform for getting data insights, A/B testing and multivariate testing with many advanced capabilities to reach to your target audience. For A/B/n or multivariate testing it has an easy to use visual editor to create your experiments quickly. It also supports server side tests through API and is compatible with React, Angular and other JS frameworks.
Website: https://www.abtasty.com/

7. Kameleoon

Kameleoon offers A/B testing and optimization services. It lets you validate your ptimization hypotheses and understand what your visitors really want. It has a simplest interface and you can modify your web page elements with just few clicks.
Website: https://www.kameleoon.com/en/ab-testing-tool

8. Conductrics

This is one great tool which helps users come up with different variations and test them via A/B and MVT testing. The tool has an easy to use editor. It uses industry standard testing methods to randomly select variations for each visitor, and helps us in learning which works best for whom. It also allows targeted selections for segment testing.
Website: https://conductrics.com/

9. Convert

It claims to be one of the high return A/B testing solution which allows you to optimize your website for more customer value. ConversionXL studies shows that Convert turns out to be one of the fastest testing tools out there. It offers you limitless testing as most of its plans comes with unlimited projects, tests, and domains.
Website: https://www.convert.com/

10. Monetate

This is yet another wonderful tool which allows you to quickly build and run A/B and multivariate tests to improve the entire user experience and your conversions. It has an easy to use interface with a powerful backend testing and segmentation engine. It has built in measures of statistical significance and provides a dynamic testing that automatically adjusts traffic allocation to ensure you are driving the maximum ROI all the time.
Website: https://www.monetate.com/products/test-and-segment
That was the list of few tools with excellent features to help you with A/B testing. Now, whatever be the tool you choose, the real success of A/B testing comes only if you have a structured approach for the entire process. Here are a few tips to keep in mind during the entire A/B testing process.

Top 9 tips and Best Practices for A/B testing

  1. Ensure you have collected enough quantitative and qualitative data from reliable sources before you start with your A/B tests.
  2. Give importance to the sample size when you do A/B testing. If you don’t perform your test on enough people, you might not be getting reliable results.
  3. It is better to test one variable at a time to precisely isolate the impact of the variable under test.
  4. Get your hypotheses right before you start testing so that you know the scope of your test.
  5. Adapt the number of versions under test and traffic you get. If you get less traffic only, the more number of versions you have, the test might take longer to give you any significant results.
  6. Ensure you run the tests long enough. Even if a test rapidly displays statistical reliability, it is advisable to run it at least for a week or two.
  7. If you feel your customer base is made up of segments, conduct A/B test based on those segments and analyze your results accordingly. This is needed to filter out outside influences to your results.
  8. Choose the right tools and platform to run your A/B tests. It is a good idea to conduct an A/A test before you start with your A/B test to check the accuracy of your A/B testing tool.
  9. Keep testing continuously to truly optimize your web page in order to keep improving your conversion rates.
    The articles in this blog are not a complete guide to A/B testing, but a good place to get started if the term is new to you. After all, knowledge can be acquired only by iterative learning and from experience. So dig in and get started with testing!