An A/B test is an online experiment with two variants and a random 50% split of users between two variants to determine the better-performing option. An A/B test contains a description of the variants and past performance. A/B tests can be conducted in Google Ads under experiments in the navigation menu.
In general, A/B test results either contain discrete or continuous metrics. Discrete metrics have specific values that can be counted or are binary, like on/off or true/false settings. In computer programming, binary values are either 1s or 0s. This is the foundation of bit manipulation. Bit manipulation is well beyond the scope of this blog post. Generally, a value of one indicates true, on, or the program returns without errors. While a value of zero indicates false, off, or the program is returning with. However, this is not always the case. In C or C++ programming, the main function generally returns zero, indicating that the program returns without compilation errors. Continuous metrics are used for comparison with other data and don’t always have defined quantities.
Planning for A/B testing
As stated, an A/B test describes the variants and includes past performance data. An A/B testing plan includes a hypothesis, variants, metrics, current performance and expected results, and other testing details.
Here is the process checklist for A/B testing recommended by Google:
- Choose a variable to test
- Identify the goal of the test
- Identify the control of the variant
- Verify your test is the only A/B test running for a campaign or website
- Split the sample groups randomly. The A/B testing platform can manage this.
- Determine the required sample size. The A/B testing platform can also manage this.
- Decide the confidence level for statistical significance.
- Select an A/B testing platform.
- Test both variants at the same time.
- Allow the test to run long enough to collect enough data.
- Review test results against your goal for the test.
- Decide on the appropriate actions to take based on the test results.
- Plan for additional A/B tests, if required.
Types of A/B testing
There are many types of A/B tests. A basic A/B test only tests a single variable or change. This blog post will discuss two: redirect and multivariate tests. A redirect test enables you to test separate web pages against each other. For redirect tests, the variants are identified by a URL or path instead of by a certain element on a page, such as a banner. Redirect tests are helpful if you want to test two completely different landing pages or are involved in completely redesigning a website.
A multivariate, sometimes called a multivariable test, simultaneously tests variants of two or more elements on a page to determine which combination yields the best results. A multivariate test identifies the most effective variant for each element and provides insights into how the combined variants work together. This allows you to identify the best combination of variants.
Tools for A/B testing
Tools for A/ B testing are available through many platforms by many companies. These include Google Ads, AB Tasty, Convert, Crazy Egg, Instapage, Optimizely, Unbounce, VWO, and Hubspot.
To view more blog posts about digital marketing, SEO, Divi, or other related topics, visit our website’s blog section. Fill out our contact form now to contact us and get a FREE analysis of your website.
