A/B testing (split testing)

Contributor(s): Ed Burns

A/B testing, sometimes called split testing, is an assessment tool for identifying which version of something helps an individual or organization meet a business goal more effectively. A/B testing is commonly used in web development to ensure that changes to a webpage or page component are driven by data and not personal opinion.

A/B tests are blind studies and the participants are unaware that a test is being conducted. In a typical A/B test on a Web page, version A is the control and version B is the variant. During the test period, half the visitors to the Web page are served version A of the Web page, which has no changes, and half are served version B, which includes a change that is designed improve a specific metric such as clickthrough rate, conversion, engagement or time spent on page. End user behavior, which is gathered throughout the test period, is analyzed to determine whether the control or the variant performed better for the desired goal.

Online streaming service Netflix is a well-known for its extensive use of A/B testing. The company uses A/B split testing in everything from fine tuning its streaming and content delivery network algorithms to selecting what images should be associated with a specific title. According to Netflix, selecting the right image can result in 20% to 30% more viewing for a specific title. 

This was last updated in August 2016

Continue Reading About A/B testing (split testing)

Dig Deeper on Customer analytics

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

Just a clarification. A/B Testing pre-dates the internet. I used it as far back as the 1970's and it wasn't a new concept then. It is used in many market research studies where alternate deliverables or paths can be used and then evaluated as to their effectiveness.
When fast and reliable release management policies and controls are in place the company can afford to conduct A/B testing. The trial feature can be selectively deployed and quickly rolled back. So the focus must be first on setting robust automated deployment pipeline, branching control, and effective rollout / rollback.
Here you can find an ab test calculator . It is free to use and beside this there is an  ab avarage test calculator and a correlation test tool.

This testing method goes back to the root of IT. It just goes to show some methods still work well today after many years.
What are your experiences conducted A/B testing?
A/B testing is very useful method not only for marketers but for developers like me too. I am using Raisemetrics for conversion optimization and the results are fine. Thanks for the useful article, it was helpful in some poitnts.