What Is an A/B Test? | Marketing Dictionary
DICTIONARY · WEB

What is an A/B test?

Comparing two versions of an experience to determine which performs better.

In plain English

An A/B test splits your traffic between two versions and measures which produces more of the outcome you want. Done properly it replaces opinion with evidence.

The catch is volume. Reaching a trustworthy conclusion needs enough conversions in each version, and most small business pages simply do not get them. Testing on thin traffic produces confident-looking numbers that mean nothing.

What to know

One meaningful change at a time
Test things that could plausibly change behaviour. Headline, offer, form length, page structure. Not a shade of blue.
Decide the metric before you start
And make it the one that matters commercially, not the easiest one to move.
Let it run to a real conclusion
Stopping early because one version is ahead is the most common way tests mislead. Run full weeks to cover the weekly cycle.
Most sites should fix before testing
If the page has obvious problems, repair them. Testing is for genuine uncertainty, not for postponing known fixes.

Why it matters

Testing is valuable where the stakes and the traffic are both high, because it settles arguments with data instead of seniority. Below a certain volume it is theatre, and the honest answer for most small sites is to fix the obvious and watch the outcome over time.

Common mistakes

×Running tests on pages without enough conversions to detect a difference.
×Stopping the moment a version looks like it is winning.
×Changing several things at once, so the result cannot be attributed.
×Declaring a winner on clicks while enquiry quality quietly falls.

FAQs

How much traffic do I need to A/B test?

Enough conversions per version to be confident, typically hundreds rather than dozens. Below that, prioritise fixing over testing.

How long should a test run?

Full weeks, usually at least two, so weekday and weekend behaviour are both represented.

WRITTEN BY TARIQ SALLAM
Marketing Consultant. Entrepreneur. Content Creator.

I'm a marketing consultant, entrepreneur and content creator. I help businesses grow through practical marketing, websites, SEO, content and AI.

More About Tariq →