A/B Test Calculator for Ads | Is Your Winner Real?

Runs in your browser. Nothing is uploaded or stored.

1Type your own numbers on the left. Everything updates as you type.
2Read the results on the right. The highlighted figure is the one that matters most.
3Copy the link to share these exact numbers, or switch currency at the top.

Use impressions and clicks to compare CTR, or clicks and conversions to compare conversion rate. Same measure for both ads.

Ad A: impressions (or clicks)Ad A: clicks (or conversions)

Ad B: impressions (or clicks)Ad B: clicks (or conversions)

Impressions per day across both ads optional, for the time estimate

Confidence the difference is real–

Ad A rate–

Ad B rate–

Lift of B over A–

How it works

The calculator runs a two-proportion z-test, the standard test for comparing two conversion rates. It pools both ads to estimate the expected variation, then measures how far apart the two rates are in those terms. Confidence is reported two-sided, so it doesn't assume in advance which ad should win.

z = (rate B − rate A) ÷ √( p(1−p)(1/nA + 1/nB) ), where p is the pooled rate

Worked example

Two Meta ads each get 20,000 impressions. Ad A gets 300 clicks (1.5% CTR), ad B gets 360 (1.8%). That is a 20% lift and the calculator reports about 98% confidence, so B is a real winner. The same 20% lift at 2,000 impressions each (30 clicks against 36) comes out at roughly 55% confidence, which is a coin toss. Most "winning" ads declared after two days are the second case.

Before you trust a result

Check the two ads ran to the same audience at the same time with similar budgets. A test where Meta gave one ad 80% of the spend is not a fair comparison, whatever the calculator says.

FAQ

A/B test calculator for ads: questions answered.

What does statistical significance mean for an ad test?

It is how confident you can be that the difference between two ads is real rather than luck. At 95% confidence there is only a one in twenty chance the gap is random.

What should I compare?

For creative tests, impressions and clicks (CTR). For landing page or ad set tests, clicks and conversions (conversion rate). Use the same measure for both variants.

How long should I run an ad test?

Until each variant has enough results for a confident read, and at least seven days so you cover every day of the week. The calculator estimates the impressions you still need.

Why does Meta say one ad is winning when this says it isn't?

Meta's delivery system picks a favourite early and gives it most of the budget, which is not the same as proving it is better. For a fair test, use Meta's A/B test feature or separate ad sets with equal budgets.

Can I test more than two ads?

Run this calculator for each pair against the control. Testing many variants at once needs more impressions per variant and raises the chance of a false winner.

What confidence level should I use?

95% is the usual standard. For low-risk creative swaps, 90% is often enough to act on. For a decision that changes your whole account, wait for 95%.

Does this work for Google Ads experiments?

Yes. Paste the impressions and clicks (or clicks and conversions) from the control and the experiment arm.

More free tools

A/B test calculator for ads.

How it works

Worked example

Before you trust a result

A/B test calculator for ads: questions answered.

Keep going.

ROAS & break-even calculator

CPM calculator

CTR calculator

All free tools

Want someone to run the numbers for you?

Paid ads insights, straight from the accounts.