A/B Test Sample Size/Duration Calculator

Plan your A/B tests with confidence!

Enable JavaScript to plan your experiments.

See the source code on GitHub.

Daily visitors

Current rate (%)

Minimum detectable effect (%)

Relative Absolute

Number of variants

Single estimate Δ → Days Days → Δ

Duration

129

days to complete

Sample size

64,410

per variant

The experiment will need to run for 4 months and 9 days to detect a 10% improvement (from 3.0% to 3.3%).

Detectable Δ (%)	From → To	Duration	Visitors/variant

Days	Visitors/variant	Detectable Δ	From → To

Advanced configuration

Statistical parameters

Significance level (α)

0.01 0.1

Statistical power (1-β)

0.7 0.95

Test type

Superiority Two-tailed Non-inferiority Equivalence

Multiple testing correction

Traffic Allocation

Traffic flow (%)

1% 100%

Buffer (%)

0% 300%

Traffic distribution

Context

Imagine you redesign a button aiming to increase sign-up rate. The only way to reliably tell whether the new design is better than the old one is to run an A/B test.

A/B testing involves showing different versions of your product to different users and measuring which version performs better. Some users (usually half) will see the original version (A); others will see the new version (B). The original version is your "control" group—the baseline against which we'll compare the new version.

Let's say your current page converts at 5%, and you want to detect if the button redesign can improve that by 10% (to 5.5%). Is it enough to test with 100 visitors per variant? 1,000? 10,000? And how many days will that take?

If you don't collect enough data, you risk making decisions based on random fluctuations. But collecting too much data means wasting time and resources. This calculator helps you find the sweet spot.

Behind the scenes

This calculator is built on the principles of power analysis for two-proportion z-tests. It uses the following standard formula to determine the required sample size (n) per variant.

n = \frac{{(z_{1 - α} + z_{1 - β})}^{2} (σ_{1}^{2} + σ_{2}^{2})}{Δ^{2}}

Where Δ is the absolute effect size, z values correspond to statistical power and significance, and σ² is the variance for each group. Variance is calculated as $σ^{2} = p (1 - p)$ , where p is the conversion rate.

Critical Values by test type

Superiority

$z_{α} = Φ$ ^-1(1-α)

$z_{β} = Φ$ ^-1(power)

Expects different performance between groups

Two-tailed

$z_{1 - α / 2}$

$z_{1 - β}$

Expects different performance between groups

Non-inferiority

$z_{1 - α}$

$z_{1 - β}$

Assumes same baseline rate for both groups

Equivalence

$z_{1 - α}$

$z_{(1 + power) / 2}$

Assumes same baseline rate for both groups

All calculations are done locally; your data never leaves your device. The calculator uses Acklam's algorithm to compute inverse normal values. You can verify your device returns expected numbers by adding ?test to this calculator's URL.

Óscar's A/B Testing Toolkit

A/B Test Sample Size/Duration Calculator

Duration

Sample size

Sample Size Per Variant

Óscar's A/B Testing Toolkit

Duration

Sample size

Sample Size Per Variant

Óscar's tips

How this works

Context

Behind the scenes

Critical Values by test type

Superiority

Two-tailed

Non-inferiority

Equivalence