A/B Test Sample Size/Duration Calculator

Plan your A/B tests with confidence!

Enable JavaScript to plan your experiments.

See the source code on GitHub.

Daily visitors

Current rate (%)

Minimum detectable effect (%)

Relative Absolute

Number of groups

Single estimate Δ → Days Days → Δ

Duration

days to complete

Sample size

24,601

per variant

The experiment will need to run for 4 months and 9 days to detect a 10% improvement (from 3.0% to 3.3%).

Detectable Δ (%)	From → To	Duration	Total visitors

Days	Total visitors	Detectable Δ	From → To

Advanced configuration

Statistical parameters

Significance level (α)

0.01 0.1

Statistical power (1-β)

0.7 0.95

Test type

Superiority Two-tailed Non-inferiority Equivalence

Multiple testing correction

Traffic Allocation

Traffic flow (%)

1% 100%

Buffer (%)

0% 300%

Traffic distribution

Context

Imagine you redesign a button aiming to increase sign-up rate. The only way to reliably tell whether the new design is better than the old one is to run an A/B test.

A/B testing involves showing different versions of your product to different users and measuring which version performs better. Some users (usually half) will see the original version (A); others will see the new version (B). The original version is your "control" group—the baseline against which we'll compare the new version.

Let's say your current page converts at 5%, and you want to detect if the button redesign can improve that by 10% (to 5.5%). Is it enough to test with 100 visitors per variant? 1,000? 10,000? And how many days will that take?

If you don't collect enough data, you risk making decisions based on random fluctuations. But collecting too much data means wasting time and resources. This calculator helps you find the sweet spot.

Behind the scenes

This calculator is built on the principles of power analysis for two-proportion z-tests. It uses the following standard formula to determine the required sample size (n) per variant.

n = \frac{{(z_{1 - α} + z_{1 - β})}^{2} (σ_{1}^{2} + σ_{2}^{2})}{Δ^{2}}

Where Δ is the absolute effect size, z values correspond to statistical power and significance, and σ² is the variance for each group. Variance is calculated as $σ^{2} = p (1 - p)$ , where p is the conversion rate.

Critical Values by test type

Superiority

$z_{α} = Φ$ ^-1(1-α)

$z_{β} = Φ$ ^-1(power)

Expects different performance between groups

Two-tailed

$z_{1 - α / 2}$

$z_{1 - β}$

Expects different performance between groups

Non-inferiority

$z_{1 - α}$

$z_{1 - β}$

Assumes same baseline rate for both groups

Equivalence

$z_{1 - α}$

$z_{(1 + power) / 2}$