# Sample size: definition, formula and examples

Source: https://leverprice.com/glossary/sample-size
Updated: 2026-11-17

> Sample size is the number of visitors each variant needs before a test can reliably detect a difference of a chosen size. It depends on the baseline conversion rate, the smallest effect you care about, the significance level and the statistical power. At 2% conversion, detecting a 20% relative change takes about 21,100 visitors per price.

Sample size answers "how many visitors before I can trust this?" If you test $29 against $35 on a page that converts at 2%, you need about 21,100 visitors on each price to reliably detect a 20% difference in conversion. Stop well short of that, and a real 20% difference will often be missed.

## How to calculate it

For two conversion rates p₁ and p₂, at 95% confidence (two-sided) and 80% power:

```text
n per variant = (1.96 + 0.84)² × [p₁(1 − p₁) + p₂(1 − p₂)] ÷ (p₂ − p₁)²
rule of thumb: n ≈ 16 × p(1 − p) ÷ δ²
```

The rule of thumb comes from Evan Miller's [notes on A/B testing](https://www.evanmiller.org/how-not-to-run-an-ab-test.html), where δ is the absolute difference you want to detect.

Example: the baseline converts at 2%, and you want to detect a lift to 2.4%, a 20% relative change.

- (1.96 + 0.84)² = 7.85
- p₁(1 − p₁) + p₂(1 − p₂) = 0.0196 + 0.0234 = 0.0430
- (p₂ − p₁)² = 0.004² = 0.000016
- n = 7.85 × 0.0430 ÷ 0.000016 ≈ 21,100 visitors per price

Two prices need about 42,200 visitors in total. At 2,000 pricing-page visitors a month, that is 21 months.

*Figure: Chart: visitors needed per price fall steeply as the effect you want to detect grows. Visitors needed per price at 95% confidence and 80% power. At 2% conversion, a 20% difference takes about 21,100 visitors per price.*

## Why it matters for pricing

Sample size tells you whether a price test is feasible before you start it. At the same 2% conversion, detecting a 10% difference takes about 80,700 visitors per price. Across two prices, a product with 2,000 pricing-page visitors a month would need more than six years to collect them.

So test prices further apart, where the effect you're looking for is bigger. Or use a method that limits the cost of testing, such as a bandit that moves traffic away from losing prices while the evidence builds. The [price test sample size calculator](https://leverprice.com/tools/price-test-sample-size-calculator) runs these numbers for your traffic, and [price testing with low traffic](https://leverprice.com/blog/price-testing-low-traffic) covers the options in detail.

## Common mistakes

- Computing one sample size for the whole test. The formula gives visitors per variant, so multiply by the number of prices.
- Counting page views or sessions instead of unique visitors.
- Stopping early because the result already looks significant, which breaks the math the sample size was based on.
- Using conversion formulas for revenue without checking the variance. When buyers pay different amounts, through plans or add-ons, revenue per visitor varies more and needs more visitors.

## Related terms

Sample size is the flip side of the [minimum detectable effect](https://leverprice.com/glossary/minimum-detectable-effect), and both rest on the [statistical significance](https://leverprice.com/glossary/statistical-significance) threshold and your baseline [conversion rate](https://leverprice.com/glossary/conversion-rate). A [multi-armed bandit](https://leverprice.com/glossary/multi-armed-bandit) still needs the data, but it spends less revenue gathering it.

## Frequently asked questions

### How many sales do I need for a price test?

Work backwards from visitors. At 2% conversion, 21,100 visitors per price is about 420 sales per price at the baseline rate, and smaller effects need many more.

### Does a Bayesian test need a smaller sample?

Not for the same certainty about the same effect. Bayesian methods report results differently and handle continuous monitoring better, but separating two close prices takes a similar amount of data.
