Definition
The minimum detectable effect (MDE) is the smallest true difference between variants that a test can reliably detect, given its sample size, significance level and statistical power. With 5,000 visitors per price and a 2.5% conversion rate, a standard test can reliably catch only a relative change of about 35% or more.
- MDE ≈ (1.96 + 0.84) × √(2 × p × (1 − p) ÷ n) at 95% confidence and 80% power.
- Four times the traffic only halves the MDE.
The MDE is the resolution of your test. If a test of $29 against $35 has an MDE of 35%, it can reliably spot one price converting 35% better or worse than the other. A smaller real difference will usually come back as "no significant difference", even though it exists.
How to calculate it
For a conversion rate p with n visitors per variant, at 95% confidence (two-sided) and 80% power:
MDE (absolute) = (1.96 + 0.84) × √(2 × p × (1 − p) ÷ n)
MDE (relative) = MDE (absolute) ÷ pExample at a 2.5% conversion rate:
| Visitors per price | MDE (absolute) | MDE (relative) |
|---|---|---|
| 2,000 | 1.38 points | 55% |
| 5,000 | 0.87 points | 35% |
| 20,000 | 0.44 points | 17.5% |
| 50,000 | 0.28 points | 11% |
The MDE shrinks with the square root of traffic, so four times the visitors buys half the MDE. Evan Miller gives the same formula in his notes on running A/B tests.
Why it matters for pricing
A 10% difference in revenue per visitor is real money over a year, and detecting a 10% change in a 2.5% conversion rate takes about 61,000 visitors per price. Many small products don't see that much pricing-page traffic in months, so the MDE decides which tests are worth running at all.
The practical answer is to test prices far apart. $29 against $49 has a fair chance of producing an effect above the MDE; $29 against $31 very likely falls below it and ends as an expensive "no difference". Low-traffic products have other options too, covered in price testing with low traffic.
For revenue per visitor, the same formula works with the variance of revenue in place of p × (1 − p). The price test sample size calculator runs these numbers for your traffic.
Common mistakes
- Setting the MDE to the effect you hope for. Pick the smallest difference that would change your decision, then check whether you can afford to detect it.
- Mixing absolute and relative MDE. A 0.5-point change on a 2.5% conversion rate is a 20% relative change.
- Forgetting that more variants mean less traffic for each. Three prices on the same traffic raise every comparison's MDE.
- Reading "no significant difference" as "no difference". It often means the effect was smaller than the MDE.
Related terms
The MDE and sample size are two sides of one calculation, both tied to the statistical significance threshold and the baseline conversion rate. Expected loss offers a different stopping rule: stop when the remaining difference is too small to matter.
Frequently asked questions
Should the MDE be relative or absolute?
Either works if you stay consistent. Relative MDE is easier to reason about for pricing ("a 15% change"), while the formulas work with absolute differences.
What MDE should a price test use?
The smallest change in revenue per visitor that would make you switch prices. If detecting it would take longer than you're willing to wait, test prices that are further apart.