Short answer
It depends on how many sales you make and how much better the best price is. In 960 simulations of our production engine, a price that earned 10% more per visitor was found in a median of 40 days at 300 sales a month, 14 days at 1,000 sales and 263 days at 50 sales. Testing bigger price gaps cuts the time the most.
- At 300 sales a month, a 10% better price took a median of 40 days to find; a 20% better price took 23 days.
- Below 50 sales a month, most tests needed a price that earns 20 to 30% more to finish within a year.
- In 870 decisions across 960 runs, the engine never settled on the worst price. It picked the best one 94.5% of the time.
- A classic even-split test sized for 95% confidence and 80% power would need 663 days for the same 300-sales, 10% case.
Most advice on test duration is borrowed from landing-page A/B tests: "run it for two weeks", "wait for 100 conversions per variant". Prices are different. The number you care about is revenue per visitor, purchases are rare, and the effect you are looking for is usually small. So we stopped guessing and ran the numbers on the engine we ship.
We simulated 960 price tests with the same allocation code that runs Leverprice in production, across 8 sales volumes and 4 effect sizes, 30 runs each. This page has the full table, what the engine decided, how often it was right, and what you can do to get an answer sooner.
How we ran the simulation
Every simulated store sells one product with three test prices: $19, $29 and $49. The current price, $29, is the control and converts 2% of visitors. We set the other two prices by revenue per visitor, because that is what a price test optimizes:
- $19 earns 15% less per visitor than $29 (it sells more often, but not often enough).
- $49 earns 5%, 10%, 20% or 30% more per visitor than $29, depending on the scenario. At +10%, for example, $49 converts about 1.3% of visitors.
Traffic is set so the store makes 30, 50, 100, 200, 300, 500, 1,000 or 2,000 sales a month at the control price, which means 1,500 to 100,000 visitors a month. Visitors arrive evenly, each one is assigned a price, and buys with that price's probability. Every 6 hours the engine recomputes how much traffic each price gets, exactly as it does in production: Thompson sampling over a three-outcome model (kept purchase, refunded purchase, no purchase), a 5% minimum share per price, and smoothing so traffic never swings wildly. Each scenario was repeated with 30 different random seeds and stopped at the first decision or after 365 days.
The engine stops in one of two ways:
- Clear winner: one price is at least 95% likely to earn the most per visitor.
- Practically equivalent: the remaining prices are so close that picking the current leader costs less than 0.5% of revenue per visitor in expectation. When the best option is only slightly better, this is the rule that ends the test.
This is a simulation, not customer data. Demand is synthetic and stable, there are no refunds, no seasonality and no marketing spikes. Real tests are noisier. The point is to show orders of magnitude and the levers that move them.
How long a price test takes at your volume
Median days until the engine reached a decision, out of 30 runs per cell. "Over a year" means at least half of the runs had no decision after 365 days; the number in brackets is how many of the 30 decided within a year.
| Sales per month | Best price earns 5% more | 10% more | 20% more | 30% more |
|---|---|---|---|---|
| 30 | over a year (12 of 30) | over a year (15 of 30) | 230 days (18 of 30) | 121 days (28 of 30) |
| 50 | over a year (15 of 30) | 263 days (20 of 30) | 89 days (29 of 30) | 79 days |
| 100 | 195 days (23 of 30) | 113 days (26 of 30) | 68 days | 55 days |
| 200 | 98 days (25 of 30) | 60 days | 40 days | 27 days |
| 300 | 66 days (29 of 30) | 40 days | 23 days | 15 days |
| 500 | 31 days | 31 days | 14 days | 9 days |
| 1,000 | 27 days | 14 days | 8 days | 7 days |
| 2,000 | 12 days | 7 days | 5 days | 3 days |
Two things stand out. First, volume matters roughly in proportion: double the sales and the test takes about half as long. Second, the size of the difference matters even more. At 100 sales a month, a price that is 30% better shows up in under two months, while a 5% improvement takes over six. You don't control your volume this month, but you do control which prices you put in the test. More on that below.
If you sell fewer than 50 times a month, read the table as a warning about small differences, not as a verdict on price testing. A test of $29 against $49 can still finish. A test of $29 against $31 almost never will.
Did it pick the right price?
Across all 960 runs, 870 reached a decision within a year. Of those 870:
- 822 (94.5%) picked $49, the price that really earned the most.
- 48 settled on the $29 control. 29 of those came from the 5% scenarios, where $49 was only slightly better.
- 0 picked $19, the price that lost money.
At 300 or more sales a month the engine was right in 115 of 120 runs when the best price was 10% better, and 118 of 120 when it was 30% better.
That pattern is what you want from a pricing engine. When it is unsure, it keeps your current price rather than gambling on a cheaper one. When it settles on the control, the measured gap was small, which is also a useful answer: your current price is close to the best one among those you tested.
Most decisions (599 of 870) came through the practically-equivalent rule rather than the 95% rule (271). That surprised us less than it might surprise you. A price that is 10% better per visitor is a big deal for your business, but a hard thing to prove with 95% certainty from a 1 to 2% conversion rate. The expected-loss rule asks a more practical question: "if I stop now and run the leader, how much do I lose if I'm wrong?" Once that answer is "less than half a percent", more testing is not worth it.
What happened to visitors during the test
A test costs money while it runs, because some visitors see a worse price. In an even split, a third of your visitors sit on the worst price until you stop the test.
In our runs, the median share of visitors who saw $19 over the whole test was 8 to 15%, and the best price received 49 to 69% of all visitors before the decision. The engine moves traffic toward the leader as evidence comes in, so most of the test is spent selling at prices that are already earning well.
Why a classic A/B test would take so much longer
The textbook way to test prices is an even split with a sample size fixed in advance. We calculated that sample size for the same scenarios with the standard two-sample formula for revenue per visitor, 95% confidence (two-sided) and 80% power, comparing the best price against the control:
n per price = (1.96 + 0.84)² × (σ²_control + σ²_best) / (μ_best − μ_control)²
μ = price × conversion rate (revenue per visitor)
σ² = price² × conversion × (1 − conversion)For the 300-sales, 10% case: μ is $0.58 for $29 and $0.64 for $49, the variances are about 16.5 and 30.9, so each price needs about 110,000 visitors. With three prices on an even split and 500 visitors a day, that is 663 days.
| Sales per month | Even split, 10% better price | Engine median, 10% better price |
|---|---|---|
| 100 | 1,989 days | 113 days |
| 300 | 663 days | 40 days |
| 1,000 | 199 days | 14 days |
| 2,000 | 100 days | 7 days |
These two columns answer different questions, and it would be dishonest to present them as a race. The fixed test is built to prove, with 80% power, that a 10% lift is real. The engine is built to stop when continuing no longer pays, which includes stopping when the remaining difference is too small to matter. If you need a precise, defensible estimate of the lift, a fixed test is the right tool and you should plan for its duration. If you want the price that earns the most, without spending a third of your traffic on a bad price for months, an adaptive test gets you there in a fraction of the time. We compare the two in detail in bandit vs A/B test for pricing.
The reason both numbers are large is the same: revenue per visitor is a noisy number. Most visitors spend $0, a few spend $29 or $49, and that spread swamps a 6-cent difference in the mean. This is why "run it for two weeks" fails for prices. Two weeks is plenty for a button color with a 30% click rate, and nowhere near enough for a price that converts 2% of visitors.
How to make your price test finish sooner
Test prices that are far apart. This is the biggest lever. Going from a 10% to a 20% advantage cut the median time by a third to two thirds in our table. If your current price is $29, test $29, $39 and $49, not $29, $30 and $32. You can narrow the range in a second test once you know which region wins.
Count the right traffic. What matters is visitors who see a price, not total site visitors. If only one in five visitors reaches the pricing page, your effective volume is a fifth of your analytics number. Put the price where people make the decision, and show it to everyone who could buy.
Test where the money is. If most of your revenue comes from the annual plan or the top tier, test that price first. A test on a plan that brings in 10% of revenue takes as long to run and pays a tenth as much.
Accept "practically equivalent" as an answer. If two prices earn about the same per visitor, the higher one usually means fewer customers to support for the same money, and the lower one may grow faster through word of mouth. Either way you've learned that you are close to the top of the curve.
Keep the test running through ordinary weeks. Launch days, discounts and press spikes bring visitors who behave differently. Don't start a test the day before a big launch, and don't stop it because a promotion skewed one week.
Don't stop a fixed test early because it looks good. Peeking at a classic A/B test and stopping at the first significant result inflates false positives badly. Adaptive engines are designed for continuous monitoring; fixed-horizon tests are not.
If you are below about 100 sales a month, read how to test prices with low traffic. It covers surveys, bigger gaps and other ways to make progress without waiting a year.
Estimate your own test
Plug your traffic, conversion rate and the gap you want to detect into the price test sample size calculator. It shows the fixed-split requirement, so you can see what a classic test would cost, and links back to the table above for an adaptive estimate.
Inside Leverprice, each running test shows an estimate of how many more visitors a decision will likely take, based on the data collected so far. It is labeled as an estimate because it is one: real traffic does not arrive at a steady 500 visitors a day.
What this simulation leaves out
- Synthetic demand. Real conversion rates drift with seasons, campaigns and word of mouth. That usually makes tests slower, not faster.
- No refunds. The production engine counts refunded purchases separately; we left them out to keep the scenarios clean.
- One baseline conversion rate. We used 2% at $29. A product with a 5% pricing-page conversion rate learns faster at the same number of sales because each visitor carries more information.
- Three prices. Two prices would finish somewhat faster; five would take longer.
- A 365-day horizon. Some runs would have decided later.
The full table above comes from one script that calls the production allocation function; we'll rerun it whenever the engine changes and update this page.
Frequently asked questions
How many visitors do I need for a price test?
It depends on your conversion rate and on how different the prices are. With a 2% conversion rate and a best price that earns 10% more per visitor, a classic fixed test needs about 110,000 visitors per price. An adaptive engine that stops once the remaining difference is negligible needed a median of about 20,000 visitors in total at that conversion rate (40 days at 500 visitors a day).
Is two weeks enough for a price test?
Only at high volume. In our simulation, two weeks was enough for a 10% better price at about 1,000 sales a month, or for a 20% better price at about 500 sales a month. Below that, two weeks mostly measures noise.
Can I stop a price test early?
You can stop an adaptive test whenever its decision rule says so, because it was designed for continuous monitoring. Stopping a fixed-horizon A/B test the first time it looks significant inflates the false positive rate, so either wait for the planned sample size or use a method built for early stopping.
What if my price test never reaches a decision?
That is information too: the prices you tested earn about the same per visitor, or your volume is too low to tell them apart. Test prices that are further apart, test the plan that brings in the most revenue, or use a survey such as the Van Westendorp price sensitivity meter to narrow the range first.
Do I need statistical significance for a price test?
You need a decision rule you trust before you start. Classic significance testing is one option and requires a fixed sample size. Bayesian rules, such as "95% probability of being best" or "expected loss below 0.5%", answer the question founders actually ask: which price should I run now, and how much could I lose if I'm wrong?