Definition
A multi-armed bandit is a testing method that shifts traffic toward the best-performing variant while the test is still running, instead of holding a fixed split until the end. It accepts a less precise estimate of the losing variants in exchange for sending fewer visitors to them, which suits price tests run on real buyers.
- Every visitor sent to a worse price costs the gap in revenue per visitor, and a bandit tries to keep that cost low.
- A bandit still needs a floor of exploration and a delay for outcomes that take time, such as purchases.
A multi-armed bandit is a test that learns as it goes. Instead of splitting visitors evenly between $19, $29 and $49 for a month and only then picking a winner, it starts even, watches who buys, and keeps sending more visitors to whichever price is earning the most. By the second week the leading price might get 70% of traffic and the weakest one 10%.
The name comes from slot machines, once nicknamed one-armed bandits: a gambler facing a row of them has to decide which arm to keep pulling while still learning which machine pays best (background).
How to calculate it
A bandit is a rule for splitting traffic. The cost it tries to keep small is called regret: the revenue lost by showing visitors anything other than the best price.
regret = Σ over visitors (RPV of best price − RPV of price shown)Example: say $49 earns $0.64 per visitor, $29 earns $0.58 and $19 earns $0.49. Over 9,000 visitors, an even split sends 3,000 to each losing price and gives up 3,000 × $0.06 + 3,000 × $0.15 = $630. A bandit that splits the first 3,000 visitors evenly ($210 of regret), then sends 80% of the next 6,000 to $49 and 10% to each of the others ($126), gives up $336, about half as much.
How fast traffic moves depends on the rule. Thompson sampling, a common choice for prices, gives each price a share of traffic roughly equal to the probability that it is the best.
Why it matters for pricing
In a price test the arms are prices and every pull is a real buyer, so a losing price costs money for as long as it gets traffic. An A/B test pays that cost on purpose to measure every price precisely. A bandit stops paying it once the evidence points one way. Leverprice runs a Thompson-sampling bandit that recomputes traffic weights every few minutes, keeps at least 5% of traffic on every price and counts only visitors older than 30 minutes, so fresh traffic that hasn't had time to buy doesn't drag a price down.
The cost is precision. Losing prices get fewer visitors, so their numbers stay noisy, and a bandit is the wrong tool if you need an exact estimate of how much worse $19 is. It is the right one when the question is which price to charge.
Common mistakes
- Reading the traffic share as the result. A price with 70% of traffic is the current favorite, and the size of its advantage still has to come from the revenue numbers.
- Reacting to purchases that haven't happened yet. Visitors from the last few minutes have had no time to buy, so counting them punishes whichever price got the newest traffic.
- Turning exploration off. With no minimum share, a price that had an unlucky first day can lose all its traffic and never get the chance to recover.
- Changing the page or the traffic mix mid-test, which the bandit reads as a change in how the prices perform.
Related terms
Expected loss is how a bandit decides it can stop. The fixed-split alternatives are a Bayesian A/B test and a classic test judged by statistical significance, both scored on revenue per visitor when prices are involved.
Frequently asked questions
Is a bandit better than an A/B test for pricing?
If your goal is to end up charging the best price while losing as little revenue as possible along the way, a bandit fits better. If you need a precise estimate of every price's effect, a fixed split does. The tradeoffs are laid out in bandit vs A/B test for pricing.
Does a bandit need less traffic?
Not to tell two close prices apart. The weaker price gets fewer visitors, so its estimate firms up more slowly. What a bandit saves is revenue, because most visitors see the leading price while the evidence builds.