The Brier score measures the accuracy of probabilistic forecasts, with lower scores indicating better predictive performance. It calculates the mean squared difference between predicted and actual outcomes.
The Brier score measures the accuracy of probabilistic forecasts, with lower scores indicating better predictive performance. It calculates the mean squared difference between predicted and actual outcomes.
The Brier score is a statistical metric designed to evaluate how well probabilistic predictions match reality. When you make a prediction about an event—say, "there's a 75% chance this market resolves YES"—you're assessing probability quantitatively. The Brier score measures whether that assessment was accurate by comparing your predicted probability to the actual outcome. If you predicted 0.75 (75%) and the outcome was 1 (YES), the metric penalizes the gap between them. The formula is straightforward: square the difference between your predicted probability and the actual outcome (0 or 1), then average across all predictions. A Brier score of 0 is perfect; a score of 1 is the worst possible.
The Brier score was introduced by meteorologist Glenn Brier in 1950 to evaluate weather forecasters. Since then, it has become the gold standard in finance, election forecasting, and pandemic modeling. In prediction markets like Polymarket, the Brier score is crucial because traders are probabilistic forecasters. When you buy YES shares at 75 cents, you're implicitly predicting YES has at least a 75% chance of occurring. Over time, a trader's Brier score reveals whether their predictions are well-calibrated or biased. A skilled forecaster should have a low Brier score. Institutions increasingly use it as a performance metric because it's objective, mathematically sound, and rewards honest probability assignments.
On Polymarket, you won't see "Brier score" in the interface, but the concept underpins serious trader performance evaluation. A skilled trader might review historical positions and calculate their Brier score to understand calibration—the alignment between predicted and actual outcomes. If you always bet 60% probability but they resolve YES only 45% of the time, your Brier score signals overconfidence. Advanced traders use Brier score reasoning when evaluating whether market prices are well-calibrated. If a 65%-priced market historically resolves YES only 50% of the time, that's a sign of market-wide miscalibration.
A common misconception is that the Brier score only measures right-or-wrong accuracy. In truth, it penalizes overconfidence. If you predict 99% YES and it resolves YES, the penalty is tiny (0.01² = 0.0001). If you predict 51% YES and it also resolves YES, the penalty is much larger (0.49² = 0.2401). This is deliberate: the metric rewards calibrated, honest probability estimates. Another pitfall is treating Brier score as a universal forecaster ranking without context. A score of 0.15 is good relative to naive 50-50 forecasting, but context matters. Finally, traders sometimes conflate Brier score with profitability. You can have excellent calibration but lose money trading at unfavorable odds, or vice versa. Brier score measures forecasting accuracy, not financial returns.
The Brier score belongs to a family of scoring rules in forecasting. Log loss is another common metric that penalizes extreme confidence even more heavily. In prediction markets, related concepts include Shannon entropy, which measures probability distribution uncertainty, and calibration itself—whether predictions are reliable across different confidence levels. Traders balance calibration (accuracy at a given confidence) with sharpness (confidence level of predictions). A perfectly calibrated forecaster predicting 50-50 on everything is accurate but useless; an ideal forecaster optimizes both dimensions.
Imagine a Polymarket question: 'Will ETH/USD close above $3,000 by June 30, 2026?' You predict 70% YES and buy shares at 70 cents. When June 30 arrives, ETH closes at $2,950, so the market resolves NO (outcome = 0). Your Brier score contribution for this prediction is (0.70 − 0)² = 0.49. If you had instead predicted 75% and the same outcome occurred, your Brier score would be (0.75 − 0)² = 0.5625, higher because your prediction was further from reality.