NCAAB model accuracy
NCAA Division I Men's Basketball
Every final NCAAB game, forecast with only what was known the day before and then scored. Lower Brier and log loss are better; a coin flip scores 0.250 Brier. All leagues
Scorecard
| Model | Games | Brier | Log loss | Accuracy | Skill | Calib. error |
|---|---|---|---|---|---|---|
| Game simulator (backtest) | 5,812 | 0.201 | 0.585 | 68.4% | +13.3% | 6.1% |
| Elo | 5,812 | 0.197 | 0.579 | 70.4% | +14.8% | 3.6% |
| Base rate | 5,812 | 0.232 | 0.656 | 64.3% | — | 6.2% |
| Published forecasts | 18small sample | 0.126 | 0.412 | 83.3% | +17.1% | 9.3% |
Calibration
When a model gives the home side a 60% chance, does it win about 60% of the time? Dots on the dashed line mean yes; whiskers show the 90% range the observed rate could land in by chance.
Brier score by month
Early-season forecasts lean on last season's regressed ratings, so expect them to improve as games accumulate.
Head to head
Game simulator (backtest) vs Elo
5,812 games both forecast
- Brier
- 0.201 vs 0.197
- Log loss
- 0.585 vs 0.579
- Accuracy
- 68.4% vs 70.4%
Elo has the lower Brier score here.
Published forecasts vs Game simulator (backtest)
18 games both forecast
- Brier
- 0.126 vs 0.148
- Log loss
- 0.412 vs 0.477
- Accuracy
- 83.3% vs 83.3%
Published forecasts has the lower Brier score here, but on too few games to call.
Market consensus isn't connected: it needs a licensed odds source, which an admin can import. Until then the base rate and Elo are the benchmarks.
How this is measured
The backtest replays the game simulator for every final game from the efficiency ratings and Elo as they stood before that date, the league's scoring and home advantage so far, rest and travel, the venue and the game's weather. Injuries and lineups of past games aren't on record, so the replay is team level; the published forecasts, which include them, are scored separately as they accumulate.
Brier = ½ Σ (probability − outcome)² over home, away. Log loss = −ln(probability given to the result), with probabilities clipped at 0.5%. Tied finals are left out. Full details are in the model methodology document.
Pick record
Every forecast the site published before a game, scored once it finished. A pick is the likeliest outcome. A well-calibrated model's 70% picks come in about 70% of the time, so compare each row's hit rate with its confidence.
Record
15–3 83%
Average confidence 80%
| Confidence | Picks | Hit |
|---|---|---|
| 70%–80% | 6 | 67% |
| 80%+ | 12 | 92% |
| Game | Pick | Result |
|---|---|---|
| Nov 6 EWU @ ORE | ORE 80% | 71–98 |
| Nov 6 VAL @ PUR | PUR 71% | 83–80 |
| Nov 6 GASO @ GT | GT 76% | 83–95 |
| Nov 6 BU @ BRWN | BRWN 74% | 72–67 |
| Nov 6 APSU @ KENN | KENN 80% | 69–71 |
| Nov 6 INST @ BALL | BALL 82% | 69–66 |
| Nov 6 ARST @ ALA | ALA 79% | 86–101 |
| Nov 6 DEL @ WIS | WIS 83% | 64–73 |
| Nov 6 FAU @ WRST | WRST 82% | 71–97 |
| Nov 6 DETM @ MARQ | MARQ 85% | 32–93 |
| Nov 6 STBK @ NU | NU 85% | 65–78 |
| Nov 6 IDST @ WYO | WYO 80% | 62–93 |
| Nov 6 MTSU @ KU | KU 84% | 45–89 |
| Nov 6 WAG @ MIA | MIA 76% | 57–83 |
| Nov 6 UNCO @ ASU | ASU 81% | 63–90 |
| Nov 6 IDHO @ CSU | CSU 80% | 67–78 |
| Nov 6 CSUN @ KSU | KSU 83% | 78–95 |
| Nov 6 CBU @ ORST | ORST 84% | 68–100 |
Data as of Source: SportsAPI Pro