The talent-blind baseline
The model that scores every shot in the tables — its structure, the reasoning behind it, and its honest scorecard.
Expected goals (xG) is the probability that a given shot attempt becomes a goal, estimated from where and how it was taken — distance, angle, shot type, whether it followed a rebound or a rush, the strength state, and more. The model is deliberately talent-blind: it scores the chance for a league-average shooter against a league-average goalie, so that a player's finishing (goals minus xG) measures their shooting talent against that neutral bar.
That choice is the whole design. A model that already knows Auston Matthews is shooting cannot tell you how good Auston Matthews is at shooting — the talent gets absorbed into the prediction and disappears from the residual. Keeping identity out is what keeps the residual meaningful, and it is still true of the model scoring this site today.
Updated 20 July 2026 — which model scores the tables. Until
now it was the v1 model described on this page. It is now
flat__joint_structure_xg__4f64c88b, the identity-blind structural half of
Model v2's joint fit: the same talent-blind
contract, but with the structure re-estimated net of player talent instead of
absorbing it. That matters for exactly the reason this page is about — a baseline fit
without identity quietly soaks up whatever talent correlates with its features (skilled
shooters over-represented in good locations inflate the location terms, which then
under-credit those shooters), and the joint fit measures the two in tension so it does not.
The scoreboard is still talent-blind; v2's player-aware
composite remains out of the tables, because you cannot measure a shooter against a
yardstick that already knows who they are. Everything below this line —
the training discipline, the segregated regimes, the held-out scorecard — describes the model
now in use. The promotion's full reads, including the ones that went against it, are in the
canonical-baseline
adjudication.
How it is trained
- Same-era, grouped by game. Evaluation uses K-fold cross-validation split by whole game (a shot and its rebound never straddle the split), with train and test drawn from the same era — the game changes over time, so we never train on old seasons to score new ones.
- Two prediction columns, never conflated. Out-of-fold predictions are the honest source for performance; the in-sample final model produces the canonical xg used for aggregation (uniform scoring across every shot). The tables on this site are the in-sample canonical column — labeled as such, not a held-out score.
- Segregated regimes. Empty-net and penalty shots are scored by dedicated models, so their potent goal environments never contaminate defended-net predictions.
Honest limitations
- xG is a model of chance quality, not a verdict — it is calibrated in aggregate, but any single shot's number carries uncertainty.
- Shooter/goalie identity is not in this model (by design — that is what makes finishing measurable). The talent-aware variant is its own post.
- Coordinates are arena-adjusted but imperfect; blocked-shot origins are imputed.
This build
- Model id:
flat__joint_structure_xg__4f64c88b(flat) - Season shown: 2025-26
- Displayed xg: in-sample canonical column (aggregation)
How well does the xG model score shots?
Every number in this section is out-of-fold — each shot scored by a model that never saw its game in training — over 2,344,674 shot attempts (112,235 goals) across the full corpus. This is the honest held-out report, not the in-sample column the tables use.
| Metric | Value | Reading |
|---|---|---|
| Danger AUC (goal vs save, on-net) | 0.773 | ranks real scoring chances; immune to easy blocked-shot separability |
| All-shots AUC | 0.822 | includes blocks/misses — flattering, shown for context |
| Brier score | 0.04122 | mean squared error of the probabilities (goal rate 4.8%) |
| Log-loss | 0.15752 | the training objective, held-out |
Calibration — when we say 10%, does it happen 10% of the time?
Shots grouped into ten equal-count bins by predicted xG; each point compares the bin's mean prediction to its observed goal rate. On the dotted line = perfectly calibrated. Log scales, so every decade from long-range point shots to point-blank chances is visible.
Stability by season
Discrimination (AUC) and calibration error (Brier) per season — one model, sixteen seasons, no season-specific tuning.
What the model looks at
This chart is temporarily missing, and rather than quietly drop the section: it broke down per cascade stage (blocked? → on net? → goal?), which only the old nested model had. The canonical model is now a single booster, so the per-stage split has nothing to read. A single-table equivalent has not been built yet, and nothing here is claiming it in the meantime.
The game engine — how well does it call games?
The matchup engine (matchup__6a856b2b) predicts game
winners from the xG pipeline. How it works, in one breath: every team carries
ratings — opponent-adjusted, league-centered expected-goals rates per 60
(5v5 attack & defense, special teams, discipline, finishing & saving residuals),
updated game by game with a 40-game half-life and shrunk ⅓ toward average each summer. A
Poisson regression turns tonight's two rating vectors + rest/schedule
features into expected goals for each side (λ); a score grid over all
possible finals converts the λs into win, tie, and total-goals probabilities; an
OT/shootout layer settles the tie mass.
Every number below is from the walk-forward backtest: to score season N, the engine is fit only on seasons before N — it never peeks. Seasons through 2018-19 were the development window (used for tuning); the shaded holdout was never used to make a modeling choice. The Games page tracker is the third, strictest tier: predictions published before puck drop this season, scored as issued.
Season by season vs the challengers
Log-loss per season (lower is better) for the shipped Poisson engine and its three comparators: a logistic classifier on the same ratings, goals-only Elo (no xG — "did the pipeline add anything?"), and the home-rate constant (always predict the league home-win rate). The first seasons train on very little history; the engine finds its feet as the ledger grows.
| Engine (holdout, 8,510 games) | log-loss | Brier | accuracy |
|---|---|---|---|
| poisson | 0.665 | 0.236 | 58.6% |
| logistic | 0.663 | 0.236 | 59.3% |
| elo | 0.670 | 0.239 | 58.6% |
| constant | 0.691 | 0.249 | 53.6% |
Yes — the plain logistic edges the Poisson on winner log-loss (~0.002). It optimizes exactly this one number; the generative Poisson is kept as the headline because only it produces the score grid behind expected goals, totals, and the matchup maps. Both clearly beat Elo: the xG pipeline earns its keep. Interestingly, when Poisson and Elo pick different winners they split those games about 50/50 — the engine's edge is probability accuracy, not contrarian picks.
Trust the number — the confidence record
Holdout games grouped by how confident the engine said it was about the favorite. A calibrated forecaster's 60–65% games should land ~62% — and they do.
| Stated confidence | record | expected | actual |
|---|---|---|---|
| 50–55% | 1644–1600 | 52.5% | 50.7% |
| 55–60% | 1473–1034 | 57.4% | 58.8% |
| 60–65% | 1008–581 | 62.2% | 63.4% |
| 65–70% | 585–234 | 67.2% | 71.4% |
| 70%+ | 280–71 | 73.4% | 79.8% |
Goals, not just winners — the totals check
The engine's λs claim to be expected goals, so hold them to it. The curve rises decile by decile: when the engine predicts more goals, teams score more goals — the ranking signal is clean. But the whole curve sits above the dotted perfect-calibration line: predictions run about 4.4 tenths of a goal per game cold, because a model trained only on past seasons lags the NHL's rising scoring environment. That bias cancels in win probabilities (both teams shift together) — it shows up here, so we show it here.
The same story at the game level: predicted vs actual distribution of total goals (5.74 predicted vs 6.18 actual per game). Even biased cold, the matchup-specific totals still beat a climatology baseline on over/under 5.5 (log-loss 0.696 vs 0.698) — there is real signal to unlock once the scoring-environment lag and the independent-Poisson assumption (it under-counts ties) are addressed. Both fixes are queued ahead of any totals-facing feature.
Deeper reading
Living analysis reports (calibration, agreement, arena adjustment, blocked-shot imputation) are in the project repository:
- OOF-vs-final agreement — why the in-sample column is safe for aggregation
- Empirical vs model — calibration in feature space
- Model baseline — flat vs nested, danger AUC, the blocked-shot fingerprint
- Full model contract
- Game-engine contract — ratings, score model, walk-forward methodology
- Engine improvement sweep — what was tried, what won, what honestly failed