rMetrics: A statistically motivated framework for player evaluation using residualized scores

Robert Bajons | Joint work with Lucas Kook

Vienna University of Economics and Business

Sep 27, 2025

Motivation

Player evaluation

How do we commonly evaluate players in sports?

Sports analytics:

Expected value metrics \(\rightarrow\) compare observed and expected outcome:
- Estimate expected outcome for game state (e.g. expected goals, expected pass completion)
- For all outcomes of a player (e.g. shots, passes): Compute differences between outcome and expected value for outcome.

Examples:
- Soccer and ice hockey: Goals (saved) above expectation (G(S)AX)
- Basketball: Quantified shooter impact (qSI)
- American football: Completion percentage above expectation (CPAE)

Motivational example

Goals above expectation (GAX):
over a time frame (e.g. a season), compute differences between goals and xG for all shots of player \(i\)
\[ \operatorname{GAX}_i = \sum_{j=1}^{N_i} (Y_j - \hat h(Z_j))\]
- \(N_i \dots\) Number of shots of player \(i\)
- \(Y_j \dots\) actual outcome for shot \(j\)
- \(\hat h(Z_j) \dots\) estimator for \(h(Z) = \mathbb{E}[Y|Z]\) (xG for shot \(j\))
- \(Z \dots\) shot-specific features (distances, angles, …)

Motivational example

Recent criticism of GAX (Baron et al. 2024; Davis and Robberechts 2024):

Instability and limited replicability (over seasons)
Low (effective) sample size \(\rightarrow\) high uncertainty, lack of uncertainty quantification
Biases arising from data:
- Top teams and top players taking more shots
- Including all shots instead of fraction (headers vs footers, long-distance vs short-distance)

\(\Rightarrow\) GAX have been labeled a poor measure for evaluating shooting skills

Motivational example

Recent criticism of GAX (Baron et al. 2024; Davis and Robberechts 2024):

Instability and limited replicability (over seasons)
Low (effective) sample size \(\rightarrow\) high uncertainty, lack of uncertainty quantification
Biases arising from data:
- Top teams and top players taking more shots
- Including all shots instead of fraction (headers vs footers, long-distance vs short-distance)

\(\Rightarrow\) GAX have been labeled a poor measure for evaluating shooting skills

Are GAX really a poor metric for player evaluation?

A framework for player evaluation

A parametric model

How can we identify outstanding shooters?

Setup:
- \(Y\) … binary outcome of a shot (goal/no goal)
- \(Z\) … shot-specific variables
- \(X\) … binary indicator of a player’s involvement (shooter/not shooter)

Logistic regression model:

\(Y \mid X,Z \sim \operatorname{Ber}(\pi(X,Z)), \quad \pi(X,Z) = P(Y=1 \mid X,Z)\) and

\[ \begin{aligned} \log\left(\frac{\pi(X,Z)}{1-\pi(X,Z)}\right) = X\beta + Z^{\top}\gamma. \end{aligned} \]

Goal: Inference on \(\beta\)
- Wald test, LR test, score test

Score test and GAX

Given i.i.d data \((Y_i,X_i,Z_i)_{i = 1}^N\) from the logistic regression model

Score of \(\beta\) (target of score tests): …………………………………………..

\[ \sum_{i = 1}^N\frac{\partial\log L(\beta,\gamma \mid Y_i,X_i,Z_i)}{\partial \beta}\]

Score test on \(\beta\) uses score under \(H_0: \beta = 0\):
\[\sum_{j=1}^{N} (Y_j - \hat h(Z_j))X_j\]
- \(\hat h(Z_j) = \text{expit} (Z_j^{\top}\hat \gamma)\) … \(\hat \gamma\) is the MLE of \(\gamma\) under \(H_0\)

Recall:

\[ \operatorname{GAX}_i = \sum_{j=1}^{N_i} (Y_j - \hat h(Z_j))\]

Since \(X_j\) binary \(\Rightarrow\) score is exactly GAX for a player
- High GAX (in absolute terms) correctly standardized indicates significant players

GAX for player evaluation

Conclusion: GAX relates to a classical score test in logistic regression model
- Uncertainty quantification (and significance testing) via score test
- Interpretation of player quality as effect on log-odds (probability) of scoring

Problems:
- Linear model assumptions unrealistic
- traditional GAX via machine learning model:
  - \(\sum_{j=1}^{N} (Y_j - \hat{h}(Z_j))X_j, \quad \hat h\) estimated via arbitrary ML algorithm
  - No (valid) uncertainty quantification

A semiparametric extension

Partially linear logistic regression model (PLLM) ………………………………….

\[ \log\left(\frac{\pi(X,Z)}{1-\pi(X,Z)}\right) = X\beta + g(Z) \]

Goal: Inference on \(\beta\) (testing \(H_0 : \beta = 0\))
- Use Generalised Covariance Measure (GCM) test (Shah and Peters 2020)
- Target of GCM test: \(\operatorname{GCM} := \mathbb{E}[\operatorname{Cov}(Y,X \mid Z)] =\mathbb{E}[(Y - \mathbb{E}[Y | Z])(X - \mathbb{E}[X | Z])]\)

GCM test uses empirical GCM:
\[\sum_{j = 1}^N (Y_j-\hat h(Z_j))(X_j - \hat f(Z_j))\]
- residualized version of GAX (rGAX) …………………………………………………
- \(\hat h \dots\) estimator for \(h(Z)\) (xG model) using suitable machine learning model
- \(\hat f \dots\) estimator for \(f(Z) := \mathbb{E}[X \mid Z]\) using suitable machine learning model

GAX vs. rGAX

GAX in parametric model:
\[\sum_{j=1}^{N} (Y_j - \hat h(Z_j))X_j\]
- Only valid inference if linear model assumptions hold

GAX via machine learning:
\[\sum_{j=1}^{N} (Y_j - \hat{h}(Z_j))X_j\]
- \(\hat h\) learned via arbitrary ML algorithm
- No valid inference

rGAX:
\[\sum_{j=1}^{N} (Y_j-\hat h(Z_j))(X_j - \hat f(Z_j))\]
- Valid inference and uncertainty quantification even when using ML models for \(\hat h\) and \(\hat f\)
- Different interpretation of player value: strength estimate in a semiparametric model
- Interpretation of additional regression: accounts for whether a player would take the shot under the circumstances described by \(Z_i\)

GAX vs. rGAX

GAX in parametric model:
\[\sum_{j=1}^{N} (Y_j - \hat h(Z_j))X_j\]
- Only valid inference if linear model assumptions hold

GAX via machine learning:
\[\sum_{j=1}^{N} (Y_j - \hat{h}(Z_j))X_j\]
- \(\hat h\) learned via arbitrary ML algorithm
- No valid inference

rGAX:
\[\sum_{j=1}^{N} (Y_j-\hat h(Z_j))(X_j - \hat f(Z_j))\]
- Valid inference and uncertainty quantification even when using ML models for \(\hat h\) and \(\hat f\)
- Different interpretation of player value: strength estimate in a semiparametric model
- Interpretation of additional regression: accounts for whether a player would take the shot under the circumstances described by \(Z_i\)