thurstone

The fast ability transform

The lattice algorithm of Cotton (2021) turns the Thurstone multi-entrant problem into bookkeeping over three lattice vectors per group of horses. The trick that makes it scale is being able to remove one runner from the field in $O(L)$ — without that the inverse loop would be $O(N^2 L)$ per iteration.

Setting

Discretise performance to a uniform lattice $\{-Lu, \dots, Lu\}$ of $2L+1$ points with spacing $u$. Each runner $i$ has a lattice density $f_i$ and survival $S_i(j) = \Pr(X_i > j) = 1 - F_i(j)$. Lower scores are better. We want the state price $$p_i \;=\; \mathbb{E}\!\left[\frac{\iota_{X_i = X^{(1)}}}{\sum_k \iota_{X_k = X^{(1)}}}\right],$$ where $X^{(1)}$ is the first order statistic. The denominator handles dead-heats: a runner who ties with $m$ others for first collects $1/(1+m)$.

The triple: $(f, S, m)$

Represent any subset $A$ of runners by three lattice vectors:

The first two are redundant ($S = 1 - F$) but it's cleaner to carry both. The third is the ingredient that lets us split tie mass correctly on the lattice — without it the discrete approximation drifts away from the underlying continuous Thurstone model. A single horse is a group with $m_i \equiv 1$.

Operation 1: union (combine two groups)

By independence the survival multiplies: $S_{A \cup B} = S_A \cdot S_B$. Difference for the density. Multiplicity is a weighted combination, indexed by which group(s) produce the tying runners at lattice point $j$: $$ m_{A \cup B}(j) \;=\; \frac{m_A f_A S_B \;+\; (m_A + m_B)\,f_A f_B \;+\; m_B f_B S_A} {f_A S_B \;+\; f_A f_B \;+\; f_B S_A}. $$ The numerator weights each group's multiplicity by the probability that the win comes from that group; the middle term is the cross-group tie. Applied to a single horse at a time, this gives the whole-field representation in $N$ steps of $O(L)$.

Operation 2: removal (take one runner out) — the scalable trick

Given the full field $\Upsilon = (f, S, m)$, the "field minus runner $i$" is $$ S_{\hat\imath}(j) \;=\; \frac{S(j)}{S_i(j)}, \qquad f_{\hat\imath}(j) \;=\; -\Delta S_{\hat\imath}(j), $$ and the multiplicity inverts cleanly from the union identity: $$ m_{\hat\imath}(j) \;=\; \frac{m\,f_i S_{\hat\imath} \;+\; m\,f_i f_{\hat\imath} \;+\; m\,f_{\hat\imath} S_i \;-\; m_i f_i S_{\hat\imath}} {f_{\hat\imath}\,(f_i + S_i)}. $$ That is the heart of the algorithm. Building the whole-field triple is $O(NL)$; producing every "field minus one" triple is then $O(L)$ per runner, instead of $O(NL)$ per runner from scratch. Without it the inverse iteration would not be tractable beyond toy fields.

Operation 3: comparison (per-runner expected payoff)

Given runner $i$'s own $(f_i, S_i)$ and the "rest" triple $(f_{\hat\imath}, S_{\hat\imath}, m_{\hat\imath})$, the expected payoff is $$ p_i \;\approx\; \sum_j f_i(j)\, \Bigl\{\, \frac{f_{\hat\imath}(j)}{1 + m_{\hat\imath}(j)} \;+\; S_{\hat\imath}(j) \,\Bigr\}. $$ The first term inside the braces handles the tie at $j$ (paid $1/(1+m_{\hat\imath}(j))$, where the multiplicity is the rest-of-field expectation at that grid point); the second is the outright-win contribution where the rest of the field strictly trails. The only approximation is replacing $\mathbb{E}[1/M\,|\,j]$ by $1/\mathbb{E}[M\,|\,j]$ in the tie term, and that bias is negligible because $M \ge 2$ on the tie event. 

import { Race } from "../js/thurstone/index.js";
const prices = new Race(densities).statePrices();
// Internally: winnerOfMany() builds (f, S, m) for the whole field once via Operation 1,
// then expectedPayoffWithMultiplicity() peels off each runner via Operations 2 + 3.

The inverse loop

With those three operations the inverse problem becomes a few cheap passes:

  1. Initialise $\mu_i = 0$ for all runners. Build the field triple $(f, S, m)$ once via Operation 1.
  2. For a shared grid of candidate offsets $g_1, \dots, g_K$, compute the implied price $\hat p(g_k)$ of a single shifted runner against the current field — one Operation 3 per grid point, reading $(f_{\hat\imath}, S_{\hat\imath}, m_{\hat\imath})$ from Operation 2. This produces a global curve $\hat p(g)$.
  3. Sort the curve by price, dedupe, and invert each observed $p_i$ by monotone linear interpolation. Read off $\mu_i = g \cdot u$.
  4. Rebuild the field with the new $\mu_i$ and repeat. Three iterations usually suffice.
import { AbilityCalibrator } from "../js/thurstone/index.js";
const cal     = new AbilityCalibrator(base, { nIter: 3 });
const ability = cal.solveFromDividends([3.2, 4.8, 12.0, 7.5, 20.0]);

Walkovers and runners off the lattice

What if a runner is so much better than the field that its offset hangs off the left edge of the lattice (or so much worse it hangs off the right)? The ClusterSplitter handles this by splitting the field at the largest gap, computing group representative winning densities, and recursing on the weaker side. Edge cases (runner at $+\infty$ → cannot win; runners at $-\infty$ → equal share of certainty) fall out naturally. See the walkovers & ties demo.

What you get back

Abilities returned by the JS calibrator are in physical units of the lattice (same units as the grid spacing). They are median-centered, since the model is translation-invariant: only differences of abilities have an unambiguous meaning, not their absolute level.

Trustworthy in your browser

Every JavaScript module on this site is verified against the Python reference using JSON golden fixtures emitted by scripts/generate_fixtures.py. The test suite (docs/tests) asserts max-absolute-difference under tight tolerances:

Play with it

Now go break things. Drag sliders, paste dividends, watch the curve invert in real time: