Statistical inference: likelihoods, SBI & calibration

Turning a simulation or observation into a boson-mass constraint is a statistics problem. This article covers the inference toolkit — likelihoods, simulation-based inference, and the calibration checks — that makes a claimed constraint trustworthy.

From data to a parameter

Inference asks: given data $D$, what is the posterior $P(m\,|\,D)\propto\mathcal L(D\,|\,m)\,\pi(m)$ over the boson mass? The likelihood $\mathcal L$ encodes how probable the data are for each $m$; the prior $\pi$ encodes what we knew before. The width of the posterior is the constraint.

When the likelihood is intractable

For simulations the likelihood is often not writable in closed form — the map from $m$ to a halo core or a power spectrum runs through an expensive, non-analytic simulation. Simulation-based inference (SBI) solves this: run the simulator at many $m$, train a neural density estimator on the (mass, observable) pairs, and read off the posterior (figure). No explicit likelihood needed.

Simulation-based inference: a broad prior on the boson mass (dashed) is sharpened by the data into a calibrated posterior (purple) centred on the true value — with credible intervals verified by simulation-based calibration.

Calibration is non-negotiable

Worked example — checking the posterior is honest

A posterior is only useful if its error bars mean what they say. We test this with simulation-based calibration: over many mock datasets, a well-calibrated 50/68/90% credible interval should contain the truth 50/68/90% of the time. Our pipeline gives coverage 50/69/91% — calibrated. An ablation that ignores the intrinsic core–halo scatter gives 26/38/58% — badly overconfident. Calibration is what separates a real constraint from a fooling one.

Why it matters for the tension

The 8.7$\sigma$ tension (Topic 10.4) is a statement about combining posteriors, so it is only as trustworthy as those posteriors are calibrated. Getting the scatter right (the core–halo dispersion) is essential — ignore it and you overstate the precision, manufacturing a tension that isn't there, or hiding one that is. Rigorous, calibrated inference is what makes the result defensible.

Simulation-based calibration: empirical coverage matches the nominal credible level (50/69/91%).

In our research

This is the statistical backbone of our differentiable inference (Topic 9.6) and the mass tension (Topic 10.4). Our SBI recovers the boson mass to ~5% with calibrated coverage (50/69/91%), and the calibration check is what lets us quote the 7–9$\sigma$ tension as robust rather than an artifact of overconfident error bars.

Key references

Cranmer, Brehmer & Louppe (2020), PNAS 117, 30055 (arXiv:1911.01429).
Talts et al. (2018), Validating Bayesian inference via SBC (arXiv:1804.06788).
Zhao et al. (2022), SBI for cosmology — methodology review.

PreviousThe mass tension: combining the probes NextQuantum algorithms for cosmological simulation