Installing the ATLAS calorimeter · Image credit: Maximilien Brice/CERN · CC BY 4.0
Topic
Statistics of Discovery
How physicists decide that something is real — and how they get fooled.
Every measurement comes with noise. Detectors count random background events, instruments drift, and nature itself is probabilistic. So before anyone can say we found something, they have to answer a harder question: how likely is it that the pattern in the data is just noise?
This hub is about that question. It covers what a p-value does and does not mean, why particle physicists ask for five sigma before they say discovery, and why even five sigma can be fooled by an error nobody noticed. It is the logic behind the name of this site.
Suggested reading order
What “five sigma” really means — and what it doesn’t
Particle physicists reserve “discovery” for five sigma: a bump chance alone would make about once in 3.5 million tries. What that measures, why it isn’t “99.99997% sure,” and how five-sigma results still went wrong.
Key concepts
- null hypothesis
- The boring explanation you try to rule out, such as “there is no new particle, only things we already know about.”
- p-value
- If the boring explanation were true, the chance of seeing a result at least as extreme as the one you actually got.
- standard deviation
- The typical size of the random scatter of measurements around their average.
- look-elsewhere effect
- The more places you search for a surprise, the more likely you are to find a fake one somewhere just by luck.
- systematic error
- A mistake or doubt in how the measuring itself is done, such as a mis-calibrated clock, that often does not shrink just by collecting more data.
Other topics: Quantum · Relativity · Cosmology · Particles & Neutrinos · Spaceflight