Not Detected Is Not the Same as Not There

You'll learnhow zero decides the verdict for a difference of means, why an interval containing zero is a failure to detect rather than a finding of equality, how much a small study could have missed, what the width does and does not measure, and how to write a justification that survives a reader who checks it.

One value decides the verdict: zero, the claim that the two population means are equal. An interval entirely on one side of it establishes a direction; an interval straddling it establishes nothing about direction — and in particular does not establish that the means are the same. That last point is usually made as a slogan, so this page computes it instead. A study with 8 per group reporting (−14, 22) would have missed a real 10-point gap 85% of the time, and a gap exactly the size of its own margin of error half the time. The 30-and-28 study of the last two topics misses that same 10-point gap 1.3% of the time. "No difference detected" is the same sentence from both, carrying wildly different amounts of information.

8 STEPS · 6 QUICK CHECKS · ZERO AND THE VERDICT · WHAT WAS MISSED · WIDTH · v1

(2.40, 12.00) excludes 0 · (−1.2, 4.8) contains it · an 8-per-group study misses a real 10-point gap 85% of the time

Before you start

What you're looking at

Intervals for a difference of two population means, drawn on one axis against a red rule at zero — the value that says the two means are equal.

The question

An interval that contains zero: what has it established? The answer is not "the means are equal", and the difference between those two sentences is the whole topic.

Watch for

Step 4, where the page computes how often a study like the one reporting that interval would have missed a real difference. The number is large, and it is what "failure to detect" actually means.

Step 1 / 8