Not Detected Is Not the Same as Not There

You'll learnhow zero decides the verdict for a difference of means, why an interval containing zero is a failure to detect rather than a finding of equality, how much a small study could have missed, what the width does and does not measure, and how to write a justification that survives a reader who checks it.

One value decides the verdict: zero, the claim that the two population means are equal. An interval entirely on one side of it establishes a direction; an interval straddling it establishes nothing about direction — and in particular does not establish that the means are the same. That last point is usually made as a slogan, so this page computes it instead. A study with 8 per group reporting (−14, 22) would have missed a real 10-point gap 85 percent of the time, and a gap exactly the size of its own margin of error half the time. The 30-and-28 study of the last two topics misses that same 10-point gap 1.3 percent of the time. "No difference detected" is the same sentence from both, carrying wildly different amounts of information. Twelve short steps; press Next for each one.

12 STEPS · 6 QUICK CHECKS · ZERO AND THE VERDICT · WHAT WAS MISSED · WIDTH · v2

Zero decides the verdict, and a study that could not detect a gap has not ruled one out.

Before you start

What you're looking at

Intervals for a difference of two population means, drawn against a red rule at zero — the value that says the two means are equal.

The question

An interval that contains zero: what has it established? The answer is not "the means are equal", and that gap is the whole topic.

Step 1 / 12