What the 95 refers to: the method over many samples, not this one interval

01The mistake

A sample gives a 95% confidence interval of $(0.52, 0.60)$ for the proportion of voters supporting a measure. Ask what the 95% means. The most common answer is that there is a 95% chance the true proportion is between 0.52 and 0.60. The parameter is a fixed number and the interval is a fixed interval, so that probability is either 1 or 0 and nobody knows which.

The second most common answer moves the 95 onto the data: 95% of the sampled voters fall in the interval. That confuses an interval for a parameter with an interval for individual observations, and the numbers make it visibly impossible — an interval of width 0.08 on a proportion scale does not contain 95% of anything.

A third version keeps the right words and loses the referent: “we are 95% confident in our sample.” Confidence attaches to the interval's capture of a population parameter, and a sentence that names the sample as the target describes a quantity nobody needs an interval for, since the sample statistic was computed exactly.

All three are costly because interval interpretation is a standing exam task with a published scoring expectation. The arithmetic is usually fine. The sentence is where the points go.

02Why it makes sense to the student

The reversed version is what students want to know, exactly as with p-values. The useful question is where the parameter is, and the confidence level is the only percentage in sight, so it gets read as the answer to that question.

The word “confidence” invites it. In ordinary use, being 95% confident in a statement is a probability claim about the statement. The technical term borrows the everyday word and means something narrower, and nothing in the phrase signals the difference.

The procedure hides the long-run idea. A student computes one interval from one sample and never sees the other 999 intervals the method would have produced, so the only object available for the 95 to describe is the one on the page. The thing the level actually describes is invisible unless it is simulated.

And the parameter's fixedness is a genuinely unfamiliar idea. Students have spent years treating unknown quantities as variables, so treating the population proportion as a fixed number that simply is not known yet takes real work. Without it, assigning a probability to its location feels unobjectionable.

03The correction

Simulate it, because this is one of the few interpretation errors that a picture dismantles completely. Draw 100 samples from a population with a known proportion, build an interval from each, and plot them as 100 horizontal segments with the true value as a vertical line. About 95 cross the line and about 5 do not. The 95 is visibly a count of intervals, not a property of any one of them.

Then point at a single interval in that display and ask what probability it has of containing the parameter. The answer is that it either does or does not, and a student looking at the picture can see which. That is the moment the fixedness of the parameter becomes concrete rather than a stated rule.

Require the method in the sentence: “We are 95% confident that the interval from 0.52 to 0.60 captures the true proportion of all voters who support the measure.” The verb is “captures,” the subject is the interval, and the object names the population and the parameter. A sentence missing any of the three is missing a scoring element.

Give them the long-run gloss as a second sentence they can fall back on: if this sampling procedure were repeated many times, about 95% of the resulting intervals would contain the true proportion. Students who can produce either sentence have the idea; students who can only produce “95% chance” have the number and not the object.

Worth testing with a stem that makes the data-based reading obviously false. An interval for a mean weight of $(142, 148)$ pounds cannot plausibly contain 95% of the people in a sample, and students who choose that option will often catch themselves when asked whether the heaviest person in the sample is inside it.

04A sample question

05What each wrong answer reveals

• A Probability assigned to a fixed parameter. The dominant wrong answer and the hardest to dislodge, because it is nearly a correct sentence and it answers the question anyone would actually ask. Ask whether the true proportion is a fixed number. Once a student agrees that it is, they can usually see that a fixed number is either inside a fixed interval or outside it, with no room for 95%.
• B Correct. The confidence level describes the long-run capture rate of the method across repeated samples. The interval is the subject and the parameter is what gets captured.
• C The level moved onto the individual observations. This student is treating a confidence interval like an interval that describes spread in the data, which is the role of a standard deviation or a percentile range. The units give it away: individual voters answered yes or no, so no voter has a value of 0.55. Asking what a single sampled voter's value even is usually ends this one quickly.
• D Right form, wrong target. The grammar of a correct interpretation with the sample substituted for the population. Worth noting that the sample proportion is 0.56 and was computed exactly, so there is nothing to be confident about. This student has learned the sentence pattern and not yet learned that inference is always about the population.

A and D are both close to the right sentence and fail on different words — A on the object of the probability, D on the population. C is a different misconception entirely, about what kind of interval this is. The simulation picture fixes A and B; C needs the distinction between describing data and estimating a parameter.

06Try it in Mistake Master