Four Ways a Survey Breaks, and They Break Different Links

You'll learnto name which link of the chain from population to recorded answer a survey broke, and to say which way that break pushes the estimate and why.

A survey runs down a chain: a population, a frame of everyone who could be picked, the people actually selected, the ones who answer, and the answers as recorded. 240 residents, 58.3% of whom ride the bus weekly. Sample from the car registry and the frame is missing riders — its target is 30.8%. Run an open web poll and nobody is selected at all; the 80 who choose to answer are 87.5% riders. Phone the selected on a weekday and the riders are out; the people you can reach are 41.2% riders. Ask "do you do your part by riding the bus" and everyone answers, but the recorded rate town-wide is 77.1%. Four numbers, two too high and two too low, and not one of them moves when the sample grows. Twelve short steps; press Next for each one.

12 STEPS · 6 QUICK CHECKS · UNDERCOVERAGE · VOLUNTARY RESPONSE · NONRESPONSE · RESPONSE BIAS · v2

p = 58.3% · undercoverage 30.8% · voluntary 87.5% · nonresponse 41.2% · wording 77.1%

Before you start

What you're looking at

One survey drawn as five stages stacked top to bottom. The bar is how many people are left; the rail on the right is what share of them ride.

The question

Four surveys of the same town return four different answers. Which link did each one break?

Step 1 / 12