The Effect Is Known, So Every Design Can Be Scored Against It

You'll learnwhat each element of an experiment is actually for, by watching designs that leave one out miss an effect whose true value the scene already knows.

A stretching routine really does make people faster: 2 seconds for an athlete, 8 seconds for everyone else. In a town of 80 athletes and 160 others that is a mean effect of exactly 6 seconds, and among the 100 people who volunteer for the study it is exactly 4.4. Both numbers are true and they are not the same number, which is the whole of the difference between random selection and random assignment. Let volunteers pick their own group and the comparison reports 14.9 seconds, because athletes pick the routine and were already 11.8 seconds faster. Assign by chance instead and forty such studies average 4.0. Nothing about the arithmetic changed — only who ended up in which group, and that is the one thing an experiment is allowed to decide.

8 STEPS · 6 QUICK CHECKS · COMPARISON · RANDOM ASSIGNMENT · REPLICATION · CONTROL · BLINDING · v1

true effect: −2 s athletes, −8 s others · town −6.0 s · volunteers −4.4 s

Before you start

What you're looking at

Each strip is one group. Every short tick is one person's 400 m time; the tall tick is that group's mean, printed at the right. Lower is faster.

The question

The routine's real effect is already known here. Which designs recover it, and which return something else?

Watch for

The gap between the two groups BEFORE either is treated. Whenever that gap is not zero, the design is measuring it along with the treatment.

Step 1 / 8