The Same Twenty Numbers, and Two Answers 32 Times Apart

You'll learnwhich question to ask before writing anything down, what treating paired data as two samples actually costs, why hypotheses name parameters and never statistics, where the alternative's direction comes from, and which conditions run once per group.

Ten runners each try two shoe models. That is twenty numbers in two columns — and whether you subtract within runners or add the two variances is not a matter of taste. This page runs both procedures over the same twenty times: pairing gives t = 15.79 on 9 degrees of freedom, and treating the columns as two independent samples gives t = 0.49 on 18. Overwhelming evidence, or none at all, from one dataset. The reason is on screen in step 3: the runners differ from each other by 5.40 seconds while each runner's own improvement varies by 0.24, so the design question decides whether a 22-fold source of variation sits in your denominator or is subtracted away before you start.

8 STEPS · 6 QUICK CHECKS · THE DESIGN QUESTION · HYPOTHESES · CONDITIONS · v1

ten runners, twenty times · paired t = 15.79 on df 9 · the same data as two samples: t = 0.49

Before you start

What you're looking at

Ten runners, each timed in two shoe models. Twenty numbers in two columns — the shape of data that makes this topic's error easy to commit.

The question

Two columns of numbers: is that two samples, or one sample measured twice? The data cannot tell you. Only how the study was run can.

Watch for

Step 2, where both procedures run over the identical twenty numbers and return t = 15.79 and t = 0.49. The design question is not bookkeeping.

Step 1 / 8