Mistake Master
Every condition gets checked twice
The two-sample interval is the one-sample interval with a wider standard error and one more thing to say at the end. What makes it easy to get wrong is that almost every check now happens twice: two random samples, two 10% conditions, two shape assessments. A study with forty observations in one group and nine in the other still has to justify the nine.
§1
Difference, plus or minus t-star times the standard error.
▸
$$(\bar{x}_1 - \bar{x}_2) \pm t^{*}\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}.$$
Two teaching methods: method A gives $\bar{x}_1 = 78.4$ with $s_1 = 8.1$ from $n_1 = 30$ students, method B gives $\bar{x}_2 = 71.2$ with $s_2 = 9.6$ from $n_2 = 28$. The difference is 7.2 points and
$$SE = \sqrt{\frac{8.1^2}{30} + \frac{9.6^2}{28}} \approx 2.341.$$
Using the conservative $df = \min(30, 28) - 1 = 27$, the 95% critical value is $t^{*} = 2.052$, so
$$7.2 \pm 2.052(2.341) = 7.2 \pm 4.80,$$
giving about $(2.40, 12.00)$ points. Technology's $df$ of about 53 would give a slightly narrower interval; either is acceptable as long as the choice is stated.
§2
The conditions, checked twice and written out.
▸
Four conditions, three of them per group:
- Random: both samples random, or subjects randomly assigned to the two methods.
- Independence between groups: no student appears in both. Two measurements on the same students would be paired, and Topic 4.2's procedure would apply instead.
- 10%: each sample at most a tenth of its own population, checked separately.
- Normal or large enough, per group: for each group, either $n \ge 30$, a stated normal population, or a plot showing no strong skew and no outliers.
The last one is where studies quietly fail. Group A here has $n_1 = 30$ and clears the guideline; group B has $n_2 = 28$ and does not, so its plot has to be examined and described. A single sentence such as "the boxplot of the 28 method-B scores is roughly symmetric with no outliers" is what turns the condition from a name into a check.
Two opposite errors follow from skipping the plots. Refusing the analysis because "the populations might not be normal" at $n_1 = 40$ and $n_2 = 45$ ignores the Central Limit Theorem. And invoking the CLT for a group of 7 asks it to do work it cannot at that size.
§3
The degrees of freedom, and what not to use.
▸
Two accepted values, as in the previous topic:
- Technology's fractional value, about 53.0 for this study.
- The conservative $\min(n_1, n_2) - 1$, which is 27, giving a slightly larger $t^{*}$ and a slightly wider interval.
Here the two differ little: $t^{*} = 2.052$ at $df = 27$ against about 2.006 at $df = 53$, so the margin of error moves from 4.80 to 4.70. The conservative choice errs toward a wider interval, which is why it is safe to use by hand.
What is never right is $n_1 + n_2 - 2$ picked from memory, or a one-sample $df = n - 1$ using one of the two group sizes. Both understate the uncertainty in one group or the other.
§4
The interpretation names both populations and the direction.
▸
"We are 95% confident that the interval from 2.40 to 12.00 points captures the true difference in mean score, method A minus method B, for all students who would be taught by these methods."
Four elements: the confidence level, the fact that the interval captures a difference between two population means, the order of subtraction, and the two populations. Losing the order makes the sign unreadable; losing the word difference turns a claim about a gap into a claim about one group's score.
The usual wrong referents apply here too. "95% of students score between 2.40 and 12.00 points" describes individuals and is not even on the right scale. "There is a 95% probability that the true difference is in this range" treats a fixed parameter as random.
Whether the difference may be called causal depends on the design. Students randomly assigned to the two methods license a causal reading; two intact classes that happened to use different methods do not, since the classes may differ in preparation, teacher, or time of day.
§5
Skill Check.
▸
Ten scenarios. Pick the chips that match your answer, then check. A scenario marks complete the first time every part is right. Progress saves on this device.