Mistake Master
Student view — seeing the site as a student does
Home Unit 3 · Inference for Categorical Data: Proportions 3.1·3.2·3.3·3.4·3.5·3.6·3.7·3.8·3.9·3.10·3.11·3.12·3.13·3.14·3.15 Lesson
Skill Check 0 / 10 complete

The test says something differs, never what

The statistic adds up how far each cell landed from what the null predicted, scaled by how much was expected there. A large total means the table as a whole does not look like the null's picture. What it never says is which cell is responsible, which direction the departure runs, or what caused it, and those three unavailable conclusions are where a correct calculation usually goes wrong.

§1

The statistic adds one contribution per cell.

$$\chi^2 = \sum \frac{(\text{observed} - \text{expected})^2}{\text{expected}}.$$

Each term measures a cell's discrepancy, squared so that overshoots and undershoots both count, and divided by the expected count so that a gap of 20 matters more where 30 were expected than where 300 were.

For the three-school table, with observed 120 and 80, 90 and 110, 66 and 84 against expected 100.4 and 99.6, 100.4 and 99.6, 75.3 and 74.7:

  1. School A: $\frac{(120-100.4)^2}{100.4} \approx 3.84$ and $\frac{(80-99.6)^2}{99.6} \approx 3.87$.
  2. School B: $\approx 1.07$ and $\approx 1.08$.
  3. School C: $\approx 1.14$ and $\approx 1.15$.

The total is $\chi^2 \approx 12.15$. Every contribution is non-negative, so the statistic can only grow as cells disagree with the null, and a value of 0 would mean the observed table matched the expected one exactly.

§2

The p-value is always an upper tail.

With $df = (3-1)(2-1) = 2$, the p-value is the area to the right of 12.15 under the chi-square distribution with 2 degrees of freedom, about $0.0023$.

The tail is always the upper one, and the reason is structural: only large values of $\chi^2$ indicate disagreement with the null. A small statistic means the table looks like what the null predicted, which is never evidence against it. There is no such thing as a two-sided chi-square test in this course, and no left tail to consider.

The chi-square distributions are right-skewed and change shape with the degrees of freedom, so the same statistic carries different evidence at different table sizes: a $\chi^2$ of 12.15 is strong evidence with $df = 2$ and much weaker with $df = 10$.

At $\alpha = 0.05$, since $0.0023 < 0.05$, reject $H_0$: there is convincing evidence that the distribution of responses is not the same across the three schools.

§3

Three conclusions the test does not license.

It does not name the cell. The alternative says the distributions differ somewhere. Rejecting does not establish that school A is the outlier, however large its contribution looks, because the statistic was never designed to test individual cells.

It does not establish causation. A significant result says the variables are associated in this population, or that the groups differ. Whether one causes the other depends on random assignment, exactly as in every other test.

A large p-value does not prove independence. "$p = 0.2$, so the variables are independent" accepts the null. The correct sentence is that there is not convincing evidence of an association, and a real association could easily go undetected in a modest sample.

One more limit worth stating: the test says nothing about the direction or size of any difference, because squaring erases the sign. A table where group A runs high and a table where it runs equally low give the same statistic.

§4

The follow-up is where the description comes from.

Once the null is rejected, two ordinary tools describe what happened, and both belong in a complete answer:

  1. The components. The largest terms point at the cells contributing most to the total. School A's two cells supply about 7.7 of the 12.15, so the departure is concentrated there. This is a description, not a second test, and it is stated as such.
  2. The conditional distributions. School A supported the change at $\frac{120}{200} = 60\%$, school B at $45\%$, school C at $44\%$. That is the direction and the magnitude the chi-square statistic threw away.

So a complete conclusion reads: there is convincing evidence that support for a later start time differs across the three schools; school A's support rate of 60% runs well above school B's 45% and school C's 44%, and school A's cells contribute most of the statistic.

§5

Skill Check.

Ten scenarios. Pick the chips that match your answer, then check. A scenario marks complete the first time every part is right. Progress saves on this device.

0 of 10 scenarios complete