Mistake Master
Student view — seeing the site as a student does
Home Unit 5 · Regression Analysis 5.1·5.2·5.3·5.4·5.5 Lesson
Skill Check 0 / 10 complete

Actual minus predicted, in that order

A residual is what the model missed for one individual, and it is computed in one order: actual minus predicted. Reverse the subtraction and every sign flips, so over-prediction and under-prediction trade places. Collected into a plot, the residuals answer a question the scatterplot answers less sharply and the correlation cannot answer at all: is a straight line the right shape.

§1

Residual equals actual minus predicted.

$$\text{residual} = y - \hat{y}.$$

The order is fixed, and the sign carries the meaning:

  1. Positive residual: the actual value is above the line, so the model under-predicted for that individual.
  2. Negative residual: the actual value is below the line, so the model over-predicted.

A student who studies 7 hours scored 86, and the model $\widehat{\text{score}} = 58.5 + 3.5(\text{hours})$ predicts $58.5 + 24.5 = 83$. The residual is $86 - 83 = +3$ points: this student scored 3 points more than the model expected, so the model under-predicted.

Computing $\hat{y} - y$ instead reverses every sign and turns a systematic under-prediction into an apparent over-prediction. The phrase to keep is that a positive residual means the point sits above the line, which is easy to check against the plot.

§2

Reading a residual in context.

The residual is in the units of the response and reports the size of the miss for one individual: "this student scored 3 points higher than the model predicted for someone who studies 7 hours."

Three things a residual is not:

  1. It is not an error in the slope. It measures a gap between one observation and the line, not a defect in the coefficients.
  2. It is not a distance in both directions. Residuals are measured vertically, from point to line, not perpendicular to the line.
  3. A large residual is not automatically an outlier to be removed. It is a point the model fits poorly, which is information about the model as much as about the point.

For the eight students the residuals are $0$, $-1.5$, $+1$, $+1.5$, $-2$, $+0.5$, $+3$, $-2.5$. They sum to 0, which is always true for a least-squares line and makes a useful arithmetic check.

§3

The residual plot puts the residuals against x.

A residual plot graphs the residuals on the vertical axis against the explanatory variable on the horizontal, with a horizontal line at 0. It magnifies departures from the model that a scatterplot compresses, because the line has been subtracted out.

What to look for is one thing: a pattern.

  1. No pattern, random scatter around 0: a linear model is appropriate. This is the good case, and it is easy to misread as evidence of a bad fit precisely because it looks like nothing.
  2. A curved pattern, such as a U shape or an arch: the relationship is not linear, and a straight-line model is the wrong shape whatever the correlation says.
  3. A fan or funnel: the spread of the residuals changes with $x$, so the model's predictions are more precise in some ranges than others.

The eight students' residuals scatter around 0 with no visible pattern, which supports the linear model.

§4

A pattern in the residuals overrules a large correlation.

The most costly misreading in the topic is excusing a clear curve because $r$ is large. A tightly curved pattern can give $r = 0.95$, and the residual plot will show a pronounced arch. The residual plot wins: it is answering the question of whether the shape is right, and $r$ only ever measured how well a straight line does.

The opposite misreading costs just as much. A residual plot showing random scatter is exactly what a good model produces, and calling it "no pattern, so the model fails" inverts the criterion. Random is the target.

So the sequence is: draw the scatterplot, fit the line if the form looks linear, then check the residual plot before trusting anything. A model defended from the scatterplot and the correlation alone has skipped the step designed to catch it.

§5

Skill Check.

Ten scenarios. Pick the chips that match your answer, then check. A scenario marks complete the first time every part is right. Progress saves on this device.

0 of 10 scenarios complete