Mistake Master
Actual minus predicted, in that order
A residual is what the model missed for one individual, and it is computed in one order: actual minus predicted. Reverse the subtraction and every sign flips, so over-prediction and under-prediction trade places. Collected into a plot, the residuals answer a question the scatterplot answers less sharply and the correlation cannot answer at all: is a straight line the right shape.
§1
Residual equals actual minus predicted.
▸
$$\text{residual} = y - \hat{y}.$$
The order is fixed, and the sign carries the meaning:
- Positive residual: the actual value is above the line, so the model under-predicted for that individual.
- Negative residual: the actual value is below the line, so the model over-predicted.
A student who studies 7 hours scored 86, and the model $\widehat{\text{score}} = 58.5 + 3.5(\text{hours})$ predicts $58.5 + 24.5 = 83$. The residual is $86 - 83 = +3$ points: this student scored 3 points more than the model expected, so the model under-predicted.
Computing $\hat{y} - y$ instead reverses every sign and turns a systematic under-prediction into an apparent over-prediction. The phrase to keep is that a positive residual means the point sits above the line, which is easy to check against the plot.
§2
Reading a residual in context.
▸
The residual is in the units of the response and reports the size of the miss for one individual: "this student scored 3 points higher than the model predicted for someone who studies 7 hours."
Three things a residual is not:
- It is not an error in the slope. It measures a gap between one observation and the line, not a defect in the coefficients.
- It is not a distance in both directions. Residuals are measured vertically, from point to line, not perpendicular to the line.
- A large residual is not automatically an outlier to be removed. It is a point the model fits poorly, which is information about the model as much as about the point.
For the eight students the residuals are $0$, $-1.5$, $+1$, $+1.5$, $-2$, $+0.5$, $+3$, $-2.5$. They sum to 0, which is always true for a least-squares line and makes a useful arithmetic check.
§3
The residual plot puts the residuals against x.
▸
A residual plot graphs the residuals on the vertical axis against the explanatory variable on the horizontal, with a horizontal line at 0. It magnifies departures from the model that a scatterplot compresses, because the line has been subtracted out.
What to look for is one thing: a pattern.
- No pattern, random scatter around 0: a linear model is appropriate. This is the good case, and it is easy to misread as evidence of a bad fit precisely because it looks like nothing.
- A curved pattern, such as a U shape or an arch: the relationship is not linear, and a straight-line model is the wrong shape whatever the correlation says.
- A fan or funnel: the spread of the residuals changes with $x$, so the model's predictions are more precise in some ranges than others.
The eight students' residuals scatter around 0 with no visible pattern, which supports the linear model.
§4
A pattern in the residuals overrules a large correlation.
▸
The most costly misreading in the topic is excusing a clear curve because $r$ is large. A tightly curved pattern can give $r = 0.95$, and the residual plot will show a pronounced arch. The residual plot wins: it is answering the question of whether the shape is right, and $r$ only ever measured how well a straight line does.
The opposite misreading costs just as much. A residual plot showing random scatter is exactly what a good model produces, and calling it "no pattern, so the model fails" inverts the criterion. Random is the target.
So the sequence is: draw the scatterplot, fit the line if the form looks linear, then check the residual plot before trusting anything. A model defended from the scatterplot and the correlation alone has skipped the step designed to catch it.
§5
Skill Check.
▸
Ten scenarios. Pick the chips that match your answer, then check. A scenario marks complete the first time every part is right. Progress saves on this device.