This lesson measures how well the least-squares line fits. R-squared compares the line's squared misses, the squared residuals, with those of predicting the mean for everyone, and reports the share the line removes: 0 if it does no better than the mean, 1 if every point lies on it. We use five workers, then 20 Premier League clubs.
In this lesson
- What the line leaves over: 20 residuals
- Residuals are not errors
- How big is a typical residual?
- The line against the mean: TSS, ESS and RSS
- R-squared for the 20 clubs
- What R-squared does not tell you
- Why a slope cannot be inverted: y on x and x on y
- Common mistakes
- Check your understanding: five marked questions
- Three questions about the arithmetic
- Where next
1. What the line leaves over
In lesson 9 we fitted a line to the 20 Premier League clubs of 2023/24: points = 52.30 + 0.6235 × goal difference. The line gives each club a fitted value, the points total its goal difference predicts. Aston Villa had a goal difference of +15, so their fitted value is 52.30 + 0.6235 × 15 = 61.65 points. They finished on 68. The gap, 68 - 61.65 = +6.35, is their residual: what the line leaves over for that club.
Figure 1 shows the residual for every club.
West Ham finished 8.4 points above what their goal difference predicted and Nottingham Forest 9.1 below (4 of those are a points deduction). Manchester City are almost exactly on the line, 0.05 of a point above it. The residuals add to zero, which is the first of the three checks from lesson 9: the line is placed so that the misses above it and the misses below it balance.
The residuals are where the line's fit can be judged. If they are small compared with how much points totals differ between clubs, the line summarises the data well. The rest of the lesson turns that sentence into a number.
What students say
“Nicky is an excellent econometrics tutor. He explains complex concepts clearly and makes them much easier to understand.”
“I was completely lost in my economics class, and then I started working Nicky and he totally saved me. He was excellent at explaining the material and extremely accommodating to meet with me many times despite us being in different time zones! Highly recommend working with Nicky!”
“Nicky is a very good and thorough teacher. He has excellent knowledge of econometrics and takes the time to work through your questions. Highly recommended!”
“Really great tutor! Encouraging, knowledgeable, friendly”
“Very good lesson, Nicky is very helpful.”
“Very helpful and clear.”
“I gather it was very useful, thank you!”
“Fantastic!”
2. Residuals are not errors
Definition: error and residual
Suppose the true relationship is \(y_i = \beta_0 + \beta_1 x_i + u_i\). The error \(u_i\) is the gap between \(y_i\) and the true line.
The residual \(\hat u_i = y_i - \hat\beta_0 - \hat\beta_1 x_i\) is the gap between \(y_i\) and the fitted line.
Remarks. The error and the residual are NOT the same thing. We never observe the true line, so we never observe the errors. The residuals are computed from the data, and they are our estimates of the errors.
The intercept and the slope are chosen to make the sum of squared residuals as small as possible, and at that minimum two sums are zero (the first-order conditions). Hence in a regression with an intercept the residuals add to exactly zero, and their products with x add to exactly zero as well, so there is no straight-line pattern left between the residuals and x. These are properties of the fitting, and they hold in every sample, whatever the data. The errors have neither property: in a sample they have no reason to add to zero.
3. How big is a typical residual?
We want one number for the size of the residuals. Their average will not do, since it is always zero. As with a standard deviation, we square them, average the squares and take the square root.
Start with the five workers of lesson 9, whose fitted line is 9.6 + 2.2x. Their residuals are 0.4, 0.2, -1.0, -0.2 and 0.6 pounds, and the squares add to 0.16 + 0.04 + 1.00 + 0.04 + 0.36 = 1.60. This total is the residual sum of squares, RSS. What we want is an estimate of \(\sigma\), the standard deviation of the errors around the true line. The residuals stand in for the errors, but they are smaller on average: the fitted line was chosen to be as close to these five points as possible, so it sits closer to them than the true line does, and dividing RSS by n would understate the error variance. The five residuals are also not five free numbers. They have to satisfy the two conditions in section 2, so once three of them are known the other two are fixed: two degrees of freedom were used in estimating the intercept and the slope, and n - 2 = 3 are left. If the true relationship is the straight line of section 2, the sample is random and the errors have the same variance at every x, RSS averages \((n - 2)\sigma^2\) across samples, so dividing by n - 2 gives an estimate of the error variance that is right on average. The square root returns it from pounds squared to pounds:
$$ s = \sqrt{\frac{\text{RSS}}{n-2}} = \sqrt{\frac{1.60}{3}} = 0.730 . $$In words: residual standard error = square root of (sum of squared residuals divided by n minus 2).
This is the residual standard error (Stata calls it the root MSE, for root mean squared error). It is in the units of y: a typical worker's pay is about 73 pence away from the line.
For the 20 clubs, the squared residuals in Figure 1 add to 576.25, so
$$ s = \sqrt{\frac{576.25}{18}} = 5.66 \text{ points.} $$A typical club finished between 5 and 6 points away from where its goal difference put it. Twelve of the 20 clubs are within 5.66 points of the line, and all 20 are within twice that.
4. The line against the mean: TSS, ESS and RSS
Is 5.66 points a good fit? It depends on what we compare it with. The natural comparison is the prediction we would make if we did not know x at all: the mean of y for everyone. R-squared asks how much better the line does than the mean.
Go back to the five workers. Their mean pay is £14, and the fitted line is 9.6 + 2.2x. Each worker's distance from the mean splits into two parts:
$$ y_i - \bar y \;=\; (\hat y_i - \bar y) \;+\; (y_i - \hat y_i) . $$In words: distance from the mean = part the line explains + residual.
Eli is paid £19, which is £5 above the mean. The line predicts £18.40 for a worker with his education. So the line accounts for £4.40 of his £5, and £0.60 is left as his residual. Ana is £4 below the mean; the line predicts she will be £4.40 below, and her residual is +£0.40.
Now square each of the three distances for every worker and add:
| Worker | y - 14 | fitted - 14 | residual | (y - 14)2 | (fitted - 14)2 | residual2 |
|---|---|---|---|---|---|---|
| Ana | -4 | -4.4 | 0.4 | 16 | 19.36 | 0.16 |
| Ben | -2 | -2.2 | 0.2 | 4 | 4.84 | 0.04 |
| Cara | -1 | 0.0 | -1.0 | 1 | 0.00 | 1.00 |
| Dev | 2 | 2.2 | -0.2 | 4 | 4.84 | 0.04 |
| Eli | 5 | 4.4 | 0.6 | 25 | 19.36 | 0.36 |
| Sum | 0 | 0.0 | 0.0 | 50 | 48.4 | 1.60 |
- Total sum of squares, TSS = 50. The variation in pay around its mean. It is what predicting £14 for everyone gets wrong.
- Explained sum of squares, ESS = 48.4. The variation in the fitted values.
- Residual sum of squares, RSS = 1.60. What the line leaves over.
The two conditions of section 2 settle how the three totals relate. Because the residuals add to zero and are uncorrelated with x, they are also uncorrelated with the fitted values, which are a straight-line function of x. Squaring the split \(y_i - \bar y = (\hat y_i - \bar y) + (y_i - \hat y_i)\) gives the two squares plus twice their product, and summed over the workers that product term is zero. Hence for every least-squares line with an intercept TSS = ESS + RSS: here 50 = 48.4 + 1.60. R-squared is the explained share of the total:
$$ \begin{aligned} R^2 &= \frac{\text{ESS}}{\text{TSS}} = 1 - \frac{\text{RSS}}{\text{TSS}} \\ &= 1 - \frac{1.60}{50} = 0.968 . \end{aligned} $$The line accounts for 96.8% of the variation in the five workers' pay. R-squared is 0 when the line does no better than the mean (RSS equals TSS) and 1 when every point lies on the line (RSS is zero).
5. R-squared for the 20 clubs
The same three steps on real data.
- TSS. Mean points are 52.3. Square each club's distance from 52.3 (the "y - 52.3" column of lesson 9's table) and add: the total is 8,060.2. Predicting 52.3 points for every club would leave this much squared error.
- ESS. Each fitted value differs from the mean by the slope times that club's x deviation, so ESS is the slope squared times the sum of squared x deviations. Since the slope is the sum of products of deviations divided by the sum of squared x deviations, that is the same as the slope times the sum of products: a shortcut in a simple regression. Lesson 9 found the sum of products to be 12,004, so ESS = 0.623455 × 12,004 = 7,484.0.
- RSS. What is left: 8,060.2 - 7,484.0 = 576.2, which agrees with the sum of the squared residuals in section 3.
- R-squared. 7,484.0/8,060.2 = 0.9285, or equivalently 1 - 576.2/8,060.2.
The table collects the quantities computed in this section and in section 3 for the 20 clubs. Regression software reports them in its own layout.
| Quantity | Value |
|---|---|
| Total sum of squares, TSS | 8,060.2 |
| Explained sum of squares, ESS | 7,483.95 (7,484.0 above, after rounding) |
| Residual sum of squares, RSS | 576.25 |
| R-squared | 0.9285 |
| Residual standard error | 5.658 points |
Choose a program to see the commands and where each number appears.
Stata: the code
import delimited "premier-league-2023-24.csv", clear regress points gd
Stata: what it prints
. regress points gd
Source | SS df MS Number of obs = 20
-------------+---------------------------------- F(1, 18) = 233.77
Model | 7483.95222 1 7483.95222 Prob > F = 0.0000
Residual | 576.247782 18 32.0137657 R-squared = 0.9285
-------------+---------------------------------- Adj R-squared = 0.9245
Total | 8060.2 19 424.221053 Root MSE = 5.6581
------------------------------------------------------------------------------
points | Coefficient Std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
gd | .6234549 .0407763 15.29 0.000 .537787 .7091227
_cons | 52.3 1.265183 41.34 0.000 49.64195 54.95805
------------------------------------------------------------------------------
R: the code
# Premier League 2023/24 final table: points (after deductions) and goal difference
pl <- read.csv("premier-league-2023-24.csv")
fit <- lm(points ~ gd, data = pl)
summary(fit)
confint(fit)
R: what it prints
> # Premier League 2023/24 final table: points (after deductions) and goal difference
> pl <- read.csv("premier-league-2023-24.csv")
> fit <- lm(points ~ gd, data = pl)
> summary(fit)
Call:
lm(formula = points ~ gd, data = pl)
Residuals:
Min 1Q Median 3Q Max
-9.078 -5.285 0.055 4.253 8.428
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 52.30000 1.26518 41.34 < 2e-16 ***
gd 0.62345 0.04078 15.29 9.36e-12 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 5.658 on 18 degrees of freedom
Multiple R-squared: 0.9285, Adjusted R-squared: 0.9245
F-statistic: 233.8 on 1 and 18 DF, p-value: 9.356e-12
> confint(fit)
2.5 % 97.5 %
(Intercept) 49.641949 54.9580511
gd 0.537787 0.7091227
Python: the code
import pandas as pd
import statsmodels.formula.api as smf
# Premier League 2023/24 final table: points (after deductions) and goal difference
pl = pd.read_csv("premier-league-2023-24.csv")
fit = smf.ols("points ~ gd", data=pl).fit()
print(fit.summary())
Python: what it prints
OLS Regression Results
==============================================================================
Dep. Variable: points R-squared: 0.929
Model: OLS Adj. R-squared: 0.925
Method: Least Squares F-statistic: 233.8
Date: Thu, 08 Oct 2026 Prob (F-statistic): 9.36e-12
Time: 14:49:35 Log-Likelihood: -61.987
No. Observations: 20 AIC: 128.0
Df Residuals: 18 BIC: 130.0
Df Model: 1
Covariance Type: nonrobust
==============================================================================
coef std err t P>|t| [0.025 0.975]
------------------------------------------------------------------------------
Intercept 52.3000 1.265 41.338 0.000 49.642 54.958
gd 0.6235 0.041 15.290 0.000 0.538 0.709
==============================================================================
Omnibus: 2.895 Durbin-Watson: 1.352
Prob(Omnibus): 0.235 Jarque-Bera (JB): 1.209
Skew: -0.004 Prob(JB): 0.546
Kurtosis: 1.795 Cond. No. 31.0
==============================================================================
Notes:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.
Stata and R print R-squared as 0.9285, and Python as 0.929. The residual standard error is called Root MSE in Stata and Residual standard error in R; Python's statsmodels does not print it. Stata also lists the three sums of squares, as Model (ESS), Residual (RSS) and Total (TSS). The code reads premier-league-2023-24.csv, which you need to save in your working folder first.
Goal difference accounts for 93% of the variation in points across the 20 clubs. Another way to see the same improvement: if we predicted 52.3 points for every club, the typical miss would be the standard deviation of points, 20.6. With the line, the typical miss is 5.66 points.
The two measures of fit answer different questions. The residual standard error, 5.66, says how far a typical club is from the line, in points. R-squared, 0.93, says what share of the differences between clubs the line accounts for, and has no units. The explainer on R-squared and adjusted R-squared takes R-squared further: what counts as a good value, and what happens when more variables are added.
6. What R-squared does not tell you
R-squared does not say whether the slope can be told apart from zero. Regress birthweight (in ounces) on the number of cigarettes the mother smoked a day, for a sample of 1,388 US births. R-squared is 0.023: cigarettes account for about 2% of the variation in birthweight. Yet the slope, -0.51 ounces per cigarette a day, is estimated precisely (its t-statistic, which lesson 11 explains, is -5.68), and a slope of zero is firmly ruled out. Both statements are true. Babies' weights differ for many reasons, so cigarettes alone account for little of the variation, and with 1,388 births even a modest average difference shows up clearly. Whether a slope can be told apart from zero is the subject of lesson 11; R-squared does not answer it.
7. Why a slope cannot be inverted: y on x and x on y
Regressing y on x and regressing x on y give two different lines through the same cloud of points. Regressing y on x minimises the vertical gaps. Regressing x on y minimises the horizontal gaps, and gives a different line unless the points lie exactly on a straight line.
For 526 US workers, the regression of log wage on years of education has a slope of 0.083. The regression of education on log wage has a slope of 2.2455. If there were only one line, the second slope would be 1 divided by the first. It is not: 1/2.2455 = 0.445, more than five times 0.083.
The slope of y on x is the sum of products of deviations divided by the sum of squared x deviations; the slope of x on y divides the same sum of products by the sum of squared y deviations. Their product is the squared sum of products divided by both sums of squares, which is the squared correlation, and in a simple regression that is R-squared:
| Data | Slope of y on x | Slope of x on y | Product = R-squared |
|---|---|---|---|
| Five workers | 2.2 | 0.44 | 0.968 |
| 20 clubs | 0.62345 | 1.48929 | 0.9285 |
| 526 workers (log wage, education) | 0.0827 | 2.2455 | 0.186 |
When the points lie close to a line, R-squared is near 1 and the two slopes are nearly reciprocals of each other: for the 20 clubs, 1/0.6235 = 1.60 against 1.49. When the cloud is wide, they are far apart. So the slope of one regression cannot be recovered by inverting the other. Which regression we want is settled by the question: the variable we are predicting goes on the left.
8. Common mistakes
- Calling the residuals errors. The residuals are computed from the fitted line. The errors are distances from the true line, which we do not observe.
- Dividing RSS by n. The residual standard error divides by n - 2 in a simple regression, because the intercept and the slope were estimated from the same data.
- Using the wrong ratio. RSS/TSS is the share that is not explained. R-squared is 1 minus that: for the five workers 1.60/50 = 0.032, and R-squared is 0.968.
- Expecting a precisely estimated slope to come with a high R-squared. In the birthweight regression the slope is clearly not zero and R-squared is 0.023.
- Reading R-squared as a share of observations. An R-squared of 0.93 does not mean that 93% of the clubs are on the line. No club is.
- Inverting a slope. The slope of x on y is not 1 divided by the slope of y on x. Run the regression you need.
9. Check your understanding
Five questions, marked on the page; every option has an explanation, including the wrong ones. (Without JavaScript the explanations are shown beneath each question.)
1. The mean of y is 50. One observation has y = 62 and a fitted value of 58. How does its distance from the mean split into an explained part and a residual?
These are the two parts the wrong way round. The explained part is fitted minus mean, 58 - 50 = 8; the residual is actual minus fitted, 62 - 58 = 4.
The residual is actual minus fitted, 62 - 58 = +4, not fitted minus actual. The point lies above the line, so its residual is positive.
Correct. Total distance from the mean: 62 - 50 = 12. The line accounts for 58 - 50 = 8 of it, and the residual is 62 - 58 = 4. The two parts add back to 12.
12 is the total distance from the mean, 62 - 50. The explained part is only as far as the fitted value: 58 - 50 = 8.
2. For the 20 Premier League clubs of 2023/24, regressing points on goal difference gives R-squared = 0.93. Which statement is a correct reading of 0.93?
R-squared measures how closely the points line up with the fitted line. It says nothing about cause, and here points and goal difference both come out of the same 38 matches.
Correct. The squared distances of points from their mean total 8,060. The fitted line accounts for 7,484 of that and leaves 576 in the residuals: 7,484/8,060 = 0.93.
R-squared is a share of squared variation, not a share of clubs. No club lies exactly on the line; the closest, Manchester City, misses by 0.05 of a point.
3. In a simple regression, the residual standard error is the square root of the sum of squared residuals divided by n - 2, not by n. Why n - 2?
Correct. Fitting forces the residuals to satisfy two conditions (they sum to zero, and so do their products with x), so only n - 2 of them are free to vary. Dividing by n would make the scatter look smaller than it is.
No observations are held back; all n are used to fit the line. The 2 counts the parameters estimated, not observations removed.
The residuals add to zero because of how the intercept is chosen, whatever we divide by afterwards. The division only affects the estimate of their typical size.
4. The Premier League regression of points on goal difference has R-squared = 0.93 and a residual standard error of 5.7 points. What does the 5.7 tell you that the 0.93 does not?
Neither number is a share of clubs. The residual standard error is a typical distance from the line, measured in the units of y.
That is the slope, 0.62 points per goal. The residual standard error describes the misses around the line, not the line's steepness.
Correct. R-squared is a unit-free share. The residual standard error is in points: a typical club finished about 5.7 points away from what its goal difference predicted, and 12 of the 20 were within that distance.
5. In the data on 526 workers, regressing education on log wage gives a slope of 2.2455. A student inverts it (1/2.2455 = 0.445) and reports 0.445 as the slope of log wage on education. The regression of log wage on education gives 0.083. Why do the two differ?
Correct. One line minimises vertical gaps and the other horizontal gaps. The product of the two slopes is R-squared, 0.083 × 2.2455 = 0.186, so the inverted slope is too large by the factor 1/0.186 = 5.4. The variable being predicted goes on the left.
Inverting works only when R-squared is 1, that is, when every point lies on the line. Otherwise the inverted slope is too large by the factor 1/R-squared, which is 5.4 here.
The gap is not noise. It is fixed by the correlation between the two variables: with a million workers and the same correlation, the two lines would be as far apart.
10. Three questions about the arithmetic
Find the fitted values and the residuals. Add the squared residuals to get RSS, and add the squared deviations of y from its mean to get TSS. Then R-squared = 1 - RSS/TSS. For the five workers RSS is 1.60 and TSS is 50, so R-squared is 1 - 0.032 = 0.968. With one regressor it equals the squared correlation.
For the residual standard error in a regression with an intercept and one slope, divide by n - 2, because two parameters were estimated. With k slopes, divide by n - k - 1. For R-squared no division is needed: it is a ratio of two sums of squares, and any common divisor would cancel.
Yes. In a simple regression ESS equals the slope times the sum of products of deviations. For the five workers that is 2.2 × 22 = 48.4, and for the 20 clubs 0.62345 × 12,004 = 7,484. Then RSS = TSS - ESS, which saves working out each residual and squaring it.
11. Where next
- Lesson 11: reading regression output, which uses the residual standard error from this lesson to build the standard error of the slope, the t-statistic and the confidence interval.
- R-squared and adjusted R-squared, the explainer: what counts as a good R-squared, four data sets with the same R-squared, adjusted R-squared, and five more marked questions.
- What is OLS?, the explainer, for why least squares minimises vertical gaps and what the method assumes.
- Lesson 9: fitting the OLS line by hand (previous) and the course home, which lists all 25 lessons and says which are published.
Send me your regression: module, output and deadline
I tutor econometrics one-to-one, from a first regression through to endogeneity, panel data and time series. Bring the question you are stuck on and we will work through it in the free 30-minute consultation.
Book a free 30-minute consultation Econometrics tutor: how it works
Data and figures. Five-worker example: illustrative data. Premier League: match results from openfootball/england (CC0), 2023/24 season, with the deductions for Everton (8 points) and Nottingham Forest (4 points) applied. Birthweight and pay: Wooldridge, Introductory Econometrics: A Modern Approach, data sets BWGHT (1,388 births, 1988 National Health Interview Survey) and WAGE1 (526 workers, 1976), via the wooldridge Python package; results quoted, data not republished. Every sum of squares was computed by two independent routines, which agree. Figures drawn for this lesson.