Diagnostic testing in the linear regression model

1. Which assumptions are at stake

The large-sample analysis of OLS rests on a set of assumptions. Two of them concern the structure of the error variance:

The key fact to fix in your mind is that violations of LS3 and LS4 do not bias OLS. Unbiasedness comes from linearity (LS1) and exogeneity, \(E[u\,|\,X]=0\) (LS2). Since \(\hat\beta=\beta_0+(X'X)^{-1}X'u\) and the exogeneity assumption makes the second term mean-zero, OLS is unbiased regardless of the error variance structure. What LS3 and LS4 affect is the variance of OLS, and therefore the validity of standard errors, t-tests and confidence intervals.

2. Heteroskedasticity: the variance is not constant

Heteroskedasticity means the conditional variance of the error changes across observations: \(E[u_t^2\,|\,X]=\sigma_t^2(x)\). A simple example in the notes is \(u_t=v_t x_{1t}\) with \(E[v_t^2\,|\,X]=\sigma^2\), which gives \(E[u_t^2\,|\,X]=\sigma^2 x_{1t}^2\) — the error variance grows with a regressor, producing a tell-tale U-shaped pattern when residuals are plotted. A classic empirical case is a regression of credit-card spending on income, where the spread of the residuals widens as income rises.

The damage shows up in the asymptotic variance of OLS, which in general is the "sandwich" form

where \(Q=E[x_tx_t']\) and \(\Omega=\lim \mathrm{Var}(T^{-1/2}\sum_t x_t u_t)\). Under homoskedasticity \(\Omega=\sigma_0^2 Q\), so the sandwich collapses to the familiar \(\sigma_0^2 Q^{-1}\) and the usual OLS standard errors are correct. Under heteroskedasticity \(\Omega e\sigma_0^2 Q\), the usual formula is biased, and the t-statistic is no longer asymptotically standard normal.

3. Testing for heteroskedasticity: the White test

To decide whether robust standard errors are needed, we test the null of homoskedasticity directly. The White test works as follows:

1. Run the original regression by OLS and obtain the residuals \(\hat u_t\).
2. Regress the squared residuals \(\hat u_t^2\) on the regressors, their squares and their cross-products. For example, with \(x_t=(1,x_{1t})'\), regress \(\hat u_t^2\) on \(x_{1t}\) and \(x_{1t}^2\).
3. Under the null of homoskedasticity, \(T R^2\) from this auxiliary regression has a \(\chi^2_q\) distribution, where \(q\) is the number of slope coefficients in the auxiliary regression.
4. Reject homoskedasticity if \(T R^2\) exceeds the critical value.

The intuition is that if the error variance is genuinely constant, nothing about the regressors should help predict the squared residuals, so the auxiliary \(R^2\) should be near zero. A large \(R^2\) indicates the variance depends on the regressors.

4. Weighted least squares: regaining efficiency

Robust standard errors fix inference but leave OLS inefficient — under heteroskedasticity OLS is no longer the best linear unbiased estimator. If we actually know the form of the heteroskedasticity, we can do better with weighted least squares (WLS).

Suppose we know \(\sigma_t(x)\), for example \(\sigma_t=\sigma x_{1t}\) (quadratic heteroskedasticity). Dividing the whole regression through by \(x_{1t}\),

produces a transformed model whose error \(u_t/x_{1t}\) has constant variance \(\sigma^2\). Running OLS on the transformed variables is WLS, and it is efficient and unbiased. The idea is that WLS gives more weight to the low-variance observations, which carry more reliable information, instead of weighting every point equally as OLS does. When the variance is known only up to parameters, such as \(\sigma_t^2(x)=\exp(\alpha_0'x_t)\), those parameters can be estimated (often by maximum likelihood) and used to construct feasible weights.

5. Serial correlation: errors that move together over time

In time-series regressions the no-serial-correlation assumption LS4 often fails: the error in one period is correlated with the error in the next, so the variance matrix \(E[uu'\,|\,X]\) has non-zero off-diagonal terms. Like heteroskedasticity, serial correlation does not bias OLS but it does bias the usual standard errors, so t-tests are invalid without a robust correction.

A natural idea would be to estimate the off-diagonal terms with products of residuals, but this fails: the cross-product \(\sum_s\sum_t \hat u_s \hat u_t x_s x_t'\) collapses because \(X'\hat u=0\) by construction of OLS. The solution is the Newey-West estimator, which adds weighted residual cross-products at increasing lags:

6. Testing for serial correlation: the LM test

To test whether the error is serially correlated, we use a Lagrange Multiplier (LM) test, mirroring the structure of the White test:

1. Run the original regression by OLS and obtain the residuals \(\hat u_t\).
2. Run the auxiliary regression of \(\hat u_t\) on the original regressors and lagged residuals \(\hat u_{t-1},\dots,\hat u_{t-q}\).
3. Under the null of no serial correlation, \(T R^2\) from this regression has a \(\chi^2_q\) distribution, with \(q\) equal to the number of lagged residual terms.

Including the original regressors in the auxiliary regression matters: it ensures the test is valid even if the regressors are correlated with the lagged errors.

7. Which standard errors should you use?

The choice is a precision-versus-robustness trade-off. The most robust option is not always the best, because robustness costs precision.

The practical rule that comes out of the tutorials: do not reach for Newey-West standard errors when there is no evidence of serial correlation. If the LM test finds no serial correlation, White standard errors are preferable because they are more precise. Always test before you choose.