Heteroskedasticity & clustering

Heteroskedasticity and clustering are the two most common reasons a textbook regression output is quietly wrong. Neither biases the coefficients; both corrupt the standard errors, and therefore every t-statistic, confidence interval and F-test built on them.

What heteroskedasticity does to OLS

Under the zero-conditional-mean assumption, OLS is unbiased and consistent even with heteroskedasticity — the point estimate \(\hat\beta\) is fine. What breaks is inference:

• the usual standard errors are biased;
• so the conventional \(t\), \(F\) and LM statistics no longer have their stated distributions;
• and the residual sum of squares can no longer be used to build valid \(F\) statistics.

The fix is not a new estimator of \(\beta\); it is a corrected estimator of its variance.

The variance of OLS under heteroskedasticity

For simple regression \(y=\beta_0+\beta_1 x+u\), the sampling error is \(\hat\beta_1=\beta_1+\dfrac{\sum(x_i-\bar x)u_i}{\sum(x_i-\bar x)^2}\), so, allowing each observation its own variance \(\sigma_i^2\),

If \(\sigma_i^2=\sigma^2\) this collapses to the familiar \(\sigma^2/\text{SST}_x\). When it does not, a valid estimator replaces the unknown \(\sigma_i^2\) with the squared OLS residual \(\hat u_i^2\) — the White (Eicker–Huber) heteroskedasticity-consistent estimator:

where in multiple regression \(\hat r_{ij}\) is the residual from regressing \(x_j\) on all the other regressors and \(\text{SSR}_j\) the corresponding residual sum of squares. Its validity rests on the law of large numbers and the central limit theorem, so it is an asymptotic justification. In matrix form this is the standard "sandwich" estimator, robust covariance = \((X'X)^{-1}\big(\sum_i \hat u_i^2\, x_i' x_i\big)(X'X)^{-1}\); the "bread" \((X'X)^{-1}\) surrounds a "meat" built from the squared residuals, and it collapses to \(\hat\sigma^2 (X'X)^{-1}\) under homoskedasticity.

Robust standard errors in practice

The square roots of these variances are the robust standard errors (attributable to White, Huber and Eicker). Caveats: they are only asymptotically justified, so in small samples t-statistics formed with them need not be close to the \(t\) distribution; and joint tests must be done on the robust covariance matrix rather than by comparing residual sums of squares.

. regress lwage educ
        educ |   .0827444   .0075667    10.94   0.000
       _cons |   .5837727   .0973358     6.00   0.000

. regress lwage educ, robust
                          Robust
        educ |   .0827444   .0077389    10.69   0.000
       _cons |   .5837727   .0982339     5.94   0.000

Testing for heteroskedasticity

The null is constant conditional variance, equivalently \(\mathbb{E}(u^2\mid x_1,\dots,x_k)=\sigma^2\). Assuming a linear relationship between \(u^2\) and the regressors, run the auxiliary regression on squared residuals

and test \(H_0:\delta_1=\dots=\delta_k=0\) (the Breusch–Pagan / White approach). Rejection is evidence of heteroskedasticity.

The two-group case: robust versus conventional

With a single dummy \(d_i\), \(\hat\beta_1=\bar y_1-\bar y_2\) is just the difference in group means. Writing \(p=n_1/n\), the conventional (pooled-variance) and robust variances are

They coincide when \(s_1=s_2\) — and, less obviously, when the data are balanced (\(n_1=n_2\)). So with balanced groups the robust correction barely moves even under genuine heteroskedasticity; the gap opens up when groups are unbalanced and their variances differ. In a log-pay regression on a manual-occupation dummy (with manual workers only about a quarter of the sample), the robust standard error was 0.0168 against a conventional 0.0191 — a 13% difference, and the equal-variances test rejected strongly (\(t\approx-6.66\)).

Clustering: when independence fails

Correct inference assumes the errors are independent across observations. That fails whenever data have a group structure: pupils in the same school, workers in the same firm, individuals in the same region or the same person observed over several years. Their errors share a common component and are therefore correlated. Ignoring this — the Moulton problem — usually biases the variance estimate downwards, so standard errors are too small and results look more significant than they are.

The variance-components structure

Consider a regressor \(x_g\) that varies only at the group level, \(y_{ig}=\beta_0+\beta_1 x_g+e_{ig}\), with the error split into a group component and an individual component, \(e_{ig}=v_g+\eta_{ig}\). For two individuals \(i\neq j\) in the same group,

so the intraclass (intra-cluster) correlation is

the share of total error variance coming from the common component \(v_g\).

The Moulton factor

With a group-level regressor and equal group sizes \(n\), the true variance of \(\hat\beta_1\) relative to the conventional one is

If the regressor also varies within groups (with within-cluster correlation \(\rho_x\)), the factor becomes \(1+(n-1)\rho_e\rho_x\); with unequal group sizes it generalises to \(1+\big[\operatorname{Var}(n_g)/\bar n+\bar n-1\big]\rho_e\rho_x\). If the regressor is uncorrelated within groups, the problem disappears.

What to do about it

The key requirement for every fix is that the errors are independent across clusters, so the number of clusters \(G\) must be reasonably large. Three routes, all with model \(y_{ig}=\beta_0+\beta_1 x_{ig}+v_g+\eta_{ig}\):

1. Moulton correction — estimate \(\rho_e\) (and \(\rho_x\)) and scale the conventional standard errors by the factor above.
2. Cluster-robust standard errors — a sandwich estimator that allows arbitrary correlation within each cluster; valid only when \(G\) is large.
3. Group averages / WLS — collapse to group means \(\bar y_g=\beta_0+\beta_1 x_g+\bar e_g\) and run weighted least squares with weights \(n_g\); this works even when \(G\) is small, but discards individual-level covariates and needs \(x\) to vary only at group level.

Related notes

• Lecture 1: Treatment effects · Lecture 3: Panel data & policy · Lecture 4: Advanced panel methods.
• Diagnostic testing & robust inference (slides) — the complementary postgraduate lecture on White, Breusch–Godfrey and HAC/Newey–West.
• Econometrics explained · ECON60052 hub.

Download