Treatment effects & causal inference

Most applied economics asks a causal question — the effect of schooling on wages, of a training scheme on employment, of a policy on behaviour. This note develops the framework econometricians use to answer such questions honestly, and shows where naive comparisons go wrong.

Why causal questions are hard

Econometrics studies methods to identify the causal impact of a change in \(x_i\) on an outcome \(y_i\). In the ordinary regression framework this requires the other drivers of \(y_i\) to be independent of \(x_i\) — the exogeneity condition. When we evaluate a policy or treatment, that condition is usually the whole battle: the people who receive a treatment typically differ systematically from those who do not, and those differences also affect the outcome.

The hospital example

Do hospitals, on average, make people healthier? Surely yes. Yet consider survey data that records (i) whether a respondent was an overnight hospital patient in the last year, and (ii) their self-reported health on a 1–5 scale (1 = poor, 5 = excellent).

The potential-outcomes framework

Let \(y\) be the outcome and \(d\) a treatment dummy. Each person has two potential outcomes:

The causal effect of treatment for person \(i\) is \(y_{1i}-y_{0i}\). The difficulty — the fundamental problem of causal inference — is that we only ever observe one of the two. The observed outcome is

Average treatment effects

Because individual effects are unobservable, we target averages:

The ATE is the effect of treating a randomly chosen member of the population; the ATT (average treatment effect on the treated) is the effect for those who actually took the treatment. It is tempting to estimate these with the observed difference \(\mathbb{E}[y_i\mid d_i=1]-\mathbb{E}[y_i\mid d_i=0]\) — but that ignores selection into treatment.

The selection-bias decomposition

Add and subtract the treated group's no-treatment counterfactual \(\mathbb{E}[y_{0i}\mid d_i=1]\):

The ATT is the genuine causal effect (positive for hospitals). The selection bias compares the no-treatment outcome of those who did versus did not select treatment. Because the sick are more likely to seek care, \(\mathbb{E}[y_{0i}\mid d_i=1] \lt \mathbb{E}[y_{0i}\mid d_i=0]\): the bias is negative and large enough to swamp a modest positive ATT, giving the perverse \(-0.72\).

Randomisation removes selection

Suppose treatment were assigned at random. Then the potential outcomes are independent of \(d\), so \(\mathbb{E}[y_{0i}\mid d_i=1]=\mathbb{E}[y_{0i}\mid d_i=0]\) and the selection-bias term vanishes:

Under random assignment the treated are a random sample, so ATT = ATE, and the simple difference in means is exactly the causal effect. Genuine randomisation is rare in economics, so most of the subject is about recovering the ATT or ATE under weaker, credible assumptions.

From experiments to regression

A constant-effect causal model writes \(y_i=\alpha+\rho\, d_i+v_i\), where \(v_i\) collects everything else and \(\rho=y_{1i}-y_{0i}\) is the (assumed constant) effect. Taking conditional expectations,

The bias is precisely the correlation between the regressor \(d_i\) and the error \(v_i\). If we can control for the observed characteristics that drive selection, we can hope to eliminate it.

Selection on observables: unconfoundedness and overlap

Two assumptions make treatment "as good as random" once we condition on covariates \(X_i\).

Matching estimators

Under unconfoundedness the within-cell difference identifies the effect at that covariate value. Define

Averaging \(\delta_X\) over the covariate distribution — weighting by the treated distribution for the ATT, or the whole-sample distribution for the ATE — recovers the parameter of interest by the law of iterated expectations. With a single binary covariate (say a gender dummy taking values \(1\) and \(0\)) there are two cell differences \(\delta_1\) and \(\delta_0\), each estimated by a difference of subgroup means, and

The only difference between the two estimators is the weights: the ATT uses the covariate shares among the treated, the ATE uses population shares. With a vector of discrete covariates we form \(M\) cells (e.g. gender × ethnicity × region gives \(M=40\); adding five-year age bands gives 400), and exact matching generalises to

As the number of cells grows this becomes unwieldy and overlap starts to fail: whenever a cell contains only treated or only untreated units, no comparison is possible, which motivates approximate matching and the propensity score in later work.

Matching versus regression

Both matching and regression lean on unconfoundedness and overlap; regression additionally imposes functional form and (in the simplest case) a constant effect. Angrist and Pischke's slogan is that "regression can be seen as a sort of matching estimator": both combine the cell-level differences \(\delta_x\) into a single number, but with different weights — regression weights by the conditional variance of treatment in each cell, matching by cell frequencies. They therefore give different numbers in general.

Application: military service and later earnings (Angrist, 1998)

Did voluntary US military service raise later wages? Because the military screened applicants on age, schooling and test scores, veterans and non-veterans differ in observables. Matching and regression on those covariates control for the differences. The finding is a clean illustration of selection bias: white veterans earned about \$1,233 more than white non-veterans in a naive comparison, but the effect turns negative once differences in covariates are matched away. Applicant screening had generated positive selection bias.

Related notes

• Lecture 2: Heteroskedasticity & clustering — getting standard errors right once you have a specification.
• Lecture 3: Policy analysis with panel data — difference-in-differences, the natural-experiment version of this framework.
• ECON60052 hub · Econometrics explained (endogeneity, IV, GMM).