ECON60052 · Lecture 3
Policy analysis with panel data
How economists evaluate policies using data with a time dimension: pooling cross-sections, reading interaction terms, and the workhorse of modern policy evaluation — difference-in-differences. It ends with the two-period panel and first differencing, which remove unobserved individual heterogeneity.
This lecture is about credible policy evaluation without a randomised experiment. The central idea — difference-in-differences — compares the change in outcomes for a treated group with the change for a control group, netting out anything common to both.
Two kinds of data with a time dimension
Pooled cross sections take different random samples at different dates. Panel (longitudinal) data follow the same units over time. A panel is balanced when every unit appears in every period. We work mostly with short, wide panels: many individuals, few periods (two is enough to make the point).
Pooling cross sections across time buys larger samples, lets us study the effect of time, test whether relationships have changed, and evaluate policy. Stacking two periods and running one regression is pooled OLS; the key question is whether the parameters — including the intercept — are stable across time.
Time dummies and interaction terms
Let \(d2\) be a dummy for the second period and \(m\) a group indicator (say male). The pooled model with a full interaction is
Reading it through the four conditional means:
- \(\mathbb{E}(y\mid m{=}0,d2{=}0)=\alpha_0\) — base group, period 1;
- \(\beta_2\) is the time change for the base group;
- \(\beta_1\) is the group differential in period 1;
- \(\delta\) is a difference in differentials: how the group gap itself changed over time.
In a pooled log-wage regression over two household-survey waves ten years apart, the interaction of the male dummy with the later-year dummy was about \(-0.093\) and significant (\(t\approx-4.45\)): the male log-wage differential fell by roughly 0.093 log points over the decade. The base group's own wages rose by about 0.306 log points (the later-year dummy). Interactions are how a pooled regression detects such structural change.
Difference-in-differences
Ideally we would randomly assign units to treatment and control; a natural experiment — a policy that hits one group but not another — mimics that. Without random assignment we compare changes, so that anything fixed about each group cancels. The difference-in-differences (DiD) estimator is
Read it either as the treated–control gap in period 2 minus the same gap in period 1, or as the treated group's before-to-after change minus the control group's before-to-after change; the two are algebraically identical.
DiD by regression
With a time dummy \(d2\) and a treated dummy \(dT\),
Here \(\beta_0=\bar y_{1,C}\), the \(d2\) coefficient is the control group's time change, and \(\delta_1\) is the DiD: \(\delta_1=(\bar y_{2,T}-\bar y_{1,T})-(\bar y_{2,C}-\bar y_{1,C})\). The regression form is preferred to raw means because it lets you add covariates to control for observable differences and delivers a standard error on \(\hat\delta_1\).
A natural experiment: the minimum wage
New Jersey raised its state minimum wage in April 1992 from \$4.25 to \$5.05, while neighbouring Pennsylvania held at \$4.25. Card and Krueger surveyed fast-food employment in both states before (February 1992) and after (November 1992). Model the outcome as
with a state effect \(\gamma_s\), a common time effect \(\lambda_t\) and the treatment effect \(\beta\). Then three candidate estimators behave very differently:
- New Jersey after minus before \(=\lambda_{Nov}-\lambda_{Feb}+\beta\) — contaminated by the time trend;
- treated minus untreated \(=\lambda_{Nov}-\lambda_{Feb}+\beta-\mathbb{E}[\gamma_s]\) — contaminated by trend and state effect;
- the DiD \(=\beta\) — clean, provided \(\lambda_t\) is common across states.
Only the difference-in-differences recovers \(\beta\), and only under the common-trends assumption. In regression form \(y_{ist}=\alpha+\gamma\,NJ_s+\lambda\,d_t+\beta\,(NJ_s\times d_t)+u_{ist}\), the interaction coefficient is the difference-in-difference.
Common (parallel) trends. In the absence of treatment, the outcome would have followed the same trend in the treatment and control groups. This is the identifying assumption for DiD; it can be partly checked with additional pre-treatment periods. Self-selection. If units choose treatment on characteristics linked to the outcome, and those characteristics are unobserved, DiD is biased.
The UK minimum wage was introduced in April 1999. Using labour-force data, take the outcome to be the probability of still being employed a year later, given employed initially. Group 1 (treated) were paid at or below the new floor; group 2 (control) were paid just above it.
| Before | After | Change | |
|---|---|---|---|
| Group 1 (treated) | 0.8872 | 0.9372 | +0.050 |
| Group 2 (control) | 0.9378 | 0.9097 | −0.028 |
A DiD regression returns the interaction coefficient 0.0781 (se 0.0346, \(t=2.26\)), just significant; adding controls leaves it at 0.070. The (illustrative) conclusion is a positive employment effect of the minimum-wage introduction — the constant recovers \(\bar y_{1,C}=0.9378\) and the other coefficients recover the remaining cell contrasts exactly.
Two-period panels and unobserved heterogeneity
Now follow the same individuals twice. The motivation is unobserved heterogeneity: fixed traits such as ability or motivation that we cannot measure but that drive both regressors (schooling, union status) and outcomes, creating omitted-variable bias. Split the error into a time-constant part and a time-varying part:
where \(a_i\) is the unobserved / fixed effect and \(u_{it}\) the idiosyncratic error. Pooled OLS treats \(a_i+u_{it}\) as one composite error and is biased and inconsistent whenever \(a_i\) is correlated with \(\mathbf{x}_{it}\) — this is heterogeneity bias.
First differencing removes the fixed effect
Write the two periods and subtract:
Because \(a_i\) is identical in both periods it cancels — so any correlation between \(a_i\) and the regressors can no longer bias the slopes. The common intercept \(\beta_0\) also cancels; \(\delta_0\), the coefficient on the differenced time dummy (a column of ones), survives as the new constant. OLS on this differenced equation is the first-difference (FD) estimator, unbiased if the differenced error is uncorrelated with the differenced regressors — a strict-exogeneity condition, \(\mathbb{E}(\mathbf{x}_{it}u_{is})=0\) for all \(t,s\).
Panel difference-in-differences
With a treatment dummy \(T_{it}\) that switches on only in period 2, differencing gives \(\Delta y_i=\gamma_0+\delta_1 T_{i2}+\Delta u_i\), so \(\hat\delta_1=\overline{\Delta y}_T-\overline{\Delta y}_C\): the panel version of DiD, now with the added advantage that it controls for the unobserved heterogeneity \(a_i\). The price of first differencing: time-constant regressors drop out, variables that change by the same amount for everyone (like age with time dummies) drop out, informative variation can be small (large standard errors), and measurement error is aggravated.
Self-check questions
A researcher finds a significant negative interaction δ in the pooled model. What does it mean, and how is δ built from the cell means?
\(\delta\) is the difference in differentials: \(\delta=[\mathbb{E}(y\mid m{=}1,d2{=}1)-\mathbb{E}(y\mid m{=}1,d2{=}0)]-[\mathbb{E}(y\mid m{=}0,d2{=}1)-\mathbb{E}(y\mid m{=}0,d2{=}0)]\) — the change in the group gap between periods. A significant \(\hat\delta \lt 0\) means the gap shrank. In the wage example the male–later-year interaction was \(-0.093\) (\(t\approx-4.45\)): the male log-wage differential fell by about 0.093 log points over the decade, and the fall is statistically significant.
Show algebraically why first differencing removes aᵢ, and why β₀ vanishes while δ₀ survives.
With \(y_{i1}=\beta_0+\mathbf{x}_{i1}\boldsymbol\beta+a_i+u_{i1}\) and \(y_{i2}=\beta_0+\delta_0+\mathbf{x}_{i2}\boldsymbol\beta+a_i+u_{i2}\), subtracting gives \(\Delta y_i=\delta_0+\Delta\mathbf{x}_i\boldsymbol\beta+\Delta u_i\). The fixed effect is identical across periods, so \(a_i-a_i=0\): it cancels, and its correlation with the regressors can no longer bias the slopes. The common intercept cancels too (\(\beta_0-\beta_0=0\)). What remains, \(\delta_0\), multiplies the differenced time dummy \(d2_2-d2_1=1\) for every unit, so it becomes the new regression constant — the one macro effect identified from two periods.
Why is New Jersey's simple before/after change a biased estimate of β, and why does DiD fix it? When does DiD still fail?
Under \(y_{ist}=\alpha+\gamma_s+\lambda_t+\beta D_{st}+u_{ist}\), New Jersey's before-to-after change equals \(\lambda_{Nov}-\lambda_{Feb}+\beta\): it mixes the treatment effect with the common time trend. Subtracting Pennsylvania's before-to-after change, which equals \(\lambda_{Nov}-\lambda_{Feb}\) (no treatment there), removes the trend and leaves exactly \(\beta\). This works only under common trends — \(\lambda_t\) identical in both states. If the states were on different trends regardless of the policy, or if unobservables correlated with treatment status differ across groups, DiD is biased.
Related notes
- Lecture 4: Advanced panel methods — fixed effects, random effects and the Hausman test for \(T\ge2\).
- Lecture 1: Treatment effects — the potential-outcomes logic behind DiD.
- ECON60052 hub · Econometrics explained.
Download
Studying this for a module?
I tutor economics, econometrics and maths at university and postgraduate level — from weekly problem sets and past exams to dissertation methods. Book a free 30–45 minute consultation to talk through exactly where you are.