Stationarity, white noise
& MA processes

A time series process and its realisation

A time series process \(Y_t\) (for \(t = \ldots,-2,-1,0,1,2,\ldots\)) is the unknown data-generating mechanism behind an economic series. Each \(Y_t\) is a random variable whose distribution we do not know. What we actually observe is a single realisation, the sample \(\{y_1, y_2, \ldots, y_T\}\), shorthand \(\{y_t\}_{t=1}^{T}\), of size \(T\). Monthly CPI from January 2000 to a fixed end date, for example, gives one path of a few hundred numbers.

The central difficulty is that we must infer the properties of the process \(Y_t\) — its mean, its variance, how strongly today depends on yesterday — from that one observed run of history. In ordinary cross-section statistics we have many independent draws; in time series we have exactly one. Stationarity is the assumption that rescues us: it says the process looks the same wherever in time we stand, so the single realisation behaves like repeated draws from one fixed distribution.

Weak (covariance) stationarity

In words: the mean is constant, the variance is finite and constant, and the covariance between two observations depends only on the gap \(j\) between them, never on where in time they sit. This is what econometricians usually mean by "stationary". Its full name is weak stationarity, also called covariance stationarity or second-order stationarity, because it restricts only the first two moments. (A stronger notion, strict stationarity, asks the entire joint distribution to be time-invariant; weak stationarity is what the standard results below require.)

Stationarity is powerful precisely because it turns one realisation into, in effect, \(T\) draws from the same distribution — the raw material for estimation. It can fail in economically important ways: the variance may drift over time (volatility clustering in stock returns), the mean may shift (a series that steps up permanently), or the process may contain a unit root such as a random walk, whose variance grows without bound.

White noise

White noise is a zero-mean, constant-variance process with no correlation across time. It is the elementary building block \(\varepsilon_t\) from which the AR, MA and ARMA models are constructed. Two facts are worth committing to memory:

• White noise implies stationarity, but not the reverse. Every white-noise process satisfies the three stationarity conditions (with \(\mu=0\) and \(\gamma(j)=0\) for \(j\neq0\)). But a stationary process may have non-zero autocovariances — it only needs them to be constant over time. So stationarity and white noise are not the same thing.
• If \(Y_t\) is i.i.d. it is also white noise; the converse need not hold, because white noise restricts only correlation, not full independence.

The autocovariance and autocorrelation function

For a stationary process the autocovariance function is \(\gamma(k):=\mathrm{Cov}(Y_t,Y_{t-k})\) for \(k=0,1,2,\ldots\), and the autocorrelation function (ACF) is

$$\rho(k):=\frac{\mathrm{Cov}(Y_t,Y_{t-k})}{\sqrt{\mathrm{Var}(Y_t)\,\mathrm{Var}(Y_{t-k})}}=\frac{\gamma(k)}{\gamma(0)}, \qquad k=0,1,2,\ldots$$

Under stationarity \(\mathrm{Var}(Y_t)=\mathrm{Var}(Y_{t-k})=\gamma(0)\), so the ACF is just the autocovariance scaled by the variance. Note \(\gamma(0)=\mathrm{Var}(Y_t)\) and hence \(\rho(0)=1\); for white noise \(\rho(k)=0\) for every \(k\ge1\). The ACF is the fingerprint of a process: the shape of \(\rho(k)\) across lags tells us which model generated the data, and the sample version is the basis of the correlogram used for model identification.

The MA(1) process

A first-order moving-average process is

$$Y_t=\alpha+\varepsilon_t+\theta_1\varepsilon_{t-1}, \qquad \varepsilon_t\sim\mathrm{WN}(\sigma^2).$$

It is a weighted sum of the current and one lagged shock. Its moments follow directly from the white-noise properties of \(\varepsilon_t\).

Mean. Taking expectations and using \(\mathbb{E}[\varepsilon_t]=0\),

$$\mathbb{E}[Y_t]=\alpha+\mathbb{E}[\varepsilon_t]+\theta_1\mathbb{E}[\varepsilon_{t-1}]=\alpha.$$

Variance. Because the shocks are uncorrelated the cross term vanishes,

$$\mathrm{Var}[Y_t]=\mathbb{E}[(\varepsilon_t+\theta_1\varepsilon_{t-1})^2]=\mathbb{E}[\varepsilon_t^2]+\theta_1^2\mathbb{E}[\varepsilon_{t-1}^2]+2\theta_1\mathbb{E}[\varepsilon_t\varepsilon_{t-1}]=\sigma^2(1+\theta_1^2).$$

Autocovariance. At lag 1, only the term in \(\varepsilon_{t-1}^2\) survives,

$$\mathrm{Cov}[Y_t,Y_{t-1}]=\mathbb{E}[(\varepsilon_t+\theta_1\varepsilon_{t-1})(\varepsilon_{t-1}+\theta_1\varepsilon_{t-2})]=\theta_1\mathbb{E}[\varepsilon_{t-1}^2]=\theta_1\sigma^2,$$

while at every lag \(j\ge2\) the two windows share no common shock, so \(\mathrm{Cov}[Y_t,Y_{t-j}]=0\). The ACF is therefore

$$\rho(k)=\begin{cases}\dfrac{\theta_1}{1+\theta_1^2}, & k=1,\\[4pt] 0, & k>1.\end{cases}$$

An MA(1) is correlated only with its immediate neighbour: its memory is exactly one period long.

The general MA(q) process

Extending to \(q\) lagged shocks (with the normalisation \(\theta_0=1\)):

$$Y_t=\alpha+\varepsilon_t+\theta_1\varepsilon_{t-1}+\cdots+\theta_q\varepsilon_{t-q}=\alpha+\sum_{j=0}^{q}\theta_j\varepsilon_{t-j}, \qquad \varepsilon_t\sim\mathrm{WN}(\sigma^2).$$

Mean: \(\mathbb{E}[Y_t]=\alpha\). Variance: the variance of a sum of uncorrelated shocks is the sum of the variances,

$$\mathrm{Var}[Y_t]=\sigma^2\left(1+\theta_1^2+\cdots+\theta_q^2\right)=\sigma^2\sum_{j=0}^{q}\theta_j^2.$$

Autocovariance function. Multiplying the windows for \(Y_t\) and \(Y_{t-k}\), only products of shocks with equal time subscripts have non-zero expectation \(\sigma^2\). Collecting them gives

$$\gamma(k)=\begin{cases}\sigma^2\sum_{s=0}^{q-k}\theta_s\theta_{s+k}, & k=0,1,\ldots,q,\\[4pt] 0, & k>q,\end{cases}$$

and dividing by \(\gamma(0)=\sigma^2\sum_{s=0}^q\theta_s^2\) yields the ACF

$$\rho(k)=\begin{cases}\dfrac{\sum_{s=0}^{q-k}\theta_s\theta_{s+k}}{\sum_{s=0}^{q}\theta_s^2}, & k=0,1,\ldots,q,\\[4pt] 0, & k>q.\end{cases}$$

Check your understanding

Show that the MA(1) process \(Y_t=\alpha+\varepsilon_t+\theta_1\varepsilon_{t-1}\) has variance \(\sigma^2(1+\theta_1^2)\) and state its lag-1 autocorrelation.

Since \(\mathbb{E}[Y_t]=\alpha\), the variance is \(\mathbb{E}[(Y_t-\alpha)^2]=\mathbb{E}[(\varepsilon_t+\theta_1\varepsilon_{t-1})^2]\). Expanding, \(=\mathbb{E}[\varepsilon_t^2]+\theta_1^2\mathbb{E}[\varepsilon_{t-1}^2]+2\theta_1\mathbb{E}[\varepsilon_t\varepsilon_{t-1}]\). By white noise \(\mathbb{E}[\varepsilon_t^2]=\mathbb{E}[\varepsilon_{t-1}^2]=\sigma^2\) and \(\mathbb{E}[\varepsilon_t\varepsilon_{t-1}]=0\), so \(\mathrm{Var}[Y_t]=\sigma^2(1+\theta_1^2)\). With \(\gamma(1)=\theta_1\sigma^2\), the lag-1 autocorrelation is \(\rho(1)=\gamma(1)/\gamma(0)=\theta_1/(1+\theta_1^2)\), which peaks at \(\pm0.5\) when \(\theta_1=\pm1\).

Explain why "stationary" and "white noise" are not the same thing.

White noise is a special stationary process with zero mean and zero autocovariance at every non-zero lag. Stationarity only requires the autocovariance \(\gamma(j)\) to be constant over time, not zero — an AR(1) or MA(1) is stationary yet has non-zero correlation across periods. So white noise implies stationarity, but a stationary process is generally correlated across time and hence need not be white noise.

For an MA(2), \(Y_t=\alpha+\varepsilon_t+\theta_1\varepsilon_{t-1}+\theta_2\varepsilon_{t-2}\), find the lag-1 and lag-2 autocovariances.

Using \(\gamma(k)=\sigma^2\sum_{s=0}^{2-k}\theta_s\theta_{s+k}\) with \(\theta_0=1\): at \(k=1\), \(\gamma(1)=\sigma^2(\theta_0\theta_1+\theta_1\theta_2)=\sigma^2(\theta_1+\theta_1\theta_2)\); at \(k=2\), \(\gamma(2)=\sigma^2\theta_0\theta_2=\sigma^2\theta_2\); and \(\gamma(k)=0\) for \(k>2\). Dividing by \(\gamma(0)=\sigma^2(1+\theta_1^2+\theta_2^2)\) gives the ACF, which cuts off after lag 2.

Related notes

Downloads