AR, ARMA
& the Wold decomposition

The MA(∞) process and summability

Letting the moving average run over infinitely many past shocks gives the MA(∞) process

$$Y_t=\alpha+\sum_{s=0}^{\infty}\theta_s\varepsilon_{t-s}, \qquad \theta_0:=1,\quad \varepsilon_t\sim\mathrm{WN}(\sigma^2).$$

Its mean is \(\mathbb{E}[Y_t]=\alpha\), unchanged as \(q\to\infty\). Its variance and autocovariances are the limits of the MA(q) formulas,

$$\mathrm{Var}[Y_t]=\sigma^2\sum_{s=0}^{\infty}\theta_s^2, \qquad \gamma(k)=\sigma^2\sum_{s=0}^{\infty}\theta_s\theta_{s+k},$$

provided the infinite sum converges. This is a genuine condition on the coefficients. We say the coefficients are squared summable (SS) if \(\sum_{s=0}^{\infty}\theta_s^2<\infty\), and absolutely summable (AS) if \(\sum_{s=0}^{\infty}|\theta_s|<\infty\). An MA(∞) has finite variance — and so is stationary — whenever its coefficients are squared summable; and since absolute summability implies squared summability, absolutely summable coefficients are enough.

The Wold decomposition

This is the workhorse result of the whole course. It means that to find the mean, variance and ACF of any stationary process — including autoregressive ones — we only have to write it in its MA(∞) form and read the coefficients \(\theta_s\) into the general MA(∞) formulas above. The next section does exactly that for the AR(1).

The AR(1) as an MA(∞)

The first-order autoregressive process makes today a fraction of yesterday plus a shock,

$$Y_t=\mu+\phi_1 Y_{t-1}+\varepsilon_t, \qquad \varepsilon_t\sim\mathrm{WN}(\sigma^2).$$

Substituting the equation into itself repeatedly,

$$Y_t=\mu+\mu\phi_1+\cdots+\mu\phi_1^{\,j}+\phi_1^{\,j+1}Y_{t-(j+1)}+\phi_1^{\,j}\varepsilon_{t-j}+\cdots+\phi_1\varepsilon_{t-1}+\varepsilon_t.$$

If \(|\phi_1|<1\) the carried-forward term \(\phi_1^{\,j+1}Y_{t-(j+1)}\to0\) as \(j\to\infty\), and the constants sum as a geometric series to \(\mu/(1-\phi_1)\). This delivers the MA(∞) representation

$$Y_t=\frac{\mu}{1-\phi_1}+\sum_{s=0}^{\infty}\phi_1^{\,s}\varepsilon_{t-s},$$

an MA(∞) with \(\alpha=\mu/(1-\phi_1)\) and \(\theta_s=\phi_1^{\,s}\). Reading these coefficients into the general formulas gives immediately

$$\mathbb{E}[Y_t]=\frac{\mu}{1-\phi_1}, \qquad \mathrm{Var}(Y_t)=\sigma^2\sum_{s=0}^{\infty}\phi_1^{2s}=\frac{\sigma^2}{1-\phi_1^2}, \qquad \gamma(k)=\phi_1^{\,k}\gamma(0),$$

and hence the AR(1) autocorrelation function

$$\rho(k)=\phi_1^{\,k}, \qquad k=0,1,2,\ldots$$

The lag operator

The lag operator \(L\) shifts a series back one period: \(LY_t=Y_{t-1}\), and more generally \(L^{\,j}Y_t=Y_{t-j}\). A constant is unaffected, \(L\alpha=\alpha\). Crucially, \(L\) behaves algebraically like a polynomial variable, for example

$$(1-L)(1+L)Y_t=(1-L^2)Y_t=Y_t-Y_{t-2}.$$

An AR(1) becomes \((1-\phi_1 L)Y_t=\mu+\varepsilon_t\); an AR(p) becomes \((1-\phi_1 L-\cdots-\phi_p L^p)Y_t=\mu+\varepsilon_t\). Writing models this way lets us factorise the lag polynomial and read off stationarity directly.

AR(p) and ARMA(p,q)

The general autoregressive and autoregressive moving-average models are

$$Y_t=\mu+\phi_1 Y_{t-1}+\cdots+\phi_p Y_{t-p}+\varepsilon_t,$$ $$Y_t=\mu+\phi_1 Y_{t-1}+\cdots+\phi_p Y_{t-p}+\varepsilon_t+\eta_1\varepsilon_{t-1}+\cdots+\eta_q\varepsilon_{t-q}.$$

Setting \(p=0\) recovers the MA(q); setting \(q=0\) recovers the AR(p). Factor the AR lag polynomial as

$$1-\phi_1 L-\cdots-\phi_p L^{p}=(1-\lambda_1 L)(1-\lambda_2 L)\cdots(1-\lambda_p L).$$

For an AR(2), \((1-\phi_1 L-\phi_2 L^2)Y_t=\mu+\varepsilon_t\), stationarity needs both roots \(|\lambda_1|<1,\ |\lambda_2|<1\). If one root equals 1 the process contains a unit root: for instance \(1-L^2=(1-L)(1+L)\) has roots \(\lambda_1=1,\ \lambda_2=-1\) and is non-stationary. When an ARMA is stationary it has the compact Wold (MA(∞)) form

$$Y_t=\frac{\mu}{\phi(1)}+\phi(L)^{-1}\eta(L)\,\varepsilon_t, \qquad \phi(L)=1-\phi_1 L-\cdots-\phi_p L^p,\ \ \eta(L)=1+\eta_1 L+\cdots+\eta_q L^q.$$

The ARMA(1,1)

The ARMA(1,1), \(Y_t=\mu+\phi_1 Y_{t-1}+\varepsilon_t+\eta_1\varepsilon_{t-1}\), is the smallest model with both features. Back-substitution (exactly as for the AR(1)) gives an MA(∞) with \(\theta_0=1\) and \(\theta_s=(\phi_1+\eta_1)\phi_1^{\,s-1}\) for \(s\ge1\), valid when \(|\phi_1|<1\). Reading these into the general MA(∞) formulas yields

$$\mathbb{E}[Y_t]=\frac{\mu}{1-\phi_1}, \qquad \mathrm{Var}[Y_t]=\sigma^2\frac{1+2\phi_1\eta_1+\eta_1^2}{1-\phi_1^2},$$ $$\rho(k)=\begin{cases}\dfrac{(\phi_1+\eta_1)(1+\phi_1\eta_1)}{1+2\phi_1\eta_1+\eta_1^2}, & k=1,\\[6pt] \phi_1^{\,k-1}\rho(1), & k>1.\end{cases}$$

For \(k\ge2\) the ACF decays geometrically like an AR(1); only the lag-1 value differs, because the extra term \(\eta_1\varepsilon_{t-1}\) shifts the immediate correlation. Setting \(\eta_1=0\) collapses everything back to the AR(1) results. A neat special case: if \(\eta_1=-\phi_1\) the process reduces to white noise around \(\mu/(1-\phi_1)\), so all its autocorrelations are zero.

Check your understanding

Derive the variance of a stationary AR(1) from its MA(∞) representation.

The MA(∞) form is \(Y_t=\mu/(1-\phi_1)+\sum_{s=0}^{\infty}\phi_1^{\,s}\varepsilon_{t-s}\), so \(\theta_s=\phi_1^{\,s}\). The general MA(∞) variance is \(\sigma^2\sum_{s=0}^{\infty}\theta_s^2=\sigma^2\sum_{s=0}^{\infty}\phi_1^{2s}\). For \(|\phi_1|<1\) this geometric series sums to \(1/(1-\phi_1^2)\), giving \(\mathrm{Var}(Y_t)=\sigma^2/(1-\phi_1^2)\). Notice the variance explodes as \(|\phi_1|\to1\), foreshadowing the unit-root case.

Is the AR(2) process \(Y_t=1.2Y_{t-1}-0.32Y_{t-2}+\varepsilon_t\) stationary?

The characteristic equation is \(\lambda^2-1.2\lambda+0.32=0\), which factors as \((\lambda-0.8)(\lambda-0.4)=0\), so \(\lambda_1=0.8,\ \lambda_2=0.4\). Both roots satisfy \(|\lambda_j|<1\), so the process is stationary. (Equivalently, \(1-1.2L+0.32L^2=(1-0.8L)(1-0.4L)\), each factor invertible.)

Why does the stationarity of an ARMA(p,q) depend only on its AR part?

A finite moving average is always stationary, since its mean, finite variance and autocovariances never depend on \(t\). Adding an MA(q) term to an AR(p) therefore cannot break stationarity; only the autoregressive polynomial \(\phi(L)\) can, through its roots. So the ARMA(p,q) stationarity condition is identical to the AR(p) one: all roots of the AR characteristic equation inside the unit circle.

Related notes

Downloads