Exercises
& worked solutions

(a) Mean. By linearity and \(\mathbb{E}[\varepsilon_s]=0\), \(\mathbb{E}[Y_t]=\alpha+\mathbb{E}[\varepsilon_t]+\theta_1\mathbb{E}[\varepsilon_{t-1}]+\theta_2\mathbb{E}[\varepsilon_{t-2}]=\alpha\).

Variance. \(\mathrm{Var}[Y_t]=\mathbb{E}[(\varepsilon_t+\theta_1\varepsilon_{t-1}+\theta_2\varepsilon_{t-2})^2]\). Expanding and using \(\mathbb{E}[\varepsilon_t^2]=\sigma^2\), \(\mathbb{E}[\varepsilon_t\varepsilon_s]=0\) for \(t\neq s\), all cross terms vanish and \(\mathrm{Var}[Y_t]=\sigma^2(1+\theta_1^2+\theta_2^2)\).

(b) Autocovariances. Multiplying the windows and keeping only matched shocks (\(\theta_0=1\)):

$$\gamma(1)=\sigma^2(\theta_1+\theta_1\theta_2),\quad \gamma(2)=\sigma^2\theta_2,\quad \gamma(k)=0\ (k>2).$$

Dividing by \(\gamma(0)=\sigma^2(1+\theta_1^2+\theta_2^2)\) gives the ACF

$$\rho(1)=\frac{\theta_1+\theta_1\theta_2}{1+\theta_1^2+\theta_2^2},\quad \rho(2)=\frac{\theta_2}{1+\theta_1^2+\theta_2^2},\quad \rho(k)=0\ (k>2),$$

confirming the MA(2) cut-off after lag 2.

(a) Stationarity. In lag-operator form \((1-\phi_2 L^2)Y_t=\mu+\varepsilon_t\). Factor \((1-\phi_2 L^2)=(1-\sqrt{\phi_2}L)(1+\sqrt{\phi_2}L)\); both roots \(\pm\sqrt{\phi_2}\) lie inside the unit circle iff \(|\phi_2|<1\).

(b) MA(∞). Inverting, \((1-\phi_2 L^2)^{-1}=1+\phi_2 L^2+\phi_2^2 L^4+\cdots\), so

$$Y_t=\frac{\mu}{1-\phi_2}+\varepsilon_t+\phi_2\varepsilon_{t-2}+\phi_2^2\varepsilon_{t-4}+\cdots$$

Here \(\theta_s=\phi_2^{\,s/2}\) for even \(s\) and \(0\) for odd \(s\). Reading these into the MA(∞) formulas:

$$\mathbb{E}[Y_t]=\frac{\mu}{1-\phi_2},\qquad \mathrm{Var}[Y_t]=\sigma^2(1+\phi_2^2+\phi_2^4+\cdots)=\frac{\sigma^2}{1-\phi_2^2}.$$

The autocovariances are zero at odd lags, and for even \(k\), \(\gamma(k)=\sigma^2\phi_2^{\,k/2}/(1-\phi_2^2)\), giving \(\rho(k)=\phi_2^{\,k/2}\) for \(k=2,4,\ldots\) and \(\rho(k)=0\) for odd \(k\) — correlation only at even (seasonal) lags.

(a) Form \(Y_t-\phi_1 Y_{t-1}\). The constant gives \(\mu-\phi_1\mu=\alpha\) with \(\alpha=(1-\phi_1)\mu\); the \(\varepsilon_t\) coefficient is 1; the \(\varepsilon_{t-1}\) coefficient is \(\eta_1-\phi_1=\theta_1\); and for \(s\ge2\) the coefficient is \(\eta_s-\phi_1\eta_{s-1}=0\) by the recursion. Hence \(Y_t-\phi_1 Y_{t-1}=\alpha+\varepsilon_t+\theta_1\varepsilon_{t-1}\), an ARMA(1,1).

(b) Iterating \(\eta_s=\phi_1\eta_{s-1}\) down to \(\eta_1\) gives \(\eta_s=\phi_1^{\,s-1}\eta_1=\phi_1^{\,s-1}(\phi_1+\theta_1)\) for \(s\ge1\).

(c) Absolute summability: \(\sum_{s=0}^{\infty}|\eta_s|=1+|\phi_1+\theta_1|\sum_{s=0}^{\infty}|\phi_1|^s=1+\dfrac{|\phi_1+\theta_1|}{1-|\phi_1|}\), finite iff \(|\phi_1|<1\). So the ARMA(1,1) is stationary exactly when \(|\phi_1|<1\).

(d) Using \(\mathrm{Var}[Y_t]=\sigma^2\sum_{s=0}^{\infty}\eta_s^2\),

$$\mathrm{Var}[Y_t]=\sigma^2\left\{1+(\phi_1+\theta_1)^2\sum_{s=0}^{\infty}\phi_1^{2s}\right\}=\sigma^2\frac{1+2\phi_1\theta_1+\theta_1^2}{1-\phi_1^2}.$$

(A neat check: if \(\theta_1=-\phi_1\) then \(\eta_s=0\) for \(s\ge1\), the process collapses to white noise around \(\mu/(1-\phi_1)\), and all autocorrelations vanish.)

(a) Taking conditional expectations and setting future shocks to zero, \(\hat Y_{T+1}=\alpha+\phi_1 y_T+\phi_2 y_{T-1}\). For two steps, substitute \(\hat Y_{T+1}\):

$$\hat Y_{T+2}=\alpha+\phi_1\hat Y_{T+1}+\phi_2 y_T=\alpha(1+\phi_1)+(\phi_1^2+\phi_2)y_T+\phi_1\phi_2 y_{T-1}.$$

(b) \(\mathrm{FE}(1)=\mathbb{E}[\varepsilon_{T+1}^2]=\sigma^2\). Two steps: the error is \(\varepsilon_{T+2}+\phi_1\varepsilon_{T+1}\), so \(\mathrm{FE}(2)=\sigma^2(1+\phi_1^2)\).

(c) Since \(\mathrm{FE}(2)>\mathrm{FE}(1)\) when \(\phi_1\neq0\), and this continues at longer horizons, forecast accuracy declines as the horizon grows: data become less informative about the distant future.

An i.i.d. sequence has a joint distribution invariant to time shifts, so it is strictly stationary. It is not weakly stationary, however: the Cauchy distribution has no finite mean or variance, so the requirements \(\mathbb{E}[Y_t]=\mu\) and \(\mathrm{Var}[Y_t]<\infty\) fail. This shows strict stationarity does not imply weak stationarity when moments do not exist.

Weak stationarity gives \(\mathbb{E}[\bar Y_T]=\mu\). With absolutely summable autocovariances \(\sum_j|\gamma(j)|<\infty\), the variance \(\mathrm{Var}(\bar Y_T)\to0\) as \(T\to\infty\) (the autocovariances contribute a bounded total that is divided by \(T\)). By Chebyshev's inequality, \(\Pr\{|\bar Y_T-\mu|>\epsilon\}\le\mathrm{Var}(\bar Y_T)/\epsilon^2\to0\), so \(\bar Y_T\xrightarrow{p}\mu\): the sample mean is consistent.