Practice problems
& full solutions

(a) Uniqueness via the determinant.

Write the system as \(A\mathbf x=\mathbf b\) with

$$A=\begin{pmatrix}2&0&1\\1&2&1\\2&\delta&1\end{pmatrix},\qquad \mathbf b=\begin{pmatrix}1\\2\\1\end{pmatrix}.$$

A square system has a unique solution exactly when \(\det A\neq 0\). Expanding along the first row,

$$\det A=2\det\!\begin{pmatrix}2&1\\\delta&1\end{pmatrix}+1\cdot\det\!\begin{pmatrix}1&2\\2&\delta\end{pmatrix}=2(2-\delta)+(\delta-4)=-\delta.$$

So \(\det A=-\delta\), which is non-zero precisely when \(\delta\neq 0\). Hence the system has a unique solution for all \(\delta\neq 0\), and fails to have a unique solution only at \(\delta=0\).

(b) Solving at \(\delta=2\).

At \(\delta=2\), \(\det A=-2\neq0\), so \(A\) is invertible and \(\mathbf x=A^{-1}\mathbf b\). The inverse is

$$A^{-1}=\begin{pmatrix}0&-1&1\\-\tfrac12&0&\tfrac12\\1&2&-2\end{pmatrix},$$

so that

$$\mathbf x=A^{-1}\begin{pmatrix}1\\2\\1\end{pmatrix}=\begin{pmatrix}0-2+1\\-\tfrac12+0+\tfrac12\\1+4-2\end{pmatrix}=\begin{pmatrix}-1\\0\\3\end{pmatrix}.$$

Check in the original equations: \(2(-1)+3=1\) ✓, \(-1+0+3=2\) ✓, \(2(-1)+2(0)+3=1\) ✓. The solution is \((x_1,x_2,x_3)=(-1,0,3)\).

(c) Adding a fourth equation.

A useful observation: subtract equation 1 from equation 3 to get \(\delta x_2=0\), and subtract equation 1 from equation 2 to get \(-x_1+2x_2=1\). For any \(\delta\neq0\) the first relation forces \(x_2=0\), then \(x_1=-1\) and \(x_3=3\). So for every \(\delta\neq0\) the three-equation system has the same unique solution \((-1,0,3)\).

The enlarged four-equation system has a unique solution only if this point also satisfies the new equation \(\delta x_1+x_2=-2\):

$$\delta(-1)+0=-2\ \Longrightarrow\ \delta=2.$$

Since \(\delta=2\neq0\), the three-equation part is indeed uniquely solvable there. Therefore the four-equation system has a (unique, consistent) solution only when \(\boxed{\delta=2}\).

(a) Eigenvalues, eigenvectors and the general solution.

The system is \(\dot{\mathbf v}=A\mathbf v\) with \(\mathbf v=(x,y,z)^{\top}\) and coefficient matrix

$$A=\begin{pmatrix}2&-1&1\\0&4&3\\0&1&2\end{pmatrix}.$$

Because the first column is \((2,0,0)^{\top}\), expanding the characteristic determinant along it gives

$$\det(A-\lambda I)=(2-\lambda)\big[(4-\lambda)(2-\lambda)-3\big]=(2-\lambda)(\lambda^2-6\lambda+5)=(2-\lambda)(\lambda-1)(\lambda-5).$$

The eigenvalues are \(\lambda=5,\,2,\,1\). Solving \((A-\lambda I)\mathbf v=\mathbf 0\) for each:

• \(\lambda=5\): from \(-v_2+3v_3=0\) and the top row, \(\mathbf v^{(1)}=(-2,9,3)^{\top}\).
• \(\lambda=2\): the second and third rows force \(v_2=v_3=0\), giving \(\mathbf v^{(2)}=(1,0,0)^{\top}\).
• \(\lambda=1\): from \(v_2+v_3=0\) and \(v_1-v_2+v_3=0\), \(\mathbf v^{(3)}=(-2,-1,1)^{\top}\).

The general solution is a combination of the modes \(e^{\lambda t}\mathbf v\):

$$\begin{aligned}x(t)&=-2C_1e^{5t}+C_2e^{2t}-2C_3e^{t},\\ y(t)&=9C_1e^{5t}-C_3e^{t},\\ z(t)&=3C_1e^{5t}+C_3e^{t},\end{aligned}$$

for arbitrary constants \(C_1,C_2,C_3\).

(b) Imposing the initial conditions.

Setting \(t=0\) and \(x(0)=y(0)=z(0)=1\) gives

$$1=-2C_1+C_2-2C_3,\qquad 1=9C_1-C_3,\qquad 1=3C_1+C_3.$$

Add the last two equations: \(2=12C_1\Rightarrow C_1=\tfrac16\). Substituting into \(1=3C_1+C_3\) gives \(C_3=1-\tfrac12=\tfrac12\). Finally, from the first equation \(C_2=1+2C_1+2C_3=1+\tfrac13+1=\tfrac{7}{3}\). Hence

$$C_1=\tfrac16,\qquad C_2=\tfrac{7}{3},\qquad C_3=\tfrac12,$$

and the particular solution follows by inserting these into the general solution above.

(c) Steady state.

A steady state has \(\dot x=\dot y=\dot z=0\), i.e. \(A\mathbf v=\mathbf 0\). Since all eigenvalues are non-zero, \(A\) is invertible, so the only solution is

$$x=y=z=0.$$

(Because every eigenvalue is positive, this steady state is unstable: any deviation grows over time.)

(a) Gradient.

Writing \(g=1+x_1^2+x_2^2\), the chain rule gives \(\partial f/\partial x_i=-2x_i\,g^{-2}\), so

$$\nabla f(x_1,x_2)=\big(-2x_1 g^{-2},\,-2x_2 g^{-2}\big).$$

At \((1,1)\), \(g=3\), so \(g^{-2}=\tfrac19\) and \(\nabla f(1,1)=\left(-\tfrac29,-\tfrac29\right)\).

(b) Directional derivative.

The vector \(\mathbf u=(1/\sqrt2,1/\sqrt2)\) is a unit vector, so the directional derivative is the dot product

$$D_{\mathbf u}f(1,1)=\nabla f(1,1)\cdot\mathbf u=-\tfrac29\cdot\tfrac{1}{\sqrt2}-\tfrac29\cdot\tfrac{1}{\sqrt2}=-\frac{4}{9\sqrt2}=-\frac{2\sqrt2}{9}.$$

(c) Second-order Taylor approximation.

We need the value, gradient and Hessian at \((1,1)\). The value is \(f(1,1)=\tfrac13\). Differentiating again,

$$f_{11}=g^{-3}(6x_1^2-2x_2^2-2),\quad f_{22}=g^{-3}(6x_2^2-2x_1^2-2),\quad f_{12}=8x_1x_2\,g^{-3}.$$

At \((1,1)\), \(g=3\) and \(g^{-3}=\tfrac{1}{27}\), giving

$$H^f(1,1)=\frac{1}{27}\begin{pmatrix}2&8\\8&2\end{pmatrix}.$$

The second-order Taylor expansion \(f(\mathbf x)\approx f(\mathbf a)+\nabla f(\mathbf a)\cdot(\mathbf x-\mathbf a)+\tfrac12(\mathbf x-\mathbf a)^{\top}H^f(\mathbf a)(\mathbf x-\mathbf a)\) with \(\mathbf a=(1,1)\) is therefore

$$f(x_1,x_2)\approx \tfrac13-\tfrac29(x_1-1)-\tfrac29(x_2-1)+\tfrac{1}{27}\Big[(x_1-1)^2+8(x_1-1)(x_2-1)+(x_2-1)^2\Big].$$

(d) Concave, convex, or neither?

Curvature is decided by the definiteness of the Hessian. At \((1,1)\),

$$\det H^f(1,1)=\frac{1}{27^2}\big(2\cdot2-8\cdot8\big)=\frac{4-64}{729}=-\frac{60}{729}<0.$$

A negative determinant means the two eigenvalues have opposite signs, so \(H^f(1,1)\) is indefinite. A function that is concave (convex) everywhere must have a negative-(positive-)semidefinite Hessian at every point; an indefinite Hessian at even one point rules both out. Hence \(f\) is neither concave nor convex on \(\mathbb{R}^2\).

This makes geometric sense: \(f\) is a "bump" peaking at the origin (where \(H^f(0,0)=-2I\) is negative definite, so \(f\) is locally concave) and flattening out far away, so its curvature changes sign across the plane.

(a) Definitions.

• Convex set. A set \(S\) is convex if \(\lambda\mathbf x+(1-\lambda)\mathbf y\in S\) for all \(\mathbf x,\mathbf y\in S\) and all \(\lambda\in[0,1]\).
• Concave function. A function \(f\) with convex domain \(S\) is concave if \(f(\lambda\mathbf x+(1-\lambda)\mathbf y)\ge \lambda f(\mathbf x)+(1-\lambda)f(\mathbf y)\) for all \(\mathbf x,\mathbf y\in S\), \(\lambda\in[0,1]\).
• Quasiconcave function. A function \(g\) with convex domain \(S\) is quasiconcave if its upper contour set \(U_a^{g}\equiv\{\mathbf x\in S:g(\mathbf x)\ge a\}\) is convex for every \(a\in\mathbb{R}\). Equivalently, \(g(\lambda\mathbf x+(1-\lambda)\mathbf y)\ge\min\{g(\mathbf x),g(\mathbf y)\}\).

A useful fact used below: every concave function is quasiconcave. Indeed, if \(\mathbf x,\mathbf y\in U_a^{f}\) then \(f(\mathbf x)\ge a\) and \(f(\mathbf y)\ge a\), and concavity gives \(f(\lambda\mathbf x+(1-\lambda)\mathbf y)\ge\lambda f(\mathbf x)+(1-\lambda)f(\mathbf y)\ge\lambda a+(1-\lambda)a=a\), so \(U_a^{f}\) is convex.

(b) A strictly increasing transformation of a concave function is quasiconcave.

Fix any \(a\in\mathbb{R}\) and consider the upper contour set of \(m=t\circ f\). Because \(t\) is strictly increasing it is invertible, and \(t(f(\mathbf x))\ge a\iff f(\mathbf x)\ge t^{-1}(a)\). Therefore

$$U_a^{m}=\{\mathbf x\in S:t(f(\mathbf x))\ge a\}=\{\mathbf x\in S:f(\mathbf x)\ge t^{-1}(a)\}=U_{t^{-1}(a)}^{f}.$$

The right-hand set is an upper contour set of the concave function \(f\), which we showed is convex. So every upper contour set of \(m\) is convex, i.e. \(m\) is quasiconcave. (Note \(m\) itself need not be concave; monotone transformations preserve quasiconcavity but not concavity, which is exactly why quasiconcavity is the natural "ordinal" curvature notion for utility.)

(c) The sum of two concave functions is quasiconcave.

First, \(h=f+g\) is concave: for \(\mathbf x,\mathbf y\in S\), \(\lambda\in[0,1]\),

$$h(\lambda\mathbf x+(1-\lambda)\mathbf y)=f(\lambda\mathbf x+(1-\lambda)\mathbf y)+g(\lambda\mathbf x+(1-\lambda)\mathbf y)\ge\big[\lambda f(\mathbf x)+(1-\lambda)f(\mathbf y)\big]+\big[\lambda g(\mathbf x)+(1-\lambda)g(\mathbf y)\big]=\lambda h(\mathbf x)+(1-\lambda)h(\mathbf y).$$

Since \(h\) is concave, it is quasiconcave by the fact established in part (a): each upper contour set \(U_a^{h}=\{\mathbf x:h(\mathbf x)\ge a\}\) is convex. Hence \(h=f+g\) is quasiconcave.

(a) Lagrangian and Kuhn–Tucker conditions.

Minimising \((y-5)^2+(x-5)^2\) is equivalent to maximising its negative. Attaching multipliers \(\lambda_1\ge0\) to \(x+y\le9\), and \(\lambda_2\ge0\), \(\lambda_3\ge0\) to the sign constraints \(x\ge0\), \(y\ge0\), the Lagrangian is

$$\mathcal L=-(y-5)^2-(x-5)^2-\lambda_1(x+y-9)+\lambda_2 x+\lambda_3 y.$$

The Kuhn–Tucker conditions are the stationarity conditions together with complementary slackness:

$$\frac{\partial\mathcal L}{\partial x}=-2(x-5)-\lambda_1+\lambda_2=0,\qquad \frac{\partial\mathcal L}{\partial y}=-2(y-5)-\lambda_1+\lambda_3=0,$$

$$\lambda_1(x+y-9)=0,\ \lambda_1\ge0,\ x+y\le9;\qquad \lambda_2 x=0,\ \lambda_2\ge0,\ x\ge0;\qquad \lambda_3 y=0,\ \lambda_3\ge0,\ y\ge0.$$

(b) Solving the system.

The unconstrained minimiser of \((x-5)^2+(y-5)^2\) is \((5,5)\), but that violates \(x+y\le9\) (since \(5+5=10>9\)), so the inequality \(x+y\le9\) must bind: \(\lambda_1>0\) and \(x+y=9\). With \(x+y=9\), both \(x\) and \(y\) are strictly positive, so complementary slackness forces \(\lambda_2=\lambda_3=0\). The two stationarity conditions then read

$$-2(x-5)=\lambda_1,\qquad -2(y-5)=\lambda_1,$$

so \(x=y\). Combined with \(x+y=9\) this gives \(x=y=4.5\), and \(\lambda_1=-2(4.5-5)=1\ge0\), consistent with all sign requirements. The solution is

$$(x^*,y^*,\lambda_1^*,\lambda_2^*,\lambda_3^*)=(4.5,\ 4.5,\ 1,\ 0,\ 0).$$

Geometrically, \((4.5,4.5)\) is the point on the line \(x+y=9\) closest to \((5,5)\), exactly the perpendicular projection, which is what minimising squared distance to \((5,5)\) subject to the constraint must deliver.

(c) Why this is the minimiser.

The objective \((x-5)^2+(y-5)^2\) has Hessian \(\begin{pmatrix}2&0\\0&2\end{pmatrix}\), which is positive definite, so the objective is (strictly) convex. The feasible region is the intersection of three half-planes \(\{x\ge0\}\), \(\{y\ge0\}\), \(\{x+y\le9\}\), and an intersection of convex sets is convex. For a convex objective minimised over a convex feasible set, the Kuhn–Tucker conditions are not just necessary but sufficient, and any point satisfying them is a global minimiser. Hence \((x^*,y^*)=(4.5,4.5)\) is the global solution.