
\documentclass[12pt]{article}
\usepackage{linalg2270strang}
\usepackage{epsfig,amsmath,amsthm,amssymb, verbatim}

% Use this one to print just the question sheet.
%\usepackage[questions]{mathtest}[2001/05/12]

% Use this one to print an entire exam booklet.
%\usepackage[answersheet]{mathtest}[2001/05/12]

% Use this one to print questions and answers.
\usepackage[answers]{mathtest}[2001/05/12]

% For use with pdflatex
% \pdfpagewidth\paperwidth
% \pdfpageheight\paperheight

% Basic User Defs

\def\ds{\displaystyle}


% Beginning of the Document

\begin{document}
\examtitle{MATH 2270-2}{Sample Exam 2}{S2012, revised April 5}

\studentinfo
\instructions{No books or notes. No electronic devices, please.\\
These problems have credits 5 to 15, which is an estimate of the time required to write the solution.
The exam will have 70 points instead of the 140 represented here.}
\finishfirstpage
% Problems Start Here % ----------------------------------------------------- %
\problem{10}{%
Let $A=\begin{pmatrix}
1 & 1 & 1\\
0 & 1 & 1
\end{pmatrix}$. Find a basis of vectors for each of the four fundamental subspaces.
}{}{
The most efficient way to begin is to compute $\rref(A)=\begin{pmatrix}
1 & 0 & 0\\
0 & 1 & 1
\end{pmatrix}$. Then the last frame algorithm supplies a basis vector $\colvector{ 0\\ -1\\ 1}$ for the nullspace of $A$. The computation
also says that the rank of $A$ is two, so the two rows of $A$ are independent. Finally, the pivot columns of $A$ are columns 1,2.

{\bf Row space}: Rows 1 and 2 of A are a basis, because they are independent.

{\bf Column space}: Columns 1 and 2 are the pivot columns of $A$, and they are a basis. This vector space equals
$\mathbb{R}^2$.

{\bf Left nullspace}: Because nullspace(C) $\perp$ rowspace(C) for any matrix C, then $C=A^T$
implies the left nullspace of $A$ is perpendicular to the column space of $A$. The column space of $A$ is the whole space. Then the left nullspace of $A$ is the zero vector alone. In some cases it is easier to find $\rref(A^T)$ and then use the last frame algorithm to compute a basis for the nullspace of $A^T$.

{\bf Nullspace}: Because nullspace($A$) $\perp$ rowspace($A$), and rowspace($A$) is two-dimensional, then nullspace($A$) is one-dimensional. We already computed
a basis vector $\colvector{ 0\\ -1\\ 1}$.
}


\problem{10}{%
Assume $V=\SPAN(\vec{v}_1,\vec{v}_2)$ with $\vec{v}_1=\begin{pmatrix}3\\ 0\\ 4\\0\end{pmatrix},
\vec{v}_2=\begin{pmatrix}1 \\1 \\1 \\ 0\end{pmatrix}$. Find the Gram-Schmidt orthonormal vectors $\vec{q}_1,\vec{q}_2$
whose span equals $V$.
}{}{
$\vec{v}_1$ is not length 1, it has length $\sqrt{3^2+4^2}=5$. Let
$\vec{q}_1$ be the unit vector $\frac{1}{5}\vec{v}_1$. The vector $\vec{q}_2$
cannot equal a scalar multiple of $\vec{v}_2$, because the latter is not perpendicular to $\vec{q}_1$.
We have to do some work to find $\vec{q}_2$.

According to the theory, $\vec{q}_2$ equals $\vec{y}_2$ divided by $\| \vec{y}_2 \|$ and $\vec{y}_2 = \vec{v}_2$ minus the shadow projection vector
of $\vec{v}_2$ onto $\SPAN(\vec{v}_1)$. Then
$$\vec{y}_2 = \vec{v}_2 - c \vec{v}_1,\quad c=\frac{\vec{v}_1\cdot \vec{v}_2}{\vec{v}_1\cdot\vec{v}_1}.$$
Finally, $$\vec{y}_2 = \begin{pmatrix}1\\ 1\\
  1\\ 0\end{pmatrix}-\frac{7}{25}\begin{pmatrix}3\\ 0\\ 4\\
  0\end{pmatrix}=\begin{pmatrix}
\frac{4}{25}\\1\\ -\frac{3}{25}\\0
\end{pmatrix}$$

The
length is $\| \vec{y}_2 \|=\sqrt{(\frac{4}{25})^2+1^2+(-\frac{3}{25})^2}=\sqrt{1+\frac{1}{25}}=\frac{1}{5}\sqrt{26}$.
Then $\vec{q}_2=\frac{1}{\frac{1}{5}\sqrt{26}}\begin{pmatrix}
\frac{4}{25}\\1\\ -\frac{3}{25}\\0 \end{pmatrix}$
and
$\{ \vec{q}_1,\vec{q}_2 \}$ is an orthonormal basis of $V$.
}

\problem{10}{
 Let $Q$ be an orthonormal matrix. The normal equations for the system $Q\vec{x}=\vec{b}$ finds the least squares solution $\vec{v}= QQ^T\vec{b}$. The equations imply that $P=QQ^T$ projects $\vec{b}$ onto the span of the columns of $Q$.
For the subspace $V=\SPAN(\vec{v}_1,\vec{v}_2)$ in the previous problem, find matrix $P$. This matrix projects $\mathbb{R}^4$ onto $V$, while $I-P$ projects
$\mathbb{R}^4$ onto $V^\perp$.
}{}{
Let $Q$ be the matrix whose columns are the vectors $\vec{v}_1$ and
$\vec{v}_2$ from the previous problem. Then
$$
P=QQ^T=
\frac{1}{25\cdot 26}\begin{pmatrix}
3\sqrt{26} & 4\\
0& 25\\
4\sqrt{26} & -3\\
0 & 0
\end{pmatrix}
\begin{pmatrix}
3\sqrt{26} & 0 & 4\sqrt{26} & 0\\
 4 & 5 & -3 & 0\\
\end{pmatrix}\\
=\frac{1}{ 26}
\begin{pmatrix}
10 & \frac{4}{5} & 12 &  0\\
4 & 5& -3 &  0\\
12 & -\frac{3}{5} & 17 &  0\\
0 & 0 & 0 & 0
\end{pmatrix}
$$
}


\problem{10}{
Find the least squares best fit line $y=v_1x+v_2$ for the points $(0,1)$, $(2, 3)$,
$(4, 4)$.}{}{
Substitute the points $(x,y)$ into $y=v_1x+v_2$ to obtain 3 equations in the two unknowns $v_1,v_2$.
Write the equations as a system $A\vec{v}=\vec{b}$, using
$$A=\begin{pmatrix}0 & 1\\ 2 & 1\\ 4 & 1\end{pmatrix},\quad \vec{b}=\begin{pmatrix} 1\\ 3 \\4\end{pmatrix}$$
Unfortunately, $\vec{b}$ is not in the column space of $A$, so $A\vec{v}=\vec{b}$ has no solution. The normal equations $A^TA\vec{v}=A^T\vec{b}$
give a least squares solution
$$\vec{v}=(A^TA)^{-1}A^T\vec{b}$$
Compute $A^TA=\begin{pmatrix}20 & 6\\6 & 3\end{pmatrix}$, then
$(A^TA)^{-1}=\frac{1}{24}\begin{pmatrix}3 & -6\\-6 &
  20\end{pmatrix}$ and
$$(A^TA)^{-1}A^T=\frac{1}{24}\begin{pmatrix} -6& 0& 6\\20 &8
  &-4 \end{pmatrix}$$
Finally,
$$\vec{v}=(A^TA)^{-1}A^T\vec{b}=(A^TA)^{-1}A^T\begin{pmatrix}1\\3\\4\end{pmatrix}=\frac{1}{24}\begin{pmatrix}
  18\\28 \end{pmatrix}$$
The best fit line $y=v_1 x + v_2$ is given by
\[y=\frac{18}{24}x+\frac{28}{24}\]
}

\problem{5}{
Find the determinant of the matrix
\[\begin{pmatrix}
1 & 2 & 3 & 0\\
1 & 1 & 1 & 0\\
0 & 1 & 1 & 0 \\
5 & 6 & 7 & 8
\end{pmatrix}\]
}{}{
The determinant is 8. The fastest method is (2), among the choices (1) The four rules; (2) Cofactor method; (3) Sarrus' $3\times 3$ rule. Please observe that (3) does not apply directly, and there is no Sarrus' Rule for $4\times 4$ matrices.
}

\problem{10}{
Let $A=\begin{pmatrix}
1 &3 \\
 2& 2
\end{pmatrix}$. Find all eigenpairs of $A$.
}{}{
The characteristic polynomial is
\[\left|\begin{matrix}1-\lambda &3 \\
 2&
 2-\lambda\end{matrix}\right|=(1-\lambda)(2-\lambda)-6=\lambda^2-3\lambda
-4=(\lambda -4)(\lambda+1)\]
The eigenvalues $\lambda$ are the roots of the characteristic polynomial: $\lambda=4,-1$. The eigenvalues are distinct, leading to two independent eigenvectors and hence two eigenpairs.

An eigenvector $\vec{v}_1$ for $\lambda_1=4$ is computed from the
nullspace of $A-4ID=\begin{pmatrix}-3 & 3\\2 & -2\end{pmatrix}$. The last frame algorithm implies
$\vec{v}_1=\begin{pmatrix}1\\1\end{pmatrix}$, which is the partial derivative of the general solution on invented symbol $t_1$.

An eigenvector $\vec{v}_2$ for $\lambda_2=-1$ is computed from the
nullspace of $A-(-1)ID=\begin{pmatrix}2 & 3\\2 & 3\end{pmatrix}$. The last frame algorithm implies
$\vec{v}_2=\begin{pmatrix} -\frac{3}{2}\\1\end{pmatrix}$, which is the partial derivative of the general solution on invented symbol $t_1$.
}


\problem{10}{
Let
$A=\begin{pmatrix}
1 & 0 & 0 & 0 & 0 & 0 \\
0 & 2 & 1 & 0 & 0 & 0 \\
0 & 0 & 2 & 0 & 0 & 0 \\
0 & 0 & 0 & 3 & 1 & 0 \\
0 & 0 & 0 & 0 & 3 & 0 \\
0 & 0 & 0 & 0 & 0 & 3
\end{pmatrix}$. Find all eigenpairs.
}{}{
The eigenvalues are the diagonal entries
1, 2, 3. Double and triple roots exist, therefore it is not always true that there will be $n$ eigenpairs ($n=6$ here). For $\lambda=1,2,3,3$
the corresponding eigenvectors are
\[\begin{pmatrix}1\\ 0\\ 0\\0 \\0\\0\end{pmatrix},
\begin{pmatrix}0\\ 1\\ 0\\0 \\0\\0\end{pmatrix},
\begin{pmatrix}0\\ 0\\ 0\\1 \\0\\0\end{pmatrix},
\begin{pmatrix}0\\ 0\\ 0\\0 \\0\\1\end{pmatrix}\]
REMARK. Jordan Theory predicts exactly 4 eigenpairs, with two eigenpairs from root $\lambda=3$. The prediction uses the theory of block matrices $\diag(B_1,B_2,\ldots,B_k)$ and knowledge of examples of Jordan forms. The number of Jordan blocks equals the number of eigenpairs, which is exactly four. Known shortcuts exist for computing the eigenpairs, but these techniques save only 5 minutes of solution time.

ANSWER CHECK. To test the answers, multiply $A\vec{v}$ for an eigenpair $(\lambda,\vec{v})$, then check that the answer
after multiplication simplifies to $\lambda \vec{v}$.
}

\problem{15}{
Find an equation for the plane in $\mathbb{R}^3$ that contains the three points
$(1,0,0)$, $(1,1,1)$, $(1, 2, 0)$. }{}{
The vector from $(1,0,0)$ to $(1,1,1)$ is $\vec{v}=\begin{pmatrix}0\\
  1\\1\end{pmatrix}$, obtained by the {\em head minus tail} rule.
The vector from $(1,0,0)$ to $(1,2,0)$ is $\vec{w}=\begin{pmatrix}0\\
  2\\0\end{pmatrix}$.
\par
The determinant equation for the plane is $(x,y,z)-(1,0,0)$ dot product with the
vector cross product of $\vec{v}$ and $\vec{w}$, known as the scalar triple product:
$$
\left|
\begin{array}{ccc}
x-1 & y-0 & z-0 \\
0 & 1 & 1 \\
0 & 2 & 0
\end{array}\right|=0.
$$
{\bf Geometry}. The plane containing the three points is a collection $V$ of vectors $\begin{pmatrix}1\\0\\0\end{pmatrix}+a \begin{pmatrix}0\\
  1\\1\end{pmatrix}+b\begin{pmatrix}0\\
  2\\0\end{pmatrix}$, where $a$ and $b$ are arbitrary real numbers.
\par
 Set $V$ is not a subspace, it is vector $\begin{pmatrix}1\\0\\0\end{pmatrix}$ translated by the
points in the vector subspace spanned by $\vec{v}$ and $\vec{w}$. This is called a {\bf linear variety}.
}

\problem{10}{
Suppose an $n\times n$ matrix $A$ has all eigenvalues equal to 0. Show
from the Cayley-Hamilton Theorem that $A^n$ has all entries equal to 0.}{}{
Solution: The proof results from showing that the characteristic equation of $A$ is $\lambda^n=0$, because then by the Cayley-Hamilton theorem the matrix $A$ satisfies its own characteristic equation, giving $A^n=0$, which is the claimed result.

The characteristic
polynomial of $A$ is $|A-\lambda I| = (-\lambda)^n + c_1 (-\lambda)^{n-1}+ \cdots + c_n$. If $\lambda=0$ is the only root of $|A-\lambda I|=0$,
then the Root-Factor theorem of college algebra implies that this polynomial has $n$ factors of $(\lambda - 0)$. Therefore, $c_1 = \cdots = c_n=0$ and $|A-\lambda I|=(-\lambda)^n=(-1)^n\lambda^n$. Then the characteristic equation of $A$ is $(-1)^n\lambda^n=0$, or equivalently, $\lambda^n=0$.
}

\problem{15}{
Prove the Cayley-Hamilton Theorem for $2\times 2$ matrices with real
eigenvalues.}{}{
\def\trace{\mbox{\bf trace}}
Start with $A=\twobytwo{a&b}{c&d}$, characteristic equation  $\lambda^2 + c_1 \lambda + c_2 = 0$, where $c_1=-\trace(A) = -(a+d)$ and $c_2=\det(A)=ad-bc$}
Write the characteristic equation as $\lambda^2+c1\lambda = -c_2$, then substitute as in the Cayley-Hamilton theorem, arriving at the proposed equation $A^2+c_1A = -c_2I$. Expand the left side:
$$A^2+c_1A = A(A+c_1I) = A(A - (a+d)I) = -A\adj(A),\quad \adj(A)=\twobytwo{d&-b}{-c&a}.$$
Because $A\adj(A) = |A|I$ (the adjugate identity), then the right side  of the preceding display simplifies to $-\det(A)I = -c_2I$. This proves the Cayley-Hamilton theorem for $2\times 2$ matrices: $A^2+c_1A = -c_2I$.

 \problem{5}{Suppose a $3\times 3$ matrix $A$ has eigenpairs
 $$\left(3,\colvector{1\\ 2\\ 0}\right),\quad
 \left(3,\colvector{1\\ 1\\ 0}\right),\quad \left(0,\colvector{0\\ 0\\ 1}\right).$$
Display an invertible matrix $P$ and a
 diagonal matrix $D$ such that $AP=PD$.}{}{ Define $P=\threebythree{1 & 1 &0}{2 & 1 & 0}{0 &0&1}$, $D=\threebythree{3&0&0}{0&3&0}{0&0&0}$.
Then $AP=PD$.}

 \problem{10}{Suppose a $3\times 3$ matrix $A$ has eigenpairs
 $$\left(3,\colvector{1\\ 2\\ 0}\right),\quad
 \left(3,\colvector{1\\ 1\\ 0}\right),\quad \left(0,\colvector{0\\ 0\\ 1}\right).$$
Find $A$.}{}{ Define $P=\threebythree{1 & 1 &0}{2 & 1 & 0}{0 &0&1}$, $D=\threebythree{3&0&0}{0&3&0}{0&0&0}$.
Then $AP=PD$. Compute $A=PDP^{-1}$ from two matrix multiplications. The answer is
$$A=\threebythree{1 & 1 &0}{2 & 1 & 0}{0 &0&1}\threebythree{3&0&0}{0&3&0}{0&0&0}\threebythree{-1 & 1 &0}{2 & -1 & 0}{0 &0&1}=\threebythree{3&0&0}{0&3&0}{0&0&0}.$$
}

\problem{10}{Assume $A$ is $2\times 2$ and Fourier's model holds:
$$A\left(c_1\colvector{1 \\ 1} + c_2\colvector{1 \\ -1}\right) =2 c_2\colvector{1 \\ -1}.$$
 Find $A$.}{}{
 % A:=matrix([[1,-1],[-1,1]]); eigenvects(A);
 % ans == [0, 1, {vector([1, 1])}], [2, 1, {vector([-1, 1])}]
Construct from Fourier's Model the eigenpairs $\left(0,\colvector{1\\ 1}\right)$, $\left(2,\colvector{-1 \\ 1}\right)$. Then
$P=\twobytwo{1 & 1}{1 & -1}$ and $D=\twobytwo{0 &0}{0&2}$. Then $AP=PD$ implies
$$A=PDP^{-1}=\twobytwo{1&1}{1&-1}\twobytwo{0&0}{0&2}
\twobytwo{.5&.5}{.5&-.5}=\twobytwo{1&-1}{-1&1}.$$}

\problem{10}{ How many eigenpairs? (a)
$\threebythree{0&1&0}{0&0&1}{0&0&0}$, (b)
$\threebythree{0&0&1}{0&0&0}{0&0&0}$.}{}{ (a) Just one. This is a Jordan
block, and such blocks have exactly one eigenpair. (b) Exactly two
eigenpairs. The eigenvalues are on the diagonal, $0,0,0$. Then for
$\lambda=0$ we have $B=A-\lambda I = A$ is already in reduced echelon
form. The equations for $\vec{v}=(x_1,x_2,x_3)$ are $x_3=0,0=0,0=0$.
Then $\vec{v}$ is a linear combination of two special solutions, because
there are two free variables, hence two eigenpairs.
}

\problem{5}{ True or False?
A Jordan block has one and only one eigenpair.
}{}{
True. The eigenvalues are on the diagonal, $\lambda,\lambda,\ldots,\lambda$. Then
$B=A-\lambda I$ is already in reduced echelon
form
$$B=\left(\begin{array}{cccccc}
0 & 1 & 0 & \cdots     & 0 & 0 \\
  &   &   & \vdots     &   & \\
0 & 0 & 0    & \cdots  & 0 &1 \\
0 & 0 & 0 & \cdots     & 0 &0
\end{array}\right)
$$
and all variables are lead variables, except the first variable, which
is a free variable. So there is one special solution and therefore just
one eigenpair.
}

\problem{5}{ True or False?
A diagonal block matrix $A=\twobytwo{B_1 & 0}{0 & B_2}$ where $B_1,B_2$ are Jordan blocks has
exactly two eigenpairs.
}{}{
True. Although this does not follow directly from the previous problem, it
can be analyzed in the same way, one Jordan block at a time.
}




\phantom{page}
\vfill \begin{center} No new questions beyond this point.\end{center}
%\anotherpage

%
%\phantom{page}
%\anotherpage
%
%\phantom{page}
%\anotherpage
%
%\phantom{page}
%\anotherpage
%
%\phantom{page}
%\anotherpage
%
%
%\phantom{page}
%\anotherpage


% Problems End Here % ------------------------------------------------------- %

\problemsdone
\end{document}

% End of the Document
