Week 7

Subspace

For a matrix $A = \begin{pmatrix} | & & | \\ \vector{v}_1 & \cdots & \vector{v}_2 \\ | & & | \end{pmatrix}$.

Kernel

$$\ker(A) = \set{\vector{x}\in\R^n : A\vector{x}=\vector{0}}$$

To find a basis for $\ker(A)$:

1. Find $\operatorname{rref}(A)$.
2. Get the solution set.
3. Write the solution as a span of vectors.
4. Then, the vectors in the span are the basis of $W$.

Image

$$\begin{align*} \operatorname{Im}(A) &= \set{A\vector{x}: \vector{x}\in\R^n} \\ &= \operatorname{span}\set{\vector{v}_1,\dots,\vector{v}_2} \end{align*}$$

To find a basis for $\operatorname{Im}(A)$:

1. Find $\operatorname{rref}(A)$.
2. Then, $\set{\vector{v}_i}$ corresponding to the pivot are the basis for $\operatorname{Im}(A)$.

So, for an $m\times n$ matrix $A$,

$$\dim(\ker(A)) = \operatorname{nullity}(A) = n - \operatorname{rank}(A) \\ \dim(\operatorname{Im}(A)) = \operatorname{rank}(A)$$

A summary of findings from the three chapters

For an $m\times n$ matrix $$A=\begin{pmatrix} | & & | \\ \vector{v}_1 & \cdots & \vector{v}_n \\ | & & |\end{pmatrix}$$	Chapter 1: Applying algorithms $$A\to\operatorname{rref}(A)$$	Chapter 2: Linear transformation $$T(\vector{x})=A\vector{x}$$	Chapter 3: $$\set{\vector{v}_1, \ldots, \vector{v}_n}$$
Only trivial solutions $\\A\vector{x}=\vector{0}\implies\vector{x}=\vector{0}$	No free variables $\\\operatorname{rank}(A)=n$	$T$ is injective $\\\ker(A)=\set{\vector{0}}$	$\set{\vector{v}_1, \ldots, \vector{v}_n}$ is linearly independent
$A\vector{x}=\vector{b}$ is solvable for all $\vector{b}\in\R^m$	Full row rank $\\\operatorname{rank}(A)=m$	$T$ is surjective $\\\operatorname{Im}(A)=\R^m$	$\operatorname{span}\set{\vector{v}_1, \ldots, \vector{v}_n}=\R^m$
If both	$A$ is a square matrix $\\\operatorname{rank}(A)=m=n$	$T$ is bijective	$\set{\vector{v}_1, \ldots, \vector{v}_n}$ is a basis

No matter what you are trying to do… bring the vectors into a matrix and Gaussian that bitch.

Change of basis

For a standard basis:

• $\R^n = \operatorname{span}\set{\vector{e}_1, \ldots, \vector{e}_n}$
• $\begin{pmatrix} x_1 \\ \vdots \\ x_n \end{pmatrix} = x_1\vector{e}_1 + \cdots + x_n\vector{e}_n$

If we have a basis $\mathfrak{B}=\set{\vector{v}_1,\ldots,\vector{v}_n}$ of $\R^n$, then

$$\begin{array}{ll} \forall\vector{x}\in\R^n & \vector{x}=\tilde{x}_1\vector{v}_1 + \cdots + \tilde{x}_n\vector{v}_n \\ & [\vector{x}]_\mathfrak{B} \stackrel{\Delta}{=} \begin{pmatrix} \tilde{x}_1 \\ \vdots \\ \tilde{x}_n \end{pmatrix}_\mathfrak{B} \end{array}$$

Homework 6 hint — Book question #55

We want a $P$ such that $[\vector{x}]_\mathfrak{R}=[\vector{x}]_\mathfrak{B}$.

Let $U = \begin{pmatrix} 1 & 1 \\ 1 & 2 \end{pmatrix}$ and $V = \begin{pmatrix} 1 & 3 \\ 2 & 4 \end{pmatrix}.$

By definition:

$$\vector{x} = U[\vector{x}]_\mathfrak{B} \\ \vector{x} = V[\vector{x}]_\mathfrak{R}$$

Rewrite the second one by applying inverse on both sides:

$$\vector{x} = V[\vector{x}]_\mathfrak{R} \implies V^{-1}\vector{x} = [\vector{x}]_\mathfrak{R}$$

Then plug in the expression for $\vector{x}$ in terms of the $U$ transformation.

$$\begin{align*} [\vector{x}]_\mathfrak{R} &= V^{-1}\vector{x} \\ &= V^{-1}U[\vector{x}]_\mathfrak{B} \end{align*}$$

By comparison, $V^{-1}U$ is the transformation $P$.

Linear transformation

Now, we look at the change of basis as a linear transformation. Using the definition and applying the inverse, we can find the transformation matrix $A$ and $B$.

$$\begin{CD} \vector{x} @>A>> T(\vector{x}) \\ @VPVV @VVPV \\ [\vector{x}]_\mathfrak{B} @>B>> [T(\vector{x})]_\mathfrak{B} \end{CD}$$

Why do we need to change basis?

Just because we can, we will jump ahead to Chapter 7: Diagonalization to explore why we do this.

Say, if we are given a square matrix $A$, how do we find $A^n$?

$$A^n = \underbrace{A\cdot A \cdots A\cdot A}_n$$

Take an easy example, where $A=I$. Then,

$$A^n = I^n = I$$

Or, how about diagonal matrices? For example: $A = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}$.

We can just multiply the diagonal — easy!

$$A = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix} \implies A^2 = \begin{pmatrix} 2^2 & 0 \\ 0 & 3^2 \end{pmatrix} \implies A^n = \begin{pmatrix} 2^n & 0 \\ 0 & 3^n \end{pmatrix}$$

But… how would we deal with non-diagonal matrices? We simply cannot do proof by induction to find a “general” form for which we can multiply matrix. And so, this is where change of basis is needed.

Under standard basis, given a transformation matrix $\begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix}$:

$$\begin{array}{ccc} \includegraphics[height=150px]{../assets/notes_wk7_stdbasis_1.svg} & \raisebox{75px}{$\xrightarrow{T}$} & \includegraphics[height=150px]{../assets/notes_wk7_stdbasis_2.svg} \end{array}$$

We transform the vectors $\vector{e}_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix}$ and $\vector{e}_2 = \begin{pmatrix} 0 \\ 1 \end{pmatrix}$, such that:

$$\begin{align*} T(\vector{e}_1) = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix} \begin{pmatrix} 1 \\ 0 \end{pmatrix} = \begin{pmatrix} 2 \\ 0 \end{pmatrix} = 2\vector{e}_1 \\ T(\vector{e}_2) = \begin{pmatrix} 2 & 0 \\ 0 & 3 \end{pmatrix} \begin{pmatrix} 0 \\ 1 \end{pmatrix} = \begin{pmatrix} 0 \\ 3 \end{pmatrix} = 3\vector{e}_2 \end{align*}$$

What we want to achieve in general

Given a matrix $A$ under the standard basis, we want to find another basis $\mathfrak{B}=\set{\vector{v}_1, \vector{v}_2}$ and $\lambda_1, \lambda_2 \in \R$ such that:
$$(A\vector{x})_\mathfrak{B} = \begin{pmatrix} \lambda_1 & 0 \\ 0 & \lambda_2 \end{pmatrix}(\vector{x})_\mathfrak{B}$$

Let $P=\begin{pmatrix} | & | \\ \vector{v_1}& \vector{v}_n \\ | & | \end{pmatrix}$. And recall that $\vector{x}=P[\vector{x}]_\mathfrak{B}$. Then:

$$\begin{align*} P^{-1}A\vector{x} &= \begin{pmatrix} \lambda_1 & 0 \\ 0 & \lambda_2 \end{pmatrix} P^{-1}\vector{x} \\ P^{-1}AP &= \begin{pmatrix} \lambda_1 & 0 \\ 0 & \lambda_2 \end{pmatrix} \end{align*}$$

First, we need to solve for $\vector{v}_1$ and $\lambda_1$.

Let $[\vector{x}]_\mathfrak{B}=\begin{pmatrix} 1 \\ 0 \end{pmatrix}_\mathfrak{B}$. Where $\vector{x}=\vector{v}_1$, we have that

$$\begin{align*} (A\vector{v}_1)_\mathfrak{B} = \begin{pmatrix} \lambda_1 & 0 \\ 0 & \lambda_2 \end{pmatrix} \begin{pmatrix} 1 \\ 0 \end{pmatrix}_\mathfrak{B} = \begin{pmatrix} \lambda_1 \\ 0 \end{pmatrix}_\mathfrak{B} \end{align*}$$

Where $\vector{v}_1,\vector{v}_2\neq\vector{0}$,
$$A\vector{v}_1 = \lambda\vector{v}_1 \\ A\vector{v}_2 = \lambda\vector{v}_2.$$

So in general, we need to solve for $A\vector{v}=\lambda\vector{v}$ such that $\vector{v}\neq\vector{0}$ and $(A-\lambda I)\vector{v}=\vector{0}$. Or, in other words where $\ker(A-\lambda I)$ is non-trivial.

---

Eigenvalues

Why diagonalize?

Because for diagonal matrices, we are easily able to find their squares, such as:

$$\begin{align*} P^{-1}AP &= \begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix} \\ (P^{-1}AP)^2 &= \begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix}^2 = \begin{pmatrix} \lambda_1^2 & & 0 \\ & \ddots & \\ 0 & & \lambda_n^2 \end{pmatrix} \end{align*}$$

And notice where

$$\begin{align*} (P^{-1}AP)^2 &= P^{-1}A\underbrace{PP^{-1}}_I AP \\ &= P^{-1}AAP \\ &= P^{-1}A^2P \end{align*}$$

We can then derive an expression for $A^2$:

$$P^{-1}A^2P = \begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix}^2 \\ A^2 = P \begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix}^2 P^{-1} = P \begin{pmatrix} \lambda_1^2 & & 0 \\ & \ddots & \\ 0 & & \lambda_n^2 \end{pmatrix} P^{-1}$$

And so, for $(P^{-1}AP)^k = \begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix}^k = \begin{pmatrix} \lambda_1^k & & 0 \\ & \ddots & \\ 0 & & \lambda_n^k \end{pmatrix}$:

$$\begin{align*} (P^{-1}AP)^k &= \underbrace{(P^{-1}A\cancel{P})(\cancel{P^{-1}}A\cancel{P})\thinspace\cdots\thinspace(\cancel{P^{-1}}A\cancel{P})(\cancel{P^{-1}}AP)}_k \\ &= P^{-1}\underbrace{AA\thinspace\cdots\thinspace AA}_k P \\ &= P^{-1}A^k P \end{align*}$$

And again:

$$P^{-1}A^kP = \begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix}^k \\ A^k = P \begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix}^k P^{-1} = P \begin{pmatrix} \lambda_1^k & & 0 \\ & \ddots & \\ 0 & & \lambda_n^k \end{pmatrix} P^{-1}$$

Now, all square matrices to the power of $k$ can be found… once we find the matrix $P$ and the eigenvalues $\lambda_1,\ldots,\lambda_n$.

Finding eigenvalues and $P$

For an $n\times n$ matrix $A$ and some invertible matrix $P$ where:

$$P^{-1}AP = \begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix} \\ \Downarrow \\ AP = P\begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix}$$

First, we write $P$ as a matrix composed of some column vectors $\vector{v}_i$.

$$P = \begin{pmatrix} | & & | \\ \vector{v}_1 & \cdots & \vector{v}_n \\ | & & | \end{pmatrix}$$

Then, on the left-hand side, we have:

$$\begin{align*} AP &= A\begin{pmatrix} | & & | \\ \vector{v}_1 & \cdots & \vector{v}_n \\ | & & | \end{pmatrix} \\ &= \begin{pmatrix} | & & | \\ A\vector{v}_1 & \cdots & A\vector{v}_n \\ | & & | \end{pmatrix} \end{align*}$$

And on the right:

$$\begin{align*} P\begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix} &= \begin{pmatrix} | & & | \\ \vector{v}_1 & \cdots & \vector{v}_n \\ | & & | \end{pmatrix} \begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix} \\ &= \begin{pmatrix} | & & | \\ \lambda_1\vector{v}_1 & \cdots & \lambda_n\vector{v}_n \\ | & & | \end{pmatrix} \end{align*}$$

Putting it together, we have:

$$\begin{align*} AP &= P\begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix} \\ \begin{pmatrix} | & & | \\ A\vector{v}_1 & \cdots & A\vector{v}_n \\ | & & | \end{pmatrix} &= \begin{pmatrix} | & & | \\ \lambda_1\vector{v}_1 & \cdots & \lambda_n\vector{v}_n \\ | & & | \end{pmatrix} \end{align*}$$

So, if there exists an invertible matrix $P$ where $P^{-1}AP = \begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix}$. Then, there must exists a basis $\set{\vector{v}_1,\ldots,\vector{v}_n}$ such that $\vector{v}_i\neq\vector{0}$.

$$\begin{array}{ccc} \begin{pmatrix} A\vector{v}_1 = \lambda_1\vector{v}_1 \\ \vdots \\ A\vector{v}_n = \lambda_n\vector{v}_n \\ \end{pmatrix} &\overset{ \raisebox{1em}{ \boxed{A\vector{v}_i - \lambda_i\vector{v}_i = \vector{0}} } }{\iff} &\begin{pmatrix} (A-\lambda_1 I)\vector{v}_1 = \vector{0} \\ \vdots \\ (A-\lambda_n I)\vector{v}_n = \vector{0} \\ \end{pmatrix} \end{array}$$

Notice on the right-hand side, after subtracting and factoring the vector $\vector{v}_i$, we have a form that is equivalent to finding the kernel of $A-\lambda_n I$.

To begin, let’s take a simple $2\times2$ matrix: $A=\begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix}$.

Typically, you’d be asked to:

1. Find eigenvalues of $A$
2. For each eigenvalues $\lambda$, find the basis for the respective eigenspaces
3. Diagonalize $A$

1. Find eigenvalues of $A$

We need to find $\lambda$ such that $A\vector{v}=\lambda\vector{v}$ and $\vector{v}\neq\vector{0}$.

$$A\vector{v} = \lambda\vector{v} \\ \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} \vector{v} = \begin{pmatrix} \lambda & 0 \\ 0 & \lambda \end{pmatrix} \vector{v}$$

Then, we subtract and factor $\vector{v}$ to obtain $(A-\lambda I)\vector{v}$.

$$\begin{align*} (A-\lambda I)\vector{v} &= \left( \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} - \lambda\begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \right)\vector{v} &= \vector{0} \\ &= \left( \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} - \begin{pmatrix} \lambda & 0 \\ 0 & \lambda \end{pmatrix} \right)\vector{v} &= \vector{0} \\ &= \begin{pmatrix} 1-\lambda & 2 \\ 2 & 1-\lambda \end{pmatrix} \vector{v} &= \vector{0} \end{align*}$$

By definition, $\lambda$ are eigenvalues $\iff\ker\begin{pmatrix} 1-\lambda & 2 \\ 2 & 1-\lambda \end{pmatrix}\neq\set{\vector{0}}$.

So… how the fuck do we solve for $\lambda$? Well, we could do Gaussian. However, it would be kinda nasty because we would need to make the diagonal $1$… but $\lambda$ is on the diagonal.

Instead, for a $2\times2$ matrix, we can use the fact that

$$\begin{pmatrix} a & b \\ c & d \end{pmatrix}^{-1} = \frac{1}{ad-bc}\begin{pmatrix} d & -b \\ -c & a \end{pmatrix},\qquad ad-bc\neq0$$

So, for our matrix (again, by definition), $\lambda$ is an eigenvalue if and only if:

$$\det\begin{pmatrix} 1-\lambda & 2 \\ 2 & 1-\lambda \end{pmatrix}=0 \iff \ker\begin{pmatrix} 1-\lambda & 2 \\ 2 & 1-\lambda \end{pmatrix}\neq\set{\vector{0}}$$

As such:

$$\begin{align*} \det\begin{pmatrix} 1-\lambda & 2 \\ 2 & 1-\lambda \end{pmatrix} &= (1-\lambda)(1-\lambda)-2\cdot2 &= 0\\ &= 1-2\lambda+\lambda^2-4 &= 0 \\ &= \lambda^2 - 2\lambda - 3 &= 0 \\ &= (\lambda + 1)(\lambda - 3) &= 0 \end{align*} \\ \therefore\lambda=-1,3$$

The eigenvalues for the matrix $\begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix}$ are $-1$ and $3$.

2. For each eigenvalues $\lambda$, find the basis for the respective eigenspaces

From the last part, we have two eigenvalues $\lambda=-1,3$.

First, take $\lambda=-1$. The eigenspace is given by:

$$\begin{align*} \ker(A-\lambda I) &= \ker(A(-1)I) \\ &= \ker(A+I) \\ &= \ker\left( \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} + \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \right) \\ &= \ker\begin{pmatrix} 2 & 2 \\ 2 & 2 \end{pmatrix} \end{align*}$$

Then, to find the kernel, we just Gaussian that bitch.

$$\operatorname{rref}\begin{pmatrix} 2 & 2 \\ 2 & 2 \end{pmatrix} = \begin{pmatrix} 1 & 1 \\ 0 & 0 \end{pmatrix} \\ \therefore x = -y \\ y\in\R \\[1em] \therefore\ker(A+I) = \Set{ y\begin{pmatrix} -1 \\ 1 \end{pmatrix}: y\in\R }$$

Now, do the same for $\lambda=3$.

$$\begin{align*} \ker(A-\lambda I) &= \ker(A-3I) \\ &= \ker\left( \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} - 3\begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \right) \\ &= \ker\left( \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} - \begin{pmatrix} 3 & 0 \\ 0 & 3 \end{pmatrix} \right) \\ &= \ker\begin{pmatrix} -2 & 2 \\ 2 & -2 \end{pmatrix} \end{align*}$$

And, again Gaussian to find the kernel.

$$\operatorname{rref}\begin{pmatrix} -2 & 2 \\ 2 & -2 \end{pmatrix} = \begin{pmatrix} 1 & -1 \\ 0 & 0 \end{pmatrix} \\ \therefore x = y \\ y\in\R \\[1em] \therefore\ker(A+I) = \Set{ y\begin{pmatrix} 1 \\ 1 \end{pmatrix}: y\in\R }$$

So, we have that the basis for $\lambda=-1$ is $\Set{\begin{pmatrix} -1 \\ 1 \end{pmatrix}}$ and $\lambda=3$ is $\Set{\begin{pmatrix} 1 \\ 1 \end{pmatrix}}$.

Then, we say that the basis for the eigenspace is $\Set{\begin{pmatrix} -1 \\ 1 \end{pmatrix}, \begin{pmatrix} 1 \\ 1 \end{pmatrix}}$. The vectors in that basis are called eigenvectors.

To illustrate this, take the eigenvectors: $\vector{v}_1=\begin{pmatrix} -1 \\ 1 \end{pmatrix},\vector{v}_2=\begin{pmatrix} 1 \\ 1 \end{pmatrix}$ and transform it using $A$.

Recall that:

$$A\vector{v}_1 = \lambda_1\vector{v}_1 \\ \vdots \\ A\vector{v}_n = \lambda_n\vector{v}_n$$

So, to transform each of our vectors, we simply plug in our eigenvalues to get:

$$A\vector{v}_1 = \lambda_1\vector{v}_1 = -1\begin{pmatrix} -1 \\ 1 \end{pmatrix} = \begin{pmatrix} 1 \\ -1 \end{pmatrix} \\ A\vector{v}_2 = \lambda_2\vector{v}_2 = 3\begin{pmatrix} 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 3 \\ 3 \end{pmatrix}$$

$$\begin{array}{ccc} \includegraphics[height=150px]{../assets/notes_wk7_eigenspace_1.svg} & \raisebox{75px}{$\xrightarrow{A}$} & \includegraphics[height=150px]{../assets/notes_wk7_eigenspace_2.svg} \end{array}$$

And sure enough, transforming the regular way using matrix multiplication would also give us the same results for each vectors.

$$\begin{align*} A\vector{v}_1 = \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} \begin{pmatrix} -1 \\ 1 \end{pmatrix} &= \begin{pmatrix} 1 \\ -1 \end{pmatrix} \\ &= -1\begin{pmatrix} -1 \\ 1 \end{pmatrix} \\ &= \lambda_1\vector{v}_1 \\ A\vector{v}_2 = \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} \begin{pmatrix} 1 \\ 1 \end{pmatrix} &= \begin{pmatrix} 3 \\ 3 \end{pmatrix} \\ &= 3\begin{pmatrix} 1 \\ 1 \end{pmatrix} \\ &= \lambda_2\vector{v}_2 \end{align*}$$

3. Diagonalize $A$

In general, we what is something in the form:

$$P^{-1}AP = \begin{pmatrix} \lambda_1 & & 0 \\ & \ddots & \\ 0 & & \lambda_n \end{pmatrix}$$

$P^{-1}AP$ is called the diagonalization of $A$.

From our previous steps, we found that $A$ has two eigenvalues $\lambda_1=-1,\lambda_2=3$.

Then, the diagonalization of $A$ is

$$P^{-1}AP = \begin{pmatrix} \lambda_1 & 0 \\ 0 & \lambda_2 \end{pmatrix} = \begin{pmatrix} -1 & 0 \\ 0 & 3 \end{pmatrix}$$

where $P$ is the matrix made up of the eigenvectors $\vector{v}_1, \vector{v}_2$.

With the diagonalization of $A$, we can now find $A^k$ for any $k\in\N$.

$$P^{-1}A^k P = \begin{pmatrix} -1 & 0 \\ 0 & 3 \end{pmatrix} \\ \therefore A^k = P\begin{pmatrix} -1 & 0 \\ 0 & 3 \end{pmatrix} P^{-1}$$

For $3\times3$ matrices

Now, what about for a $3\times3$ matrices? For example $A=\begin{pmatrix} 1 & 1 & 0 \\ 1 & 2 & 1 \\ 0 & 0 & 3 \end{pmatrix}$.

We start with finding eigenvalues of $A$ by solving for $\det(A-\lambda I) = 0$. Which means, we’ll need to find the determinant for a $3\times3$ matrix.

Then, we’d solve for

$$\det(A-\lambda I) = 0 \\ \det\begin{pmatrix} 1-\lambda & 1 & 0 \\ 1 & 2-\lambda & 1 \\ 0 & 0 & 3-\lambda \end{pmatrix} = 0$$

to find the eigenvalues… and so on.

Notice that for a $2\times2$ matrix, solving for the $\lambda$ yields a polynomial expression of degree two i.e., $a\lambda^2 + b\lambda + c = 0$. And so, you’d have at most two eigenvalues.

That means that, here, for a $3\times3$ matrix, there will be at most three eigenvalues. And for $n\times n$ matrix, there’d be at most $n$ eigenvalues.

For $4\times4$ matrices and beyond…

Well, you’d need to find the determinant for a $4\times4$ matrix. Which, at this point, you may as well kill yourself.