Homework 10

1. Consider R4\R^4. Let u=[1234]\mathbf{u} = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4\end{bmatrix} and v=[5678]\mathbf{v} = \begin{bmatrix} 5 \\ 6 \\ 7 \\ 8\end{bmatrix}. Find the basis for the orthogonal complement of W=span⁡{ u,v }W = \operatorname{span}\set{\mathbf{u}, \mathbf{v}}.

Let A=[∣∣uv∣∣]A = \begin{bmatrix} | & | \\ \mathbf{u} & \mathbf{v} \\ | & | \end{bmatrix}. Then, W⊥=ker⁡A⊤=ker⁡[12345678]W^\perp = \ker A^\top = \ker\begin{bmatrix} 1 & 2 & 3 & 4 \\ 5 & 6 & 7 & 8 \end{bmatrix}.

rref⁡[12345678]=[10−1−20123]∴y=−2z−3wx=z+2wz,w∈R\operatorname{rref}\begin{bmatrix} 1 & 2 & 3 & 4 \\ 5 & 6 & 7 & 8 \end{bmatrix} = \begin{bmatrix} 1 & 0 & -1 & -2 \\ 0 & 1 & 2 & 3 \end{bmatrix} \\[1em] \therefore y = -2z-3w \\ x = z + 2w \\ z,w\in\R

As such,

W⊥={ [z+2w−2z−3wzw]:z,w∈R }=span⁡{ [1−210],[2−301] }.W^\perp = \Set{ \begin{bmatrix} z + 2w \\ -2z-3w \\ z \\ w \end{bmatrix} : z,w \in\R } = \operatorname{span}\Set{ \begin{bmatrix} 1 \\ -2 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 2 \\ -3 \\ 0 \\ 1 \end{bmatrix} }.

2. Find the orthogonal projection of the vector (1,1,1)(1, 1, 1) onto the subspace defined by the equations {x+y+z=0,x−y−2z=0,\begin{cases}x + y + z = 0, \\x − y − 2z = 0, \\\end{cases}

Let WW be the subspace defined by the above equation.

rref⁡[1111−1−2]=[10−120132]∴y=−32zx=12zz∈R∴W={ [12z−32zz]:z∈R }=span⁡{ [12−321] }\operatorname{rref}\begin{bmatrix} 1 & 1 & 1 \\ 1 & -1 & -2 \end{bmatrix} = \begin{bmatrix} 1 & 0 & -\frac{1}{2} \\[.5em] 0 & 1 & \frac{3}{2} \end{bmatrix} \\[1em] \therefore y = -\frac{3}{2}z \\[.5em] x = \frac{1}{2}z \\[.5em] z\in\R \\[.5em] \therefore W = \Set{ \begin{bmatrix} \frac{1}{2}z \\[.5em] -\frac{3}{2}z \\[.5em] z \end{bmatrix}: z\in\R } = \operatorname{span}\Set{ \begin{bmatrix} \frac{1}{2} \\[.5em] -\frac{3}{2} \\[.5em] 1 \end{bmatrix} }

Let x⃗=[111]\vector{x}=\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix} and v⃗=[12−321]\vector{v}=\begin{bmatrix} \frac{1}{2} \\[.5em] -\frac{3}{2} \\[.5em] 1 \end{bmatrix}. Then, the orthogonal projection of x⃗=[111]\vector{x}=\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix} is given by

proj⁡W(x⃗)=⟨x⃗,v⃗⟩∣∣v⃗∣∣2v⃗=⟨[111],[12−321]⟩∣∣[12−321]∣∣2[12−321]=12−32+114+94+1[12−321]=0⃗.\begin{align*} \operatorname{proj}_W(\vector{x}) = \frac{\langle\vector{x},\vector{v}\rangle}{||\vector{v}||^2}\vector{v} &= \frac{\left\langle\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix},\begin{bmatrix} \frac{1}{2} \\[.5em] -\frac{3}{2} \\[.5em] 1 \end{bmatrix}\right\rangle} {\left|\left|\begin{bmatrix} \frac{1}{2} \\[.5em] -\frac{3}{2} \\[.5em] 1 \end{bmatrix}\right|\right|^2} \begin{bmatrix} \frac{1}{2} \\[.5em] -\frac{3}{2} \\[.5em] 1 \end{bmatrix} \\ &= \frac{\frac{1}{2}-\frac{3}{2}+1} {\frac{1}{4}+\frac{9}{4}+1}\begin{bmatrix} \frac{1}{2} \\[.5em] -\frac{3}{2} \\[.5em] 1 \end{bmatrix} \\ &= \vector{0}. \end{align*}

3. Find the orthogonal basis of R3\R^3 with [110]\begin{bmatrix} 1 \\ 1 \\ 0\end{bmatrix} as one of the vectors. Hint: You can use Gram-Schmidt process on a basis with [110]\begin{bmatrix} 1 \\ 1 \\ 0\end{bmatrix} as the first vector. e.g. { [110],[010],[001] }.\Set{ \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}}.

Let w⃗1=[110]\vector{w}_1 = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}, w⃗2=[010]\vector{w}_2 = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, and w⃗3=[001]\vector{w}_3 = \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}.

Let v⃗1=w⃗1=[110]\vector{v}_1 = \vector{w}_1 = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}. Then, using the Gram–Schmidt process, the orthogonal vectors v⃗2\vector{v}_2 and v⃗3\vector{v}_3 are given by the following.

v⃗2=w⃗2−proj⁡v1⃗(w⃗2)=w⃗2−⟨w⃗2,v⃗1⟩∣∣v⃗1∣∣2v⃗1=[010]−⟨[010],[110]⟩∣∣[110]∣∣2[110]=[010]−0(1)+1(1)+0(0)12+12+02[110]=[−12120]v⃗3=w⃗3−proj⁡v⃗1(w⃗3)−proj⁡v⃗2(w⃗3)=w⃗3−⟨w⃗3,v⃗1⟩∣∣v⃗1∣∣2v⃗1−⟨w⃗3,v⃗2⟩∣∣v⃗2∣∣2v⃗2=[001]−⟨[001],[110]⟩∣∣[110]∣∣2[110]−⟨[001],[−12120]⟩∣∣[−12120]∣∣2[−12120]=[001]−0[110]−0[−12120]=[001]\def<{\left\langle} \def>{\right\rangle} \def\norm#1{\left|\left|#1\right|\right|} \begin{align*} \vector{v}_2 &= \vector{w}_2 - \operatorname{proj}_{\vector{v_1}}(\vector{w}_2) \\ &= \vector{w}_2 - \frac{<\vector{w}_2,\vector{v}_1>}{||\vector{v}_1||^2}\vector{v}_1 \\ &= \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} - \frac{<\begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}>}{\norm{\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}}^2}\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} \\ &= \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} - \frac{0(1) + 1(1) + 0(0)}{1^2 + 1^2 + 0^2} \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} \\ &= \begin{bmatrix} -\frac{1}{2} \\ \frac{1}{2} \\ 0 \end{bmatrix} \\[3em] \vector{v}_3 &= \vector{w}_3 - \operatorname{proj}_{\vector{v}_1}(\vector{w}_3) - \operatorname{proj}_{\vector{v}_2}(\vector{w}_3) \\ &= \vector{w}_3 - \frac{<\vector{w}_3, \vector{v}_1>}{\norm{\vector{v}_1}^2}\vector{v}_1 - \frac{<\vector{w}_3, \vector{v}_2>}{\norm{\vector{v}_2}^2}\vector{v}_2 \\ &= \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} - \frac{<\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}, \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}>}{\norm{\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}}^2}\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} - \frac{<\begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}, \begin{bmatrix} -\frac{1}{2} \\ \frac{1}{2} \\ 0 \end{bmatrix}>}{\norm{\begin{bmatrix} -\frac{1}{2} \\ \frac{1}{2} \\ 0 \end{bmatrix}}^2}\begin{bmatrix} -\frac{1}{2} \\ \frac{1}{2} \\ 0 \end{bmatrix} \\ &= \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} - 0\begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} - 0\begin{bmatrix} -\frac{1}{2} \\ \frac{1}{2} \\ 0 \end{bmatrix} \\ &= \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} \end{align*}

As such, an orthogonal basis of R3\R^3 is { [110],[−12120],[001] }\Set{ \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} -\frac{1}{2} \\ \frac{1}{2} \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix} }.

4.

(a) Find an orthonormal basis for the kernel of the following matrix. A=[210−132−1−1].A = \begin{bmatrix}2 & 1 & 0 & −1 \\3 & 2 & −1 & −1\end{bmatrix}.

rref⁡(A)=[101−101−21]∴y=2z−wx=−z+wker⁡(A)={ [−z+w2z−wzw]:z,w∈R }=span⁡{ [−1210],[1−101] }\operatorname{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & -1 \\ 0 & 1 & -2 & 1 \end{bmatrix} \\ \therefore y = 2z-w \\ x = -z+w \\ \ker(A) = \Set{ \begin{bmatrix} -z+w \\ 2z-w \\ z \\ w \end{bmatrix}: z,w\in\R } = \operatorname{span}\Set{ \begin{bmatrix} -1 \\ 2 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 1 \\ -1 \\ 0 \\ 1 \end{bmatrix} }

Let u⃗1=[−1210],u⃗2=[1−101]\vector{u}_1 = \begin{bmatrix} -1 \\ 2 \\ 1 \\ 0 \end{bmatrix}, \vector{u}_2 = \begin{bmatrix} 1 \\ -1 \\ 0 \\ 1 \end{bmatrix}. Then, { u⃗1,u⃗2 }\set{\vector{u}_1, \vector{u}_2} is a basis of ker⁡(A)\ker(A) (and are linearly independent).

To produce an orthogonal basis, we apply the Gram–Schmidt process. Let v⃗1=u⃗1=[−1210]\vector{v}_1 = \vector{u}_1 = \begin{bmatrix} -1 \\ 2 \\ 1 \\ 0 \end{bmatrix}. Then,

v⃗2=u⃗2−proj⁡v⃗1(u⃗2)=u⃗2−⟨u⃗2,v⃗1⟩∣∣v⃗1∣∣2v⃗1=[1−101]−⟨[1−101],[−1210]⟩∣∣[−1210]∣∣2[−1210]=[1−101]−1(−1)+(−1)2+0(1)+1(0)(−1)2+22+12+02[−1210]=[120121].\def<{\left\langle} \def>{\right\rangle} \def\norm#1{\left|\left|#1\right|\right|} \begin{align*} \vector{v}_2 &= \vector{u}_2 - \operatorname{proj}_{\vector{v}_1}(\vector{u}_2) \\ &= \vector{u}_2 - \frac{<\vector{u}_2,\vector{v}_1>}{||\vector{v}_1||^2}\vector{v}_1 \\ &= \begin{bmatrix} 1 \\ -1 \\ 0 \\ 1 \end{bmatrix} - \frac{<\begin{bmatrix} 1 \\ -1 \\ 0 \\ 1 \end{bmatrix}, \begin{bmatrix} -1 \\ 2 \\ 1 \\ 0 \end{bmatrix}>}{\norm{\begin{bmatrix} -1 \\ 2 \\ 1 \\ 0 \end{bmatrix}}^2}\begin{bmatrix} -1 \\ 2 \\ 1 \\ 0 \end{bmatrix} \\ &= \begin{bmatrix} 1 \\ -1 \\ 0 \\ 1 \end{bmatrix} - \frac{1(-1)+(-1)2+0(1)+1(0)}{(-1)^2 + 2^2 + 1^2 + 0^2}\begin{bmatrix} -1 \\ 2 \\ 1 \\ 0 \end{bmatrix} \\ &= \begin{bmatrix} \frac{1}{2} \\[.2em] 0 \\[.2em] \frac{1}{2} \\[.2em] 1 \end{bmatrix} \end{align*}.

Hence, { v⃗1,v⃗2 }\set{\vector{v}_1, \vector{v}_2} is an orthogonal basis of ker⁡(A)\ker(A). Finally, an orthonormal basis of ker⁡(A)\ker(A) is

{ 16[−1210],132[120121] }={ [−1623160],[1601623] }.\Set{ \frac{1}{\sqrt{6}}\begin{bmatrix} -1 \\ 2 \\ 1 \\ 0 \end{bmatrix}, \frac{1}{\sqrt{\frac{3}{2}}}\begin{bmatrix} \frac{1}{2} \\[.5em] 0 \\[.5em] \frac{1}{2} \\[.5em] 1 \end{bmatrix} } = \Set{ \begin{bmatrix} -\frac{1}{\sqrt{6}} \\[.5em] \sqrt{\frac{2}{3}} \\[.5em] \frac{1}{\sqrt{6}} \\[.5em] 0 \end{bmatrix}, \begin{bmatrix} \frac{1}{\sqrt{6}} \\[.5em] 0 \\ \frac{1}{\sqrt{6}} \\[.5em] \sqrt{\frac{2}{3}} \end{bmatrix} }.

(b) Find an orthonormal basis for (ker⁡(A))⊥(\ker(A))^\perp, the orthogonal complement of ker⁡(A)\ker(A).

Since (ker⁡(A))⊥=Im⁡(A⊤)=Im⁡[23120−1−1−1](\ker(A))^\perp = \operatorname{Im}(A^\top) = \operatorname{Im}\begin{bmatrix} 2 & 3 \\ 1 & 2 \\ 0 & -1 \\ -1 & -1 \end{bmatrix}.

rref⁡(A⊤)=[10010000]∴Im⁡(A⊤)=span⁡{ [210−1],[32−1−1] }\operatorname{rref}(A^\top) = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 0 & 0 \\ 0 & 0 \end{bmatrix} \\ \therefore \operatorname{Im}(A^\top) = \operatorname{span}\Set{ \begin{bmatrix} 2 \\ 1 \\ 0 \\ -1 \end{bmatrix}, \begin{bmatrix} 3 \\ 2 \\ -1 \\ -1 \end{bmatrix} }

Let u⃗1=[210−1],u⃗2=[32−1−1]\vector{u}_1 = \begin{bmatrix} 2 \\ 1 \\ 0 \\ -1 \end{bmatrix}, \vector{u}_2 = \begin{bmatrix} 3 \\ 2 \\ -1 \\ -1 \end{bmatrix}. { u⃗1,u⃗2 }\set{\vector{u}_1, \vector{u}_2} is a basis of Im⁡(A⊤)\operatorname{Im}(A^\top). Then, we apply Gram–Schmidt to orthogonalize them.

Let v⃗1=u⃗1=[210−1]\vector{v}_1 = \vector{u}_1 = \begin{bmatrix} 2 \\ 1 \\ 0 \\ -1 \end{bmatrix}. Then,

v⃗2=u⃗2−proj⁡v⃗1(u⃗2)=u⃗2−⟨u⃗2,v⃗1⟩∣∣v⃗1∣∣2v⃗1=[32−1−1]−⟨[32−1−1],[210−1]⟩∣∣[210−1]∣∣2[210−1]=[32−1−1]−3(2)+2(1)+(−1)(0)+(−1)(−1)22+12+02+(−1)2[210−1]=[012−112].\def<{\left\langle} \def>{\right\rangle} \def\norm#1{\left|\left|#1\right|\right|} \begin{align*} \vector{v}_2 &= \vector{u}_2 - \operatorname{proj}_{\vector{v}_1}(\vector{u}_2) \\ &= \vector{u}_2 - \frac{<\vector{u}_2,\vector{v}_1>}{||\vector{v}_1||^2}\vector{v}_1 \\ &= \begin{bmatrix} 3 \\ 2 \\ -1 \\ -1 \end{bmatrix} - \frac{<\begin{bmatrix} 3 \\ 2 \\ -1 \\ -1 \end{bmatrix}, \begin{bmatrix} 2 \\ 1 \\ 0 \\ -1 \end{bmatrix}>}{\norm{\begin{bmatrix} 2 \\ 1 \\ 0 \\ -1 \end{bmatrix}}^2}\begin{bmatrix} 2 \\ 1 \\ 0 \\ -1 \end{bmatrix} \\ &= \begin{bmatrix} 3 \\ 2 \\ -1 \\ -1 \end{bmatrix} - \frac{3(2)+2(1)+(-1)(0)+(-1)(-1)}{2^2+1^2+0^2+(-1)^2}\begin{bmatrix} 2 \\ 1 \\ 0 \\ -1 \end{bmatrix} \\ &= \begin{bmatrix} 0 \\ \frac{1}{2} \\ -1 \\ \frac{1}{2} \end{bmatrix}. \end{align*}

Then, { v⃗1,v⃗2 }\set{\vector{v}_1, \vector{v}_2} is an orthogonal basis of (ker⁡(A))⊥(\ker(A))^\perp. Finally, an orthonormal basis of (ker⁡(A))⊥(\ker(A))^\perp is

{ 16[210−1],132[012−112] }={ [23160−16],[016−2316] }.\Set{ \frac{1}{\sqrt{6}}\begin{bmatrix} 2 \\ 1 \\ 0 \\ -1 \end{bmatrix}, \frac{1}{\sqrt{\frac{3}{2}}}\begin{bmatrix} 0 \\ \frac{1}{2} \\ -1 \\ \frac{1}{2} \end{bmatrix} } = \Set{ \begin{bmatrix} \sqrt{\frac{2}{3}} \\ \frac{1}{\sqrt{6}} \\ 0 \\ -\frac{1}{\sqrt{6}} \end{bmatrix}, \begin{bmatrix} 0 \\ \frac{1}{\sqrt{6}} \\ -\sqrt{\frac{2}{3}} \\ \frac{1}{\sqrt{6}} \end{bmatrix} }.

(c) Does the orthonormal basis in (i) combined with the orthonormal basis in (ii)for an orthonormal basis for R4\R^4? Explain.

The union of the orthonormal bases for AA and A⊤A^\top found in parts (i) and (ii) is

{ 16[−1210],132[120121],16[210−1],132[012−112] }.\Set{ \frac{1}{\sqrt{6}}\begin{bmatrix} -1 \\ 2 \\ 1 \\ 0 \end{bmatrix}, \frac{1}{\sqrt{\frac{3}{2}}}\begin{bmatrix} \frac{1}{2} \\[.5em] 0 \\[.5em] \frac{1}{2} \\[.5em] 1 \end{bmatrix}, \frac{1}{\sqrt{6}}\begin{bmatrix} 2 \\ 1 \\ 0 \\ -1 \end{bmatrix}, \frac{1}{\sqrt{\frac{3}{2}}}\begin{bmatrix} 0 \\ \frac{1}{2} \\ -1 \\ \frac{1}{2} \end{bmatrix} }.

Placing them as column vectors in a matrix:

rref⁡[−1616230230161616160−23023−1616]=[1000010000100001]\operatorname{rref}\begin{bmatrix} -\frac{1}{\sqrt{6}} & \frac{1}{\sqrt{6}}& \sqrt{\frac{2}{3}} & 0 \\ \sqrt{\frac{2}{3}} & 0 & \frac{1}{\sqrt{6}} & \frac{1}{\sqrt{6}} \\ \frac{1}{\sqrt{6}} & \frac{1}{\sqrt{6}} & 0 & -\sqrt{\frac{2}{3}} \\ 0 & \sqrt{\frac{2}{3}} & -\frac{1}{\sqrt{6}} & \frac{1}{\sqrt{6}} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}

We find that the reduced-row echelon form is full rank. As such, these vectors span R4\R^4.

Then, we need to check if they are mutually orthogonal to determine if they are an orthonormal basis of R4\R^4. By checking all pairs in the set, we find that they are orthogonal.

As such, the abovementioned set is an orthonormal basis of R4\R^4.

5. Consider the following subspace of R4\R^4 V=span⁡{ [1111],[1001],[021−1] }V = \operatorname{span}\Set{ \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}, \begin{bmatrix} 1 \\ 0 \\ 0 \\ 1 \end{bmatrix}, \begin{bmatrix} 0 \\ 2 \\ 1 \\ -1 \end{bmatrix}}

(i) What is the dimension of VV?

rref⁡[11010210111−1]=[100010001000]\operatorname{rref}\begin{bmatrix} 1 & 1 & 0 \\ 1 & 0 & 2 \\ 1 & 0 & 1 \\ 1 & 1 & -1 \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{bmatrix}

A matrix composed of the three vectors has a full column rank. As such, they form a basis of VV and thus dim⁡V=3\dim V = 3.

(ii) Using Gram-Schmidt Process, find an orthogonal basis for VV.

Let u⃗1=[1111]\vector{u}_1 = \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}, u⃗2=[1001]\vector{u}_2 = \begin{bmatrix} 1 \\ 0 \\ 0 \\ 1 \end{bmatrix}, and u⃗3=[021−1]\vector{u}_3 = \begin{bmatrix} 0 \\ 2 \\ 1 \\ -1 \end{bmatrix}. As shown in (i), { u⃗1,u⃗2,u⃗3 }\set{\vector{u}_1, \vector{u}_2, \vector{u}_3} is a basis of VV.

Now let v⃗1=u⃗1=[1111]\vector{v}_1 = \vector{u}_1 = \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}. Then, applying Gram–Schmidt:

v⃗2=u⃗2−proj⁡v⃗1(u⃗2)=u⃗2−⟨u⃗2,v⃗1⟩∣∣v⃗1∣∣2v⃗1=[1001]−⟨[1001],[1111]⟩∣∣[1111]∣∣2[1111]=[1001]−1(1)+0(1)+0(1)+0(1)+1(1)12+12+12+12[1111]=[12−12−1212]v⃗3=u⃗3−proj⁡v⃗1(u⃗3)−proj⁡v⃗2(u⃗3)=u⃗3−⟨u⃗3,v⃗1⟩∣∣v⃗1∣∣2v⃗1−⟨u⃗3,v⃗2⟩∣∣v⃗2∣∣2v⃗2=[021−1]−⟨[021−1],[1111]⟩∣∣[1111]∣∣2[1111]−⟨[021−1],[12−12−1212]⟩∣∣[12−12−1212]∣∣2[12−12−1212]=[021−1]−0(1)+2(1)+1(1)+(−1)(1)12+12+12+12[1111]−0(12)+2(−12)+1(−12)+(−1)(12)(12)2+(−12)2+(−12)2+(12)2[12−12−1212]=[1212−12−12]\def<{\left\langle} \def>{\right\rangle} \def\norm#1{\left|\left|#1\right|\right|} \begin{align*} \vector{v}_2 &= \vector{u}_2 - \operatorname{proj}_{\vector{v}_1}(\vector{u}_2) \\ &= \vector{u}_2 - \frac{<\vector{u}_2,\vector{v}_1>}{||\vector{v}_1||^2}\vector{v}_1 \\ &= \begin{bmatrix} 1 \\ 0 \\ 0 \\ 1 \end{bmatrix} - \frac{<\begin{bmatrix} 1 \\ 0 \\ 0 \\ 1 \end{bmatrix},\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}>}{\norm{\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}}^2}\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix} \\ &= \begin{bmatrix} 1 \\ 0 \\ 0 \\ 1 \end{bmatrix} - \frac{1(1)+0(1)+0(1)+0(1)+1(1)}{1^2+1^2+1^2+1^2}\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix} \\ &= \begin{bmatrix} \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] \frac{1}{2} \end{bmatrix} \\[3em] \vector{v}_3 &= \vector{u}_3 - \operatorname{proj}_{\vector{v}_1}(\vector{u}_3) - \operatorname{proj}_{\vector{v}_2}(\vector{u}_3) \\ &= \vector{u}_3 - \frac{<\vector{u}_3, \vector{v}_1>}{\norm{\vector{v}_1}^2}\vector{v}_1 - \frac{<\vector{u}_3, \vector{v}_2>}{\norm{\vector{v}_2}^2}\vector{v}_2 \\ &= \begin{bmatrix} 0 \\ 2 \\ 1 \\ -1 \end{bmatrix} - \frac{<\begin{bmatrix} 0 \\ 2 \\ 1 \\ -1 \end{bmatrix}, \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}>}{\norm{\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}}^2}\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix} - \frac{<\begin{bmatrix} 0 \\ 2 \\ 1 \\ -1 \end{bmatrix}, \begin{bmatrix} \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] \frac{1}{2} \end{bmatrix}>}{\norm{\begin{bmatrix} \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] \frac{1}{2} \end{bmatrix}}^2}\begin{bmatrix} \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] \frac{1}{2} \end{bmatrix} \\ &= \begin{bmatrix} 0 \\ 2 \\ 1 \\ -1 \end{bmatrix} - \frac{0(1)+2(1)+1(1)+(-1)(1)}{1^2+1^2+1^2+1^2}\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix} - \frac{0(\frac{1}{2})+2(-\frac{1}{2})+1(-\frac{1}{2})+(-1)(\frac{1}{2})}{(\frac{1}{2})^2 + (-\frac{1}{2})^2 + (-\frac{1}{2})^2 + (\frac{1}{2})^2}\begin{bmatrix} \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] \frac{1}{2} \end{bmatrix} \\ &= \begin{bmatrix} \frac{1}{2} \\[.3em] \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \end{bmatrix} \end{align*}

And so, an orthogonal basis of VV is { [1111],[12−12−1212],[1212−12−12] }\Set{\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}, \begin{bmatrix} \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] \frac{1}{2} \end{bmatrix},\begin{bmatrix} \frac{1}{2} \\[.3em] \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \end{bmatrix} }.

(iii) Find the orthogonal projection of [1234]\begin{bmatrix} 1 \\ 2 \\ 3 \\ 4\end{bmatrix} to VV.

From (ii), { [1111],[12−12−1212],[1212−12−12] }\Set{\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}, \begin{bmatrix} \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] \frac{1}{2} \end{bmatrix},\begin{bmatrix} \frac{1}{2} \\[.3em] \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \end{bmatrix} } is an orthogonal basis of VV. As such,

proj⁡V[1234]=⟨[1234],[1111]⟩∣∣[1111]∣∣2[1111]+⟨[1234],[12−12−1212]⟩∣∣[12−12−1212]∣∣2[12−12−1212]+⟨[1234],[1212−12−12]⟩∣∣[1212−12−12]∣∣2[1212−12−12]=1(1)+2(1)+3(1)+4(1)12+12+12+12[1111]+0[12−12−1212]+1(12)+2(12)+3(−12)+4(−12)(12)2+(12)2+(−12)2+(−12)2[1212−12−12]=12[3377].\def<{\left\langle} \def>{\right\rangle} \def\norm#1{\left|\left|#1\right|\right|} \begin{align*} \operatorname{proj}_V \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \end{bmatrix} &= \frac{<\begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \end{bmatrix}, \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}>}{\norm{\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}}^2}\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix} + \frac{<\begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \end{bmatrix}, \begin{bmatrix} \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] \frac{1}{2} \end{bmatrix}>}{\norm{\begin{bmatrix} \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] \frac{1}{2} \end{bmatrix}}^2}\begin{bmatrix} \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] \frac{1}{2} \end{bmatrix} + \frac{<\begin{bmatrix} 1 \\ 2 \\ 3 \\ 4 \end{bmatrix}, \begin{bmatrix} \frac{1}{2} \\[.3em] \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \end{bmatrix}>}{\norm{\begin{bmatrix} \frac{1}{2} \\[.3em] \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \end{bmatrix}}^2}\begin{bmatrix} \frac{1}{2} \\[.3em] \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \end{bmatrix} \\ &= \frac{1(1)+2(1)+3(1)+4(1)}{1^2+1^2+1^2+1^2}\begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix} + 0 \begin{bmatrix} \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] \frac{1}{2} \end{bmatrix} + \frac{1(\frac{1}{2})+2(\frac{1}{2})+3(-\frac{1}{2})+4(-\frac{1}{2})}{(\frac{1}{2})^2+(\frac{1}{2})^2+(-\frac{1}{2})^2+(-\frac{1}{2})^2}\begin{bmatrix} \frac{1}{2} \\[.3em] \frac{1}{2} \\[.3em] -\frac{1}{2} \\[.3em] -\frac{1}{2} \end{bmatrix} \\ &= \frac{1}{2}\begin{bmatrix} 3 \\ 3 \\ 7 \\ 7 \end{bmatrix}. \end{align*}

6.

(i) Find the least square solution of the following system [234−21520]x=[2−113]\begin{bmatrix}2 & 3 \\4 & −2 \\1 & 5 \\2 & 0\end{bmatrix}\mathbf{x} =\begin{bmatrix}2 \\−1 \\1 \\3\end{bmatrix}

Let A=[234−21520]A = \begin{bmatrix}2 & 3 \\4 & −2 \\1 & 5 \\2 & 0\end{bmatrix} and b⃗=[2−113]\vector{b}=\begin{bmatrix}2 \\−1 \\1 \\3\end{bmatrix}. Then, A⊤=[24123−250]A^\top = \begin{bmatrix} 2 & 4 & 1 & 2 \\ 3 & -2 & 5 & 0 \end{bmatrix}.

Since A⊤A=[24123−250][234−21520]=[253338]A^\top A = \begin{bmatrix} 2 & 4 & 1 & 2 \\ 3 & -2 & 5 & 0 \end{bmatrix}\begin{bmatrix}2 & 3 \\4 & −2 \\1 & 5 \\2 & 0\end{bmatrix} = \begin{bmatrix} 25 & 3 \\ 3 & 38 \end{bmatrix} and its inverse exists. Then, the least square solution x^\mathbf{\hat{x}} can be derived by applying A⊤A^\top to both sides.

A⊤Ax^=A⊤b⃗∴x^=(A⊤A)−1A⊤b⃗=[253338]−1[24123−250][2−113]=1941[227304]A^\top A\mathbf{\hat{x}} = A^\top\vector{b} \\[1em] \begin{align*} \therefore \mathbf{\hat{x}} &= (A^\top A)^{-1} A^\top\vector{b} \\ &= \begin{bmatrix} 25 & 3 \\ 3 & 38 \end{bmatrix}^{-1}\begin{bmatrix} 2 & 4 & 1 & 2 \\ 3 & -2 & 5 & 0 \end{bmatrix} \begin{bmatrix}2 \\−1 \\1 \\3\end{bmatrix} \\ &= \frac{1}{941}\begin{bmatrix} 227 \\ 304 \end{bmatrix} \end{align*}

(ii) Find the orthogonal projection of bb onto the image of AA using the least square solution.

From (i), where A=[234−21520]A = \begin{bmatrix}2 & 3 \\4 & −2 \\1 & 5 \\2 & 0\end{bmatrix} and x^=1941[227304]\mathbf{\hat{x}}= \displaystyle\frac{1}{941}\begin{bmatrix} 227 \\ 304 \end{bmatrix}. Then,

b⃗=Ax^=[234−21520]1941[227304]=1941[13663001747454].\vector{b} = A\mathbf{\hat{x}} = \begin{bmatrix} 2 & 3 \\ 4 & −2 \\ 1 & 5 \\2 & 0 \end{bmatrix} \frac{1}{941} \begin{bmatrix} 227 \\ 304 \end{bmatrix} = \frac{1}{941} \begin{bmatrix} 1366 \\ 300 \\ 1747 \\ 454 \end{bmatrix}.

7. Find the least square fitting straight line y=C+Dty = C + Dt given the following set of data. ti−2013yi0125\begin{array}{|c|c|c|c|c|}\hline t_i & -2 & 0 & 1 & 3 \\\hline y_i & 0 & 1 & 2 & 5\\\hline\end{array}

Using the equation of a straight line, we have the following system of equation:

{0=C+D(−2)1=C+D(0)2=C+D(1)5=C+D(3)  ⟺  [0125]=[1−2101113][CD]\begin{cases} 0 &= C+D(-2) \\ 1 &= C+D(0) \\ 2 &= C+D(1) \\ 5 &= C+D(3) \end{cases} \iff \begin{bmatrix} 0 \\ 1 \\ 2 \\ 5 \end{bmatrix} = \begin{bmatrix} 1 & -2 \\ 1 & 0 \\ 1 & 1 \\ 1 & 3 \end{bmatrix}\begin{bmatrix} C \\ D \end{bmatrix}

Let b⃗=[0125]\vector{b} = \begin{bmatrix} 0 \\ 1 \\ 2 \\ 5 \end{bmatrix}, A=[1−2101113]A = \begin{bmatrix} 1 & -2 \\ 1 & 0 \\ 1 & 1 \\ 1 & 3 \end{bmatrix}, and x⃗=[CD]\vector{x} = \begin{bmatrix} C \\ D \end{bmatrix}. Here, we want to find x⃗\vector{x} such that Ax⃗=b⃗A\vector{x} = \vector{b}, will produce an inconsistent solution. Instead, we find a least square solution for x^=[C^D^]\mathbf{\hat{x}}=\begin{bmatrix} \hat{C} \\ \hat{D} \end{bmatrix} by applying A⊤A^\top to both sides, such that

A⊤Ax^=A⊤b⃗.A^\top A\mathbf{\hat{x}} = A^\top\vector{b}.

As such, we have:

[1111−2013][1−2101113][C^D^]=[1111−2013][0125][42214][C^D^]=[817]∴[C^D^]=[42214]−1[817]=[321]\begin{bmatrix} 1 & 1 & 1 & 1 \\ -2 & 0 & 1 & 3 \end{bmatrix} \begin{bmatrix} 1 & -2 \\ 1 & 0 \\ 1 & 1 \\ 1 & 3 \end{bmatrix} \begin{bmatrix} \hat{C} \\ \hat{D} \end{bmatrix} = \begin{bmatrix} 1 & 1 & 1 & 1 \\ -2 & 0 & 1 & 3 \end{bmatrix} \begin{bmatrix} 0 \\ 1 \\ 2 \\ 5 \end{bmatrix} \\ \begin{bmatrix} 4 & 2 \\ 2 & 14 \end{bmatrix} \begin{bmatrix} \hat{C} \\ \hat{D} \end{bmatrix} = \begin{bmatrix} 8 \\ 17 \end{bmatrix} \\ \therefore \begin{bmatrix} \hat{C} \\ \hat{D} \end{bmatrix} = \begin{bmatrix} 4 & 2 \\ 2 & 14 \end{bmatrix}^{-1} \begin{bmatrix} 8 \\ 17 \end{bmatrix} = \begin{bmatrix} \frac{3}{2} \\[.3em] 1 \end{bmatrix}

Therefore, our line of best fit is given by the equation y=32+ty=\frac{3}{2}+t.

8. Consider the non-standard inner product on R2\R^2. ⟨u,v⟩=[u1u2][1225][v1v2]\langle\mathbf{u},\mathbf{v}\rangle = \begin{bmatrix} u_1 & u_2 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5\end{bmatrix} \begin{bmatrix} v_1 \\ v_2\end{bmatrix}

(a) Verify this is an inner product of R2\R^2.

First, notice that this definition results in a 1×11\times1 matrix. For u⃗,v⃗∈R2\vector{u},\vector{v}\in\R^2,

⟨u⃗,v⃗⟩=[u1u2][1225][v1v2]=[v1(u1+2u2)+v2(2u1+5u2)]\begin{align*} \langle\vector{u},\vector{v}\rangle &= \begin{bmatrix} u_1 & u_2 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5 \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} \\ &= \begin{bmatrix}v_1(u_1+2u_2) + v_2(2u_1+5u_2)\end{bmatrix} \end{align*}

Symmetry and bilinearity should be quiet obvious since we can just apply commutative, associative, and distributive properties of addition and multiplication here.

But for the sake of completion, consider u⃗,v⃗,w⃗∈R2\vector{u},\vector{v},\vector{w}\in\R^2 and c,d∈Rc,d\in\R.

Symmetry

⟨u⃗,v⃗⟩=[u1u2][1225][v1v2]=[v1(u1+2u2)+v2(2u1+5u2)]⟨v⃗,u⃗⟩=[v1v2][1225][u1u2]=[u1(v1+2v2)+u2(2v1+5v2)]\begin{align*} \langle\vector{u},\vector{v}\rangle &= \begin{bmatrix} u_1 & u_2 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5 \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} \\ &= \begin{bmatrix}v_1(u_1+2u_2) + v_2(2u_1+5u_2)\end{bmatrix} \\ \langle\vector{v},\vector{u}\rangle &= \begin{bmatrix} v_1 & v_2 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5 \end{bmatrix} \begin{bmatrix} u_1 \\ u_2 \end{bmatrix} \\ &= \begin{bmatrix}u_1(v_1+2v_2) + u_2(2v_1+5v_2)\end{bmatrix} \end{align*}

And indeed,

v1(u1+2u2)+v2(2u1+5u2)=u1(v1+2v2)+u2(2v1+5v2)v_1(u_1+2u_2) + v_2(2u_1+5u_2) = u_1(v_1+2v_2) + u_2(2v_1+5v_2)

if you expand each of the term. Hence, ⟨u⃗,v⃗⟩=⟨v⃗,u⃗⟩\langle\vector{u},\vector{v}\rangle=\langle\vector{v},\vector{u}\rangle.

Bilinearity

⟨cu⃗,v⃗⟩=[cu1cu2][1225][v1v2]=[v1(cu1+2cu2)+v2(2cu1+5cu2)]=c[u1u2][1225][v1v2]=[cv1(u1+2u2)+cv2(2u1+5u2)]=c⟨u⃗,v⃗⟩=[c(v1(u1+2u2)+v2(2u1+5u2))]\begin{align*} \langle c\vector{u},\vector{v}\rangle &= \begin{bmatrix} cu_1 & cu_2 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5 \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} &&= \begin{bmatrix}v_1(cu_1+2cu_2) + v_2(2cu_1+5cu_2)\end{bmatrix} \\ &= c\begin{bmatrix} u_1 & u_2 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5 \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} &&= \begin{bmatrix}cv_1(u_1+2u_2) + cv_2(2u_1+5u_2)\end{bmatrix} \\ &= c \langle \vector{u},\vector{v}\rangle &&= \begin{bmatrix}c(v_1(u_1+2u_2) + v_2(2u_1+5u_2))\end{bmatrix} \end{align*}

Hence, ⟨cu⃗,v⃗⟩=c⟨u⃗,v⃗⟩\langle c\vector{u},\vector{v}\rangle = c \langle \vector{u},\vector{v}\rangle.

⟨u⃗,v⃗⟩=[u1u2][1225][v1v2]=[v1(u1+2u2)+v2(2u1+5u2)]⟨w⃗,v⃗⟩=[w1w2][1225][v1v2]=[v1(w1+2w2)+v2(2w1+5w2)]⟨u⃗+w⃗,v⃗⟩=[u1+w1u2+w2][1225][v1v2]=[v1(u1+w1+2(u2+w2))+v2(2(u1+w1)+5(u2+w2))]\begin{align*} \langle\vector{u},\vector{v}\rangle &= \begin{bmatrix} u_1 & u_2 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5 \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} \\ &= \begin{bmatrix}v_1(u_1+2u_2) + v_2(2u_1+5u_2)\end{bmatrix} \\ \langle\vector{w},\vector{v}\rangle &= \begin{bmatrix} w_1 & w_2 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5 \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} \\ &= \begin{bmatrix}v_1(w_1+2w_2) + v_2(2w_1+5w_2)\end{bmatrix} \\ \langle \vector{u} + \vector{w}, \vector{v}\rangle &= \begin{bmatrix} u_1+w_1 & u_2+w_2 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5 \end{bmatrix} \begin{bmatrix} v_1 \\ v_2 \end{bmatrix} \\ &= \begin{bmatrix}v_1(u_1+w_1+2(u_2+w_2)) + v_2(2(u_1+w_1) + 5(u_2+w_2))\end{bmatrix} \end{align*}

By inspection, combining the first term in ⟨u⃗,v⃗⟩\langle\vector{u},\vector{v}\rangle and ⟨w⃗,v⃗⟩\langle\vector{w},\vector{v}\rangle together produces the first term in ⟨u⃗+w⃗,v⃗⟩\langle\vector{u}+\vector{w},\vector{v}\rangle. And the same applies for the second term.

⟨u⃗,v⃗⟩+⟨w⃗,v⃗⟩=[v1(u1+2u2)+v2(2u1+5u2)+v1(w1+2w2)+v2(2w1+5w2)]=[v1(u1+2u2)+v1(w1+2w2)+v2(2u1+5u2)+v2(2w1+5w2)]=[v1(u1+w1+2(u2+w2))+v2(2(u1+w1)+5(u2+w2))]=⟨u⃗+w⃗,v⃗⟩\begin{align*} \langle\vector{u},\vector{v}\rangle + \langle\vector{w},\vector{v}\rangle &= &&\big[ v_1(u_1+2u_2) + v_2(2u_1+5u_2) + v_1(w_1+2w_2) + v_2(2w_1+5w_2) &\big] \\ &= &&\big[ v_1(u_1+2u_2) + v_1(w_1+2w_2) + v_2(2u_1+5u_2) + v_2(2w_1+5w_2) &\big] \\ &= &&\big[ v_1(u_1+w_1+2(u_2+w_2)) + v_2(2(u_1+w_1) + 5(u_2+w_2)) &\big] \\ &= &&\langle \vector{u} + \vector{w}, \vector{v}\rangle \end{align*}

Hence, ⟨u⃗,v⃗⟩+⟨w⃗,v⃗⟩=⟨u⃗+w⃗,v⃗⟩\langle\vector{u},\vector{v}\rangle + \langle\vector{w},\vector{v}\rangle=\langle \vector{u} + \vector{w}, \vector{v}\rangle. As such, this inner product satisfies bilinearity.

Positive-definite

⟨u⃗,u⃗⟩=[u1u2][1225][u1u2]=[u1(u1+2u2)+u2(2u1+5u2)]=[u12+5u22+4u1u2]\begin{align*} \langle\vector{u},\vector{u}\rangle &= \begin{bmatrix} u_1 & u_2 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5 \end{bmatrix} \begin{bmatrix} u_1 \\ u_2 \end{bmatrix} \\ &= \begin{bmatrix}u_1(u_1+2u_2) + u_2(2u_1+5u_2)\end{bmatrix} \\ &= \begin{bmatrix}u_1^2 + 5u_2^2 + 4u_1u_2\end{bmatrix} \end{align*}

Notice that ⟨u⃗,u⃗⟩\langle\vector{u},\vector{u}\rangle is positive for all real u1,u2u_1, u_2 and that ⟨u⃗,u⃗⟩=0  ⟺  u1=u2=0\langle\vector{u},\vector{u}\rangle = 0 \iff u_1 = u_2 = 0. As such, ⟨u⃗,u⃗⟩=0  ⟺  u⃗=0⃗\langle\vector{u},\vector{u}\rangle=0 \iff \vector{u}=\vector{0} and ⟨u⃗,u⃗⟩≥0\langle\vector{u},\vector{u}\rangle\ge 0 for all u⃗∈R2\vector{u}\in\R^2.

Thus, satisfying the properties of an inner product space.

(b) Using Gram-Schmidt process, starting with the vectors [10]\begin{bmatrix} 1 \\ 0\end{bmatrix} and [01]\begin{bmatrix} 0 \\ 1\end{bmatrix}, find an orthogonal basis under this inner product.

For this part, we will interpret the output of the inner product (the 1×11\times1 matrix) as a scalar. Otherwise, we will not be able to define an orthogonal basis because division is not generally defined as a matrix operation.

Let v⃗1=e⃗1=[10]\vector{v}_1 = \vector{e}_1 = \begin{bmatrix} 1 \\ 0\end{bmatrix} and e⃗2=[01]\vector{e}_2 = \begin{bmatrix} 0 \\ 1\end{bmatrix}. Then,

v⃗2=u⃗2−proj⁡v⃗1(e⃗2)=e⃗2−⟨e⃗2,v⃗1⟩∣∣v⃗1∣∣2v⃗1=[01]−⟨[01],[10]⟩∣∣[10]∣∣2[10]=[01]−[01][1225][10][10][1225][10][10]=[01]−21[10]=[−21]\def<{\left\langle} \def>{\right\rangle} \def\norm#1{\left|\left|#1\right|\right|} \begin{align*} \vector{v}_2 &= \vector{u}_2 - \operatorname{proj}_{\vector{v}_1}(\vector{e}_2) \\ &= \vector{e}_2 - \frac{<\vector{e}_2,\vector{v}_1>}{||\vector{v}_1||^2}\vector{v}_1 \\ &= \begin{bmatrix} 0 \\ 1\end{bmatrix} - \frac{<\begin{bmatrix} 0 \\ 1\end{bmatrix},\begin{bmatrix} 1 \\ 0\end{bmatrix}>}{\norm{\begin{bmatrix} 1 \\ 0\end{bmatrix}}^2}\begin{bmatrix} 1 \\ 0\end{bmatrix} \\ &= \begin{bmatrix} 0 \\ 1\end{bmatrix} - \frac{ \begin{bmatrix} 0 & 1 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5 \end{bmatrix} \begin{bmatrix} 1 \\ 0 \end{bmatrix} }{ \begin{bmatrix} 1 & 0 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5 \end{bmatrix} \begin{bmatrix} 1 \\ 0 \end{bmatrix} }\begin{bmatrix} 1 \\ 0\end{bmatrix} \\ &= \begin{bmatrix} 0 \\ 1\end{bmatrix} - \frac{2}{1}\begin{bmatrix} 1 \\ 0\end{bmatrix} \\ &= \begin{bmatrix} -2 \\ 1 \end{bmatrix} \end{align*}

And so, an orthogonal basis under this inner product is { [10],[−21] }\Set{ \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \begin{bmatrix} -2 \\ 1\end{bmatrix} }.

9. Consider the space of all continuous functions on [0,1][0, 1], C[0,1]C[0, 1] with the standard inner product. ⟨f,g⟩=∫01f(x)g(x) ⁣dx\langle f, g\rangle = \int_0^1 f(x)g(x)\d x

(a) Use Gram-Schmidt process. Find an orthogonal basis using span⁡{ x2,x,1 }\operatorname{span}\Set{x^2, x, 1}.

Let U1=x2U_1=x^2, U2=xU_2=x, and U3=1U_3=1.

Take V1=U1=x2V_1 = U_1 = x^2. Then,

V2=U2−proj⁡V1(U2)=U2−⟨U2,V1⟩∣∣V1∣∣2V1=x−⟨x,x2⟩∣∣x2∣∣2x2=x−∫01x⋅x2 ⁣dx∫01x2⋅x2 ⁣dxx2=x−54x2V3=U3−proj⁡V1(U3)−proj⁡V2(U3)=U3−⟨U3,V1⟩∣∣V1∣∣2V1−⟨U3,V2⟩∣∣V2∣∣2V2=1−⟨1,x2⟩∣∣x2∣∣2x2−⟨1,x−54x2⟩∣∣x−54x2∣∣2(x−54x2)=1−∫011⋅x2 ⁣dx∫01(x2)2 ⁣dxx2−∫011⋅(x−54x2) ⁣dx∫01(x−54x2)2 ⁣dx(x−54x2)=10x2−12x3+1\def<{\left\langle} \def>{\right\rangle} \def\norm#1{\left|\left|#1\right|\right|} \begin{align*} \href{https://www.wolframalpha.com/input?i=x+-+%5Cfrac%7B%5Cint_0%5E1+x%5Ccdot+x%5E2+dx%7D%7B%5Cint_0%5E1+x%5E2%5Ccdot+x%5E2+dx%7Dx%5E2+} {V_2} &= U_2 - \operatorname{proj}_{V_1}(U_2) \\ &= U_2 - \frac{<U_2,V_1>}{||V_1||^2}V_1 \\ &= x - \frac{<x,x^2>}{||x^2||^2}x^2 \\ &= x - \frac{\int_0^1 x\cdot x^2 \d x}{\int_0^1 x^2\cdot x^2 \d x}x^2 \\ &= x - \frac{5}{4}x^2 \\[2em] \href{https://www.wolframalpha.com/input?i=1%5C%3A-%5C%3A%5Cfrac%7B%5Cint+_0%5E1%5C%3A1%5Ccdot+%5C%3Ax%5E2dx%7D%7B%5Cint+_0%5E1%5C%3Ax%5E2%5Ccdot+%5C%3Ax%5E2%5C%3Adx%7Dx%5E2%5C%3A-%5Cfrac%7B%5Cint+_0%5E1%5C%3A1%5Ccdot+%5Cleft%28x-%5Cfrac%7B5%7D%7B4%7Dx%5E2%5Cright%29dx%7D%7B%5Cint+_0%5E1%5C%3A%5Cleft%28x-%5Cfrac%7B5%7D%7B4%7Dx%5E2%5Cright%29%5Cleft%28x-%5Cfrac%7B5%7D%7B4%7Dx%5E2%5Cright%29%5C%3Adx%7D%5Cleft%28x-%5Cfrac%7B5%7D%7B4%7Dx%5E2%5Cright%29} {V_3} &= U_3 - \operatorname{proj}_{V_1}(U_3) - \operatorname{proj}_{V_2}(U_3) \\ &= U_3 - \frac{<U_3,V_1>}{||V_1||^2}V_1 - \frac{<U_3,V_2>}{||V_2||^2}V_2 \\ &= 1 - \frac{<1,x^2>}{||x^2||^2}x^2 - \frac{<1,x-\frac{5}{4}x^2>}{||x-\frac{5}{4}x^2||^2}\(x-\frac{5}{4}x^2\) \\ &= 1 - \frac{\int_0^1 1\cdot x^2\d x}{\int_0^1 (x^2)^2 \d x}x^2 - \frac{\int_0^1 1\cdot (x-\frac{5}{4}x^2)\d x}{\int_0^1 (x-\frac{5}{4}x^2)^2 \d x}\(x-\frac{5}{4}x^2\) \\ &= \frac{10x^2 - 12x}{3} + 1 \\ \end{align*}

And so, we have that { x2,x−54x2,10x2−12x3+1 }\displaystyle\Set{ x^2, x - \frac{5}{4}x^2, \frac{10x^2 - 12x}{3} + 1 } is an orthogonal basis of this inner product space.

(b) We prove previously that for any m≠nm\neq n, sin⁡2πmx\sin 2\pi mx and sin⁡2πnx\sin2\pi nx are always mutually orthogonal. Write down the formula of the orthogonal projection of x2x^2 onto the subspace span⁡{ 1,sin⁡(2πx),sin⁡(2π2x) }.\operatorname{span}\set{1,\sin(2\pi x),\sin(2\pi2x)}. Compute it. You need to do integration by part to find the coefficient, but you can use an online integration calculator to find it.

Let W⊂C[0,1]W\sub C[0,1] be a subspace where { 1,sin⁡(2πx),sin⁡(2π2x) }\set{1,\sin(2\pi x),\sin(2\pi2x)} is an orthonormal basis of WW, as shown in the previous homework. Then, the orthogonal projection of x2x^2 on to WW is given by:

proj⁡W(x2)=⟨x2,1⟩⟨1,1⟩1+⟨x2,sin⁡(2πx)⟩⟨sin⁡(2πx),sin⁡(2πx)⟩sin⁡(2πx)+⟨x2,sin⁡(2π2x)⟩⟨sin⁡(2π2x),sin⁡(2π2x)⟩sin⁡(2π2x)=∫01x2⋅1 ⁣dx∫0112 ⁣dx1+∫01x2⋅sin⁡(2πx) ⁣dx∫01sin⁡2(2πx) ⁣dxsin⁡(2πx)+∫01x2⋅sin⁡(2π2x) ⁣dx∫01sin⁡2(2π2x) ⁣dxsin⁡(2π2x)=13−sin⁡(2πx)π−sin⁡(4πx)2π\def<{\left\langle} \def>{\right\rangle} \def\norm#1{\left|\left|#1\right|\right|} \begin{align*} \operatorname{proj}_W(x^2) &= \frac{<x^2, 1>}{<1,1>}1 + \frac{<x^2, \sin(2\pi x)>}{<\sin(2\pi x), \sin(2\pi x)>}\sin(2\pi x) + \frac{<x^2, \sin(2\pi2x)>}{<\sin(2\pi2x),\sin(2\pi2x)>}\sin(2\pi2x) \\ &= \frac{\int_0^1 x^2\cdot 1\d x}{\int_0^1 1^2 \d x}1 + \href{https://www.wolframalpha.com/input?i=%5Cfrac%7B%5Cint+_0%5E1%5C%3Ax%5E2%5Ccdot+%5Csin+%5Cleft%282%5Cpi+%5C%3Ax%5Cright%29%5C%3Adx%7D%7B%5Cint+_0%5E1%5C%3A%5Csin+%5E2%5Cleft%282%5Cpi+%5C%3Ax%5Cright%29%5C%3Adx%7D%5Csin+%5Cleft%282%5Cpi+%5C%3Ax%5Cright%29} {\frac{\int_0^1 x^2\cdot \sin(2\pi x) \d x}{\int_0^1 \sin^2(2\pi x) \d x}\sin(2\pi x)} + \href{https://www.wolframalpha.com/input?i=%5Cfrac%7B%5Cint+_0%5E1%5C%3Ax%5E2%5Ccdot+%5Csin+%5Cleft%282%5Cpi+2x%5Cright%29%5C%3Adx%7D%7B%5Cint+_0%5E1%5C%3A%5Csin+%5E2%5Cleft%282%5Cpi+2x%5Cright%29dx%7D%5Csin+%5Cleft%282%5Cpi+2x%5Cright%29} {\frac{\int_0^1 x^2\cdot\sin(2\pi2x) \d x}{\int_0^1 \sin^2(2\pi2x)\d x}\sin(2\pi2x)} \\ &= \frac{1}{3} - \frac{\sin(2\pi x)}{\pi} - \frac{\sin(4\pi x)}{2\pi} \end{align*}

Homework 10
  1. Consider R4\R^4. Let u=[1234]\mathbf{u} = \begin{bmatrix} 1 \\ 2 \\ 3 \\ 4\end{bmatrix} and v=[5678]\mathbf{v} = \begin{bmatrix} 5 \\ 6 \\ 7 \\ 8\end{bmatrix}. Find the basis for the orthogonal complement of W=span⁡{ u,v }W = \operatorname{span}\set{\mathbf{u}, \mathbf{v}}.
  1. Find the orthogonal projection of the vector (1,1,1)(1, 1, 1) onto the subspace defined by the equations {x+y+z=0,x−y−2z=0,\begin{cases}x + y + z = 0, \\x − y − 2z = 0, \\\end{cases}
  1. Find the orthogonal basis of R3\R^3 with [110]\begin{bmatrix} 1 \\ 1 \\ 0\end{bmatrix} as one of the vectors. Hint: You can use Gram-Schmidt process on a basis with [110]\begin{bmatrix} 1 \\ 1 \\ 0\end{bmatrix} as the first vector. e.g. { [110],[010],[001] }.\Set{ \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 0 \\ 1 \end{bmatrix}}.
  1. Consider the following subspace of R4\R^4 V=span⁡{ [1111],[1001],[021−1] }V = \operatorname{span}\Set{ \begin{bmatrix} 1 \\ 1 \\ 1 \\ 1 \end{bmatrix}, \begin{bmatrix} 1 \\ 0 \\ 0 \\ 1 \end{bmatrix}, \begin{bmatrix} 0 \\ 2 \\ 1 \\ -1 \end{bmatrix}}
  1. Find the least square fitting straight line y=C+Dty = C + Dt given the following set of data. ti−2013yi0125\begin{array}{|c|c|c|c|c|}\hline t_i & -2 & 0 & 1 & 3 \\\hline y_i & 0 & 1 & 2 & 5\\\hline\end{array}
  1. Consider the non-standard inner product on R2\R^2. ⟨u,v⟩=[u1u2][1225][v1v2]\langle\mathbf{u},\mathbf{v}\rangle = \begin{bmatrix} u_1 & u_2 \end{bmatrix}\begin{bmatrix} 1 & 2 \\ 2 & 5\end{bmatrix} \begin{bmatrix} v_1 \\ v_2\end{bmatrix}
  1. Consider the space of all continuous functions on [0,1][0, 1], C[0,1]C[0, 1] with the standard inner product. ⟨f,g⟩=∫01f(x)g(x) ⁣dx\langle f, g\rangle = \int_0^1 f(x)g(x)\d x
≡