Lecture 22

[[lecture-data]]

2024-10-21

Exam Debrief

5/6 problems graded - results soon

• make sure to read all of the comments even if you got full credit for things

Some systemic issues

• uni-directional language in if and only if proofs

Another exam coming soon

• SVD and Courant-Fisher
• in general, not comprehensive but still sharp on the basics
• midterm 3 in the last week of classes seemed popular

7. Chapter 7

Recall that we will have $A\in M_{m,n}$ and $B\in M_{m,n}$ and we have a unique $B$ such that

• $ABA=A$
• $AB$ is hermitian
• $BAB=B$
• $BA$ is hermitian

If we let $A=U\Sigma V^{\ast }$ be its singular value decomposition, then the Moore-Penrose inverse is given as $A^{†}=V{\Sigma }^{†}{U}^{\ast }$. We can see this by seeing

• $A={\sum }_{i=1}^{\text{rank}(A)}{\sigma }_i{u}_i{v}_i^{\ast }$
• $A^{†}={\sum }_{i=1}^{\text{rank}A}\frac{1}{{\sigma }_i}{v}_i{u}_i^{\ast }$ And note that ${A^{†}}^{†}=A$
• The moore-penrose inverse is a 1-2-3-4 generalized inverse.

Note

If $A\in M_{m,n}$ is full-column rank (ie, $A$ is “tall” and the columns are each linearly independent). Then $A^{†}=(A^{\ast }A{)}^{-1}{A}^{\ast }$

Note

This is like the regression setting :)

If $A$ is full row rank (ie $A$ is “fat” and the rows are each linearly independent), then $A^{†}=A^{\ast }(A{A}^{\ast }{)}^{-1}$

$A=U\Sigma V^{\ast }$ is full column rank. Then $A^{\ast }A=V{\Sigma }^{\ast }{U}^{\ast }U\Sigma V^{\ast }=V{\Sigma }^2{V}^{\ast }$ And all $n$ values in ${\Sigma }^2$ are nonzero and real since $A$ is full column rank. Thus $A^{\ast }A$ is invertible.

Say

Now, check for 1-2-3-4 generalized inverse conditions with $(A^{\ast }A{)}^{-1}{A}^{\ast }$. (All conditions are satisfied trivially).

The proof for the full row rank case follows analogously.

(see full column (or row) rank matrices have an easy psuedoinverse)

~ end of material for midterm 2 ~

5. Chapter 5

Here, we deal with a vector space $V$ over a field $K$ (either $\mathbb{R}$ or $\mathbb{C}$ in this case, and we will have statements for both)

Inner Product

An inner product on a vector space $V$ over field $K$ is a function $⟨,⟩:V\times V\to K$ such that for all $x,y,z\in V$ and all $c\in K$ we have

1. $⟨x,x⟩$ is real, nonnegative. Also, $⟨x,x⟩=0⟺x=0$

2. $⟨x+y,z⟩=⟨x,z⟩+⟨y,z⟩$

3. $⟨cx,z⟩=c⟨x,z⟩$ (2 and 3 indicate that this is linear)

4. $⟨x,z⟩=\overline{⟨z,x⟩}$ (ie symmetric)

If all of these hold, then $V$ is an inner product space.

${\mathbb{C}}^n$ over $\mathbb{C}$.

• Euclidean inner product: $⟨x,z⟩:=z^{\ast }x$
• some $A\in M_n$ positive definite: $⟨x,z{⟩}_A:=z^{\ast }Ax$ Note that the euclidean inner product is a special case of the second, where $A$ is the identity!

(see inner product)

Note

$\forall x,y,z,w\in V$ and $a,b,c,d\in K$ then

1. $⟨x,cz⟩=\overline{⟨cz,x⟩}=\bar{c}\overline{⟨z,x⟩}=\bar{c}⟨x,z⟩$
2. $⟨x,y+z⟩=\overline{⟨y,x⟩}+\overline{⟨z,x⟩}=⟨x,y⟩+⟨x,z⟩$
3. $⟨ax+by,cz+dw⟩=a\bar{c}⟨x,z⟩+a\bar{d}⟨x,w⟩+b\bar{c}⟨y,z⟩+b\bar{d}⟨y,w⟩$

And we must show these, because we began only with the four axioms we had given in the definition of the inner product

(see additivity of inner products)

Norm

Let $V$ be a vector space over field $K$. Then a norm on $V$ is a function $||\cdot ||:V\to \mathbb{R}\ge 0$ such that for all $x,y\in V,c\in K$

1. $||x||=0⟺x=0$ (“idea of length of a vector” so no length only if no length! this is called positivity)
2. $||cx||=|c|\times ||x||$(homogeneity)
3. $||x+y||\le ||x||+||y||$ (triangle inequality)

Spaced where these hold are called normed linear spaces (they are the same as vector spaces)

(see norm)

$L^p$ norm

For $K^n$ over $K$ the $L^p$ norm for $p\in \mathbb{Z}$ is defined as $||x|{|}_p={\left({\sum }_{i=1}^n|x_i{|}^p\right)}^{1/p}$

• When $p=1$ this is the “manhattan norm”
• When $p=2$ this is the euclidean norm
• $||x|{|}_{\infty }=\max (x_i)$

(see L-p norm)