Lecture 22

[[lecture-data]]

2024-10-21

Exam Debrief

5/6 problems graded - results soon

make sure to read all of the comments even if you got full credit for things

Some systemic issues

uni-directional language in if and only if proofs

Another exam coming soon

SVD and Courant-Fisher
in general, not comprehensive but still sharp on the basics
midterm 3 in the last week of classes seemed popular

7. Chapter 7

Recall that we will have $A \in M_{m, n}$ and $B \in M_{m, n}$ and we have a unique $B$ such that

$A B A = A$
$A B$ is hermitian
$B A B = B$
$B A$ is hermitian

If we let $A = U Σ V^{*}$ be its singular value decomposition, then the Moore-Penrose inverse is given as $A^{†} = V Σ^{†} U^{*}$ . We can see this by seeing

$A = \sum_{i = 1}^{rank (A)} σ_{i} u_{i} v_{i}^{*}$
$A^{†} = \sum_{i = 1}^{rank A} \frac{1}{σ_{i}} v_{i} u_{i}^{*}$
And note that ${A^{†}}^{†} = A$
The moore-penrose inverse is a 1-2-3-4 generalized inverse.

Note

If $A \in M_{m, n}$ is full-column rank (ie, $A$ is "tall" and the columns are each linearly independent). Then

A^{†} = (A^{*} A)^{- 1} A^{*}

Note

This is like the regression setting :)

If $A$ is full row rank (ie $A$ is "fat" and the rows are each linearly independent), then

A^{†} = A^{*} (A A^{*})^{- 1}

Proof

Say $A = U Σ V^{*}$ is full column rank. Then

A^{*} A = V Σ^{*} U^{*} U Σ V^{*} = V Σ^{2} V^{*}

And all $n$ values in $Σ^{2}$ are nonzero and real since $A$ is full column rank. Thus $A^{*} A$ is invertible.

Now, check for 1-2-3-4 generalized inverse conditions with $(A^{*} A)^{- 1} A^{*}$ . (All conditions are satisfied trivially).

The proof for the full row rank case follows analogously.

(see full column (or row) rank matrices have an easy psuedoinverse)

~ end of material for midterm 2 ~

5. Chapter 5

Here, we deal with a vector space $V$ over a field $K$ (either $R$ or $C$ in this case, and we will have statements for both)

Inner Product

An inner product on a vector space $V$ over field $K$ is a function $⟨, ⟩ : V \times V \to K$ such that for all $x, y, z \in V$ and all $c \in K$ we have

$⟨ x, x ⟩$ is real, nonnegative. Also, $⟨ x, x ⟩ = 0 ⟺ x = 0$
$⟨ x + y, z ⟩ = ⟨ x, z ⟩ + ⟨ y, z ⟩$
$⟨ c x, z ⟩ = c ⟨ x, z ⟩$ (2 and 3 indicate that this is linear)
$⟨ x, z ⟩ = \bar{⟨ z, x ⟩}$ (ie symmetric)

If all of these hold, then $V$ is an inner product space.

Example

$C^{n}$ over $C$ .

Euclidean inner product: $⟨ x, z ⟩ := z^{*} x$
some $A \in M_{n}$ positive definite: $⟨ x, z ⟩_{A} := z^{*} A x$
Note that the euclidean inner product is a special case of the second, where $A$ is the identity!

(see inner product)

Note

$\forall x, y, z, w \in V$ and $a, b, c, d \in K$ then

$⟨ x, c z ⟩ = \overset{―}{⟨ c z, x ⟩} = \bar{c} \overset{―}{⟨ z, x ⟩} = \bar{c} ⟨ x, z ⟩$
$⟨ x, y + z ⟩ = \overset{―}{⟨ y, x ⟩} + \overset{―}{⟨ z, x ⟩} = ⟨ x, y ⟩ + ⟨ x, z ⟩$
$⟨ a x + b y, c z + d w ⟩ = a \bar{c} ⟨ x, z ⟩ + a \bar{d} ⟨ x, w ⟩ + b \bar{c} ⟨ y, z ⟩ + b \bar{d} ⟨ y, w ⟩$

And we must show these, because we began only with the four axioms we had given in the definition of the inner product

(see additivity of inner products)

Norm

Let $V$ be a vector space over field $K$ . Then a norm on $V$ is a function $| | \cdot | | : V \to R \geq 0$ such that for all $x, y \in V, c \in K$

$| | x | | = 0 ⟺ x = 0$ ("idea of length of a vector" so no length only if no length! this is called positivity)
$| | c x | | = | c | \times | | x | |$ (homogeneity)
$| | x + y | | \leq | | x | | + | | y | |$ (triangle inequality)

Spaced where these hold are called normed linear spaces (they are the same as vector spaces)

(see norm)

L^{p}

norm

For $K^{n}$ over $K$ the $L^{p}$ norm for $p \in Z$ is defined as

| | x | |_{p} = {(\sum_{i = 1}^{n} | x_{i} |^{p})}^{1 / p}

When $p = 1$ this is the "manhattan norm"
When $p = 2$ this is the euclidean norm
$| | x | |_{\infty} = max (x_{i})$

(see L-p norm)