Lecture 26

[[lecture-data]]

2024-11-01

Readings

5. Chapter 5

Recall the definition of continuity for linear functions and that linear functions with finite dimensional domain are continuous.

Note

This means we can think of matrices both as

vectors in matrix-vector space
as analytic objects (as functions)

Notation

Let $A \in M_{m, n}$ . We can view " $A$ " $: C^{n}, | | \cdot | |_{p} \to C^{m}, | | \cdot | |_{q}$ . Then the operator norm

| | A | |_{p, q} = max_{x \in C^{n} \neq 0} = \frac{| | A x | |_{q}}{| | x | |_{p}} = max_{z \in C^{n}, | | z | |_{p} = 1} | | A x | |_{q}

since the unit sphere is compact (closed and bounded is equivalent for vector spaces) and $A$ is continuous (and the norm is also continuous), the function will obtain its maximum.

Example

Let $A \in M_{m, n}$ . Then the $| | A | |_{2, 2}$ is the greatest singular vlaue of $A$ .

To see this, consider $A = U Σ V^{*}$ . Then since $U, V$ are unitary we have that

| | A x | |_{2, 2} = max_{x \neq 0} \frac{| | A x | |_{2}^{2}}{| | x | |_{2}} = max_{x \neq 0} \frac{x^{*} A^{*} A x}{x^{*} x} = λ_{max} (A^{*} A)

And $λ_{max} (A^{*} A) = λ_{max} (Σ^{*} Σ) = σ_{max}^{2}$

(see)

Proposition

Let $A \in M_{m, n}$ . Then

| | A | |_{1, 1} = max_{j} \sum_{i = 1}^{m} | a_{i j} | $ $ (i e, t h e m a x i m u m c o l u m n s u m) a n d $ $ | | A | |_{\infty, \infty} = m a x_{i} \sum_{j = 1}^{n} | a_{i j} | $ $ (i e t h e m a x i m u m r o w s u m) > [! p r o o f] > L e t $ x \neq 0 \in C^{n} $ . C o n s i d e r $ | | A x | |_{1} $ (t h e " M a n h a t t a n n o r m " b e c a u s e w e " w a l k a l o n g t h e b l o c k s ") . W e h a v e $ $ \begin{aligned} > | | A x | |_{1} & = \sum_{i = 1}^{m} | \sum_{j = 1}^{n} a_{i j} x_{j} | \\ > & \leq \sum_{i = 1}^{m} \sum_{j = 1}^{n} | a_{i j} | | x_{j} | \\ > & = \sum_{j = 1}^{n} | x_{j} | \sum_{i = 1}^{m} | a_{i j} | \\ > & \leq | | x | |_{1} [max_{j} \sum_{i = 1}^{m} (| a_{i j} |)] > \end{aligned}

Let $\hat{j}$ be the index for the maximum column sum. Then $| | e_{\hat{j}} | |_{1} = 1$ and $| | A e_{\hat{j}} | |_{1} = max_{j} \sum_{i = 1}^{m} | a_{i j} |$ .

Thus $max_{x \in C^{n} \neq 0} \frac{| | A x | |_{1}}{| | x | |_{1}} = max_{j} \sum_{i = 1}^{m} | a_{i j} |$

(see matrix 1 norm is max column sum)

Proposition

Let $V, | | \cdot | |$ be a finite dimensional NLS with basis $B$ . Now, consider $V, V^{*}$ as vector spaces. (recall the dual space of a vector space).

Dual Norm

Let $| | \cdot | |$ be a norm on $K^{n}$ . The dual norm $| | \cdot | |^{D}$ on $K^{n}$ is defined as : for all $y \in K^{n}$ ,

\lvert \lvert y \rvert \rvert {#D} = \max_{x \in K^n, \lvert \lvert x \rvert \rvert =1} \lvert y^*x \rvert = \max_{x \in K^n \neq 0} \frac{\lvert y^*x \rvert}{\lvert \lvert x \rvert \rvert }

Note

It is called the dual norm, because it is the norm on the dual space!

(see dual norm)

Note

Let $| | \cdot | |$ be a norm on $K^{n}$ . Then for all $x, y \in K^{n}$ we have

\lvert y^*x \rvert \leq \lvert \lvert x \rvert \rvert \cdot \lvert \lvert y \rvert \rvert {#D}

And for all $x, y \in K$ , we have
$| y^{*} x | \leq | | x | |_{1} | | y | |_{\infty}$
ie the dual of the 1 norm is the infinity norm

see dual norm

These are "holder-like" inequalities

Theorem

The 1 norm is the dual norm of the infinity norm (and vice versa):

Proof

$\begin{aligned} | y^{*} x | & = | \sum_{i = 1}^{n} \overset{―}{y_{i}} x_{i} | \\ \leq \sum_{i = 1}^{n} | y_{i} | | x_{i} | \\ \leq \sum_{i = 1}^{n} | | y | |_{\infty} | x_{i} | \\ = | | x | |_{1} | | y | |_{\infty} \end{aligned}$

(see the dual of the 1 norm is the infinity norm)

Proposition

On $K^{n}$ , we have $| | \cdot | |_{1}^{D} = | | \cdot | |_{\infty}$ and $| | \cdot | |_{2}^{2} = | | \cdot | |_{2}^{2}$

Proof

By definition, the dual norm is $max_{x \in K^{n}, | | x | | = 1} | y^{*} x |$ .

Given $y \in K^{n}$ m if we restrict to $x : | | x | |_{1} = 1$ , then by definition of the dual, we have $| y^{*} x | \leq max_{| | x | |_{1} = 1} | y^{*} x | = | | y | |_{\infty}$ . Equality holds exactly when $x$ is all zeros, except for a $1$ in the position corresponding to $\arg max_{i} y_{i}$ .

Given an $x \in K^{n}$ . If we restrict to the $y : | | y | |_{\infty} = 1$ , then $| y^{*} x | \leq | | x | |_{1}$ . Equality holds exactly when we choose an $y$ such that each entry is some unit $e^{i θ}$ so that all components of $y^{*} x$ are real.

Given $y \in K^{n}$ . If we restrict to $x : | | x | |_{2} = 1$ , then $| y^{*} x | \leq | | y | |_{2}$ . Equality holds for (unit length $ℓ_{2}$ ) when $x = \frac{1}{| | y | |_{2}} y$ , and an exactly analogous argument occurs for the reverse.