Lecture 26

[[lecture-data]]

2024-11-01

Readings

• a

5. Chapter 5

Recall the definition of continuity for linear functions and that linear functions with finite dimensional domain are continuous.

Note

This means we can think of matrices both as

• vectors in matrix-vector space
• as analytic objects (as functions)

Notation

Let $A\in M_{m,n}$. We can view "$A$"$:{\mathbb{C}}^n,||\cdot |{|}_p\to {\mathbb{C}}^m,||\cdot |{|}_q$. Then the operator norm $||A|{|}_{p,q}={\max }_{x\in {\mathbb{C}}^n\ne 0}=\frac{||Ax|{|}_q}{||x|{|}_p}={\max }_{z\in {\mathbb{C}}^n,||z|{|}_p=1}||Ax|{|}_q$

• since the unit sphere is compact (closed and bounded is equivalent for vector spaces) and $A$ is continuous (and the norm is also continuous), the function will obtain its maximum.

$A\in M_{m,n}$. Then the $||A|{|}_{2,2}$ is the greatest singular vlaue of $A$.

Let

To see this, consider $A=U\Sigma V^{\ast }$. Then since $U,V$ are unitary we have that $||Ax|{|}_{2,2}={\max }_{x\ne 0}\frac{||Ax|{|}_2^2}{||x|{|}_2}={\max }_{x\ne 0}\frac{x^{\ast }{A}^{\ast }Ax}{x^{\ast }x}={\lambda }_{\max }(A^{\ast }A)$ And ${\lambda }_{\max }(A^{\ast }A)={\lambda }_{\max }({\Sigma }^{\ast }\Sigma )={\sigma }_{\max }^2$

(see)

Proposition

Let $A\in M_{m,n}$. Then $||A|{|}_{1,1}={\max }_j{\sum }_{i=1}^m|a_{ij}|$ (ie, the maximum column sum) and $||A|{|}_{\infty ,\infty }=ma{x}_i{\sum }_{j=1}^n|a_{ij}|$ (ie the maximum row sum)

$x\ne 0\in {\mathbb{C}}^n$. Consider $||Ax|{|}_1$ (the "Manhattan norm" because we "walk along the blocks"). We have $\begin{array}{rl}||Ax|{|}_1& =\sum _{i=1}^m\left|\sum _{j=1}^n{a}_{ij}{x}_j\right|\\ & \le \sum _{i=1}^m\sum _{j=1}^n|a_{ij}||x_j|\\ & =\sum _{j=1}^n|x_j|\sum _{i=1}^m|a_{ij}|\\ & \le ||x|{|}_1[\underset{j}{\max }\sum _{i=1}^m(|a_{ij}|)]\end{array}$ Let $\hat{j}$ be the index for the maximum column sum. Then $||e_{\hat{j}}|{|}_1=1$ and $||A{e}_{\hat{j}}|{|}_1={\max }_j{\sum }_{i=1}^m|a_{ij}|$.

Let

Thus ${\max }_{x\in {\mathbb{C}}^n\ne 0}\frac{||Ax|{|}_1}{||x|{|}_1}={\max }_j{\sum }_{i=1}^m|a_{ij}|$

(see matrix 1 norm is max column sum)

Proposition

Let $V,||\cdot ||$ be a finite dimensional NLS with basis $B$. Now, consider $V,V^{\ast }$ as vector spaces. (recall the dual space of a vector space).

Dual Norm

Let $||\cdot ||$ be a norm on $K^n$. The dual norm $||\cdot |{|}^D$ on $K^n$ is defined as : for all $y\in K^n$, $||y|{|}^D={\max }_{x\in K^n,||x||=1}|y^{\ast }x|={\max }_{x\in K^n\ne 0}\frac{|y^{\ast }x|}{||x||}$

dual space!

It is called the dual norm, because it is the norm on the

(see dual norm)

Note

Let $||\cdot ||$ be a norm on $K^n$. Then for all $x,y\in K^n$ we have $|y^{\ast }x|\le ||x||\cdot ||y|{|}^D$ And for all $x,y\in K$, we have $|y^{\ast }x|\le ||x|{|}_1||y|{|}_{\infty }$ ie the dual of the 1 norm is the infinity norm

• see dual norm
• These are “holder-like” inequalities

Theorem

The 1 norm is the dual norm of the infinity norm (and vice versa):

Proof

\lvert y^*x \rvert &= \left\lvert \sum_{i=1}^n \overline{y_{i}}x_{i} \right\rvert \\ &\leq \sum_{i=1}^n \lvert y_{i} \rvert \lvert x_{i} \rvert \\ &\leq \sum_{i=1}^n \lvert \lvert y \rvert \rvert _{\infty} \lvert x_{i} \rvert \\ &= \lvert \lvert x \rvert \rvert _{1} \lvert \lvert y \rvert \rvert _{\infty} \end{aligned}$$

(see the dual of the 1 norm is the infinity norm)

Proposition

On $K^n$, we have $||\cdot |{|}_1^D=||\cdot |{|}_{\infty }$ and $||\cdot |{|}_2^2=||\cdot |{|}_2^2$

dual norm is ${\max }_{x\in K^n,||x||=1}|y^{\ast }x|$.

By definition, the

Given $y\in K^n$m if we restrict to $x:||x|{|}_1=1$, then by definition of the dual, we have $|y^{\ast }x|\le {\max }_{||x|{|}_1=1}|y^{\ast }x|=||y|{|}_{\infty }$. Equality holds exactly when $x$ is all zeros, except for a $1$ in the position corresponding to $\arg {\max }_i{y}_i$.

Given an $x\in K^n$. If we restrict to the $y:||y|{|}_{\infty }=1$, then $|y^{\ast }x|\le ||x|{|}_1$ . Equality holds exactly when we choose an $y$ such that each entry is some unit $e^{i\theta }$ so that all components of $y^{\ast }x$ are real.

Given $y\in K^n$. If we restrict to $x:||x|{|}_2=1$, then $|y^{\ast }x|\le ||y|{|}_2$. Equality holds for (unit length ${\ell }_2$) when $x=\frac{1}{||y|{|}_2}y$, and an exactly analogous argument occurs for the reverse.