Lecture 30

[[lecture-data]]

2024-11-11

Nearing the end of:

5. Chapter 5

Last time, we saw how condition number determines the amount of noise in a linear system. We also saw a similar thing when we are inverting a matrix with some noise thrown in!

Absolute Vector Norm

The vector norm on ${\mathbb{C}}^n$ is called absolute precisely when for all $x\in {\mathbb{C}}^n$ it holds that $|||x|||=||x||$ where $|x|$ is taken component wise

${\ell }_p$ norms are absolute since ${\ell }_p(x)={\left(\sum |x{|}^p\right)}^{1/p}$

All

(see absolute vector norm)

Monotone Vector Norm

We call a vector norm monotone precisely when for all $x,y\in {\mathbb{C}}^n$, $|x|\le |y|⟹||x||\le ||y||$ again, where $|x|$ is taken component wise.

${\ell }_p$ norms are also monotone!

All

(see monotone vector norm)

Theorem

Suppose $||\cdot ||$ is a norm on ${\mathbb{C}}^n$. The following are equivalent:

1. $||\cdot ||$ is monotone

2. $||\cdot ||$ is absolute

3. The matrix norm $||\cdot |{|}^{\prime }$ induced by $||\cdot ||$ satisfies the following: for all diagonal $D\in M_n$ $||D|{|}^{\prime }=\max \{|d_{ii}|\}$ this is called the “funky diagonal property” 😊

Example

• $||\cdot |{|}_{1,1}$ since the matrix 1 norm is max column sum
• $||\cdot |{|}_{2,2}$
• $||\cdot |{|}_{\infty ,\infty }$ since matrix infinity norm is max row sum

Proof

$(1)⟹(2)$ Suppose $||\cdot ||$ is monotone. For all $x\in {\mathbb{C}}^n$, we have that $|x|\le ||x||⟹||x||\le |x|⟹|||x|||=||x||$

$(2)⟹(1)$ Read the book :)

$(1)⟹(3)$ Suppose $||\cdot ||$ is monotone. Let $D\in M_n$ be diagonal. For any $x\in {\mathbb{C}}^n\ne 0$, $Dx=[d_{11}{x}_1{d}_{22}{x}_2\dots d_{nn}{x}_n]$. So $|Dx|\le |{\max }_i|d_{ii}|x|$. So by monotonicity, we have

&= \max_{i}|d_{i i}|\cdot \lvert \lvert x \rvert \rvert \\ \implies \frac{\lvert \lvert Dx \rvert \rvert }{\lvert \lvert x \rvert \rvert } \leq \max_{i} |d_{ᵢ}| \end{aligned}$$ Equality is attained with $x = e_{k}$ the standard basis vector with $1$ in the $k$th position, and where $k$ is the index of the largest $|d_{i,i}|$. Thus we get $\lvert \lvert D \rvert \rvert' := \max\frac{\lvert \lvert Dx \rvert \rvert}{\lvert \lvert x \rvert \rvert} = \max_{i} | d_{i i}|$

$(3)⟹(1)$ Suppose $||\cdot ||$ induces $||\cdot |{|}^{\prime }$ with the funky diagonal property. Suppose $x,y\in {\mathbb{C}}^n$ such that $|x|\le y$. We define the following diagonal matrix $D$:

• for $i=1,2,\dots ,n$, let $d_{ii}=\{\begin{array}{l}\frac{x_i}{y_i}\text{if }{y}_i\ne 0\\ 0\text{if }{y}_i=0\end{array}$ So, $Dy=x$ and $||D|{|}^{\prime }={\max }_i|d_{ii}|\le 1$. So

\lvert \lvert x \rvert \rvert =\lvert \lvert Dy \rvert \rvert \leq (*)\lvert \lvert D \rvert \rvert' \lvert \lvert y \rvert \rvert \leq (**)\lvert \lvert y \rvert \rvert \end{aligned}$$ Where $(*)$ is by definition of the [[Concept Wiki/matrix norm\|induced matrix norm]] and $(**)$ is because $\lvert \lvert D \rvert \rvert' \leq 1$. And the result follows!

Thus the result follows :)

(see funky diagonal property)

Berer-Fike Theorem

Let $||\cdot ||$ be a matrix norm on $M_n$ induced by a monotone vector norm. Let $A,\Delta A\in M_n$ be such that $A$ is diagonalizable as $A=SD{S}^{-1}$.

Then for all $\lambda \in \sigma (A+\Delta A)$, there exists a $\tau \in \sigma (A)$ such that $|\lambda -\tau |\le ||\Delta A||{\kappa }_{||\cdot ||}(S)$

• where $\kappa$ is the condition number

If $A$ is also normal, then $||\lambda -\tau ||\le ||\Delta A|{|}_{2,2}$

$\lambda \notin \sigma (A)$. (otherwise, the result is trivial). $\lambda I-[A+\Delta A]$ is singular (since $\lambda$ is an eigenvalue of $A+\Delta A$). So $S^{-1}[\lambda I-[A+\Delta A]]S$ is also singular. But note that $\lambda I-D$ is invertible (since $\lambda \notin \sigma (A)$) and this inverse has diagonal entries $\frac{1}{\lambda -d_{ii}}$.

Assume

S^{-1}[\lambda I - [A+\Delta A]]S &= \lambda I - D - S^{-1}\Delta AS\\ \implies (\lambda I-D)^{-1}[\lambda I-D-S^{-1}\Delta AS] &= I - [\lambda I-D]^{-1}S^{-1}\Delta AS \text{ is singular} \\ \implies 1 &\leq \lvert \lvert (\lambda I - D)^{-1}S^{-1}\Delta AS \rvert \rvert \;\;(*) \\ \implies 1 &\leq \lvert \lvert (\lambda I - D)^{-1} \rvert \rvert \cdot \lvert \lvert S^{-1}\Delta AS \rvert \rvert\\ &= \frac{1}{|\lambda-\tau|} \lvert \lvert S^{-1}AS \rvert \rvert \;\;\;\; (* *)\\ \end{aligned}$$ - $(*)$ follows since [[Concept Wiki/matrices with norm less than 1 define an infinite series inverse]], but we know that the matrix $I - [\lambda I-D]^{-1}S^{-1}\Delta AS$ is singular. - $(* *)$ follows by the [[Concept Wiki/funky diagonal property]]! We take the largest element in our definition of $[\lambda I - D]^{-1}$ If $A$ is normal, then $S$ can be chosen to be [[Concept Wiki/unitary matrices\|unitary]]! So $$\lvert \lvert S \rvert \rvert^2_{2,2} = \rho(S^*S) = \rho(I) = 1$$ So ${\kappa_{\lvert \lvert \cdot \rvert \rvert}}_{2,2}(S) = 1\cdot1=1$

(see Berer-Fike Theorem)

Note

If $A,\Delta A$ are hermitian, Weyl’s Theorem says

\lambda_{1}(\Delta A) + \lambda_{k}(A) &\leq \lambda_{k}(A+\Delta A) \leq \lambda_{n}(\Delta A) + \lambda_{k}(A) \\ \implies \lambda_{1}(\Delta A) &\leq \lambda_{k}(A + \Delta A) - \lambda_{k}(A) \leq \lambda_{n}(A) \\ \implies \lvert \lambda_{k}(A+\Delta A) - \lambda_{k}(A) \rvert &\leq \rho(\Delta A) \leq \lvert \lvert \Delta A \rvert \rvert _{2,2} \end{aligned}$$ ie, we can say WHICH eigenvalue each eigenvalue of the perturbed matrix is "close to"