Lecture 17

[[lecture-data]]

2024-10-07

Caution

Exam on the 15th from 7-10pm

4. Chapter 4

The main theorem of this chapter is Courant-Fisher Theorem

Courant-Fisher Theorem

let $A\in M_n$ be hermitian with eigenvalues ${\lambda }_1\le \cdots \le {\lambda }_n$ (since $A$ is hermitian, its eigenvalues are real (see this theorem)). Then $\forall k=1,2,\dots n$ we have ${\max }_{y_1,y_2,\dots ,y_{k-1}\in {\mathbb{C}}^n}\left\{{\min }_{x\in {\mathbb{C}}^n\ne 0,x\perp y_1,y_2,\dots ,y_{k+1}}\frac{x^{\ast }Ax}{x^{\ast }x}\right\}={\lambda }_k$ and ${\min }_{y_{k+1},\dots ,y_n\in {\mathbb{C}}^n}\left\{{\max }_{x\in {\mathbb{C}}^n\ne 0,x\perp y_{k+1},\dots ,y_n}\frac{x^{\ast }Ax}{x^{\ast }x}\right\}={\lambda }_k$

Claim

Consider $\{a_i{\}}_{i\in I}$ and $\{b_i{\}}_{i\in I}$. Suppose for all $i$, we have that $a_i\le b_i$. Then ${\min }_{i\in I}{a}_i\le {\min }_{i\in I}{b}_i$ and ${\max }_{i\in I}{a}_i\le {\max }_{i\in I}{b}_i$

Further, if I minimize (or maximize) over some conditions $\kappa$, these inequalities still hold provided I apply the same conditions to both sides. I can even make one of these conditions a minimization (or maximization) over some more conditions!

We also have for conditions $\kappa$ and additional conditions ${\kappa }^{\prime }$ that ${\min }_{\kappa }xyz\le {\min }_{\kappa \cup {\kappa }^{\prime }}xyz$ (“the minimization might not work as well”)

And so $\max {\min }_{\kappa }xyz\le \max {\min }_{\kappa \cup {\kappa }^{\prime }}xyz$

Weyl's Theorem

For any $A,B\in M_n$ hermitian, for any $k=1,2,\dots ,n$ we have ${\lambda }_k(A)+{\lambda }_1(B)\le {\lambda }_k(A+B)\le {\lambda }_k(A)+{\lambda }_n(B)$ (where ${\lambda }_i$ is the $i$th largest eigenvalue)

Proof

\lambda_{k}(A+B) &= \min\max_{\text{courant-fisher conditions}} \frac{x^*(A+B)x}{x^*x} \\ &= \min \max_{CF} \frac{x^*Ax}{x^*x} + \frac{x^*Bx}{x^*x} \\ &= \min \max_{CF} \frac{x^*Ax}{x^*x} + \lambda_{n}(B) \;\;\;(*) \\ &= \lambda_{n}(B) + \lambda_{k}(A) \;\;\;\;(* *) \end{aligned}$$ Where $(*)$ is due to [[Concept Wiki/Rayleigh-Ritz Theorem\|Rayleigh-Ritz]] and $(* *)$ is due to [[Concept Wiki/Courant-Fisher Theorem\|Courant-Fisher]]. The other inequality follows analogously

(see Weyl’s Theorem)

Corollary

If $A,B\in M_n$ hermitian and $B$ is postive semidefinite, then for all $k=1,2,\dots ,n$ we have ${\lambda }_k(A)\le {\lambda }_k(A+B)$

${\lambda }_1(B)>0$, by Weyl ${\lambda }_k(A)\le {\lambda }_k(A)+{\lambda }_1(B)\le {\lambda }_k(A+B)$

Since

Interlacing Theorem I

If $A\in M_n$ is hermitian, $a\in \mathbb{R}$, $z\in {\mathbb{C}}^n$, then for all $k$: ${\lambda }_k(A+az{z}^{\ast })\le {\lambda }_{k+1}(A)$

Proof

\lambda_{k}(A + azz^*) &= \max_{y_{1}, y_{2}, \dots, y_{k-1} \in \mathbb{C}^n} \min_{x \in \mathbb{C}^n \neq 0, x \perp y_{1}, \dots, y_{k-1}} \frac{x^*(A + azz^*)x^*}{x^*x} \\ &\leq \max_{y_{1}, \dots, y_{k-1} \in \mathbb{C}^n} \min_{x \in \mathbb{C}^n \neq 0, x \perp y_{1}, \dots, y_{k-1}, x \perp z} \frac{x^*(A + azz^*)x^*}{x^*x} \\ &= \max_{y_{1}, \dots, y_{k-1} \in \mathbb{C}^n} \min_{x \in \mathbb{C}^n \neq 0, x \perp y_{1}, \dots, y_{k-1},z} \frac{x^*Ax^*}{x^*x} \;\;\; (*) \\ &\leq \max_{y_{1}, y_{2}, \dots, y_{k} \in \mathbb{C}^n} \min_{x \in \mathbb{C}^n \neq 0, x \perp y_{1}, \dots, y_{k}} \frac{x^*Ax^*}{x^*x} \;\;\;\;(* *) \\ &= \lambda_{k+1}(A) \;\;\;\;\text{ by Courant-Fisher} \end{aligned}$$ - $(*)$ since if $z \perp x$, then $ax^*zz^*x = 0$ - $(* *)$ since we can choose $y_{k} = z$

(see interlacing theorem 1)

Theorem

In general, if $B\in M_n$ is hermitian, then it is unitarily diagonalizable, say $B=UD{U}^{\ast },U=[u_1|u_2|\dots |u_n]$ and ${\lambda }_1,{\lambda }_2,\dots ,{\lambda }_n$ the diagonal elements of $D$. Then $B=UD{U}^{\ast }={\sum }_{i=1}^n{\lambda }_i{u}_i{u}_i^{\ast }$

(see normal matrices are the sum of rank-1 matrices)

Corollary

Let $A,B\in M_n$ be hermitian and rank of $B$ is $r$, then for all $k$, ${\lambda }_k(A+B)\le {\lambda }_{k+r}(A)$

interlacing theorem 1 and the fact that hermitian matrices are the sum of rank-1 matrices

This follows directly from the

Say $B=UD{U}^{\ast },U=[u_1|u_2|\dots |u_n]$ unitary. WLOG let the entries $d_{11},d_{22},\dots ,d_{rr}$ be the only non-zero eigenvalues / nonzero entries of $D$ (we can do this because $B$ is diagonalizable). Then

\lambda_{k+r}(A) & \geq \lambda_{k+r-1}(A + d_{rr}u_{r}u_{r}^*) \;\;\;\;(*)\\ & \geq \lambda_{k+r-2}(A + d_{rr}u_{r}u_{r}^* + d_{(r-1)(r-1)}u_{r-1}u_{r-1}^*) \\ & \geq \dots \\ & \geq \lambda_{k}\left( A + \sum_{i=1}^r d_{ii}u_{i}u_{i}^* \right) \\ &= \lambda_{k}(A+B) \;\;\;\; (* *) \end{aligned}$$ Where we get $(*)$ from the [[Concept Wiki/interlacing theorem 1]] and $( * *)$ from [[Concept Wiki/normal matrices are the sum of rank-1 matrices\|hermitian matrices are the sum of rank-1 matrices]]

Corollary

Let $A,B\in M_n$ be hermitian and $B$ is rank $r$. Then for all $k$ ${\lambda }_{k-r}(A+B)\le {\lambda }_k(A)\le {\lambda }_{k+r}(A+B)$

${\lambda }_{k-r}(A)\le {\lambda }_k(A+B)\le {\lambda }_{k+r}(A)$

This is a direct result/generalization of the above corollary.

Let $A+B=\text{scr}A$ and $-B=\text{scr}B$. Then from the above we have ${\lambda }_k(\text{scr}A+B)\le {\lambda }_{k+r}(\text{scr}A)$

(see bounds on eigenvalues for sums of hermitian matrices)