Lecture 18

[[lecture-data]]

2024-10-09

Readings

• a

4. Chapter 4

Interlacing II / Inclusion Principle

Suppose $A\in M_n$ is hermitian and $B$ is an $r\times r$ principal submatrix. Then for all $k$, ${\lambda }_k(A)\le {\lambda }_k(B)\le {\lambda }_{k+n-r}(A)$

Proof (via Courant-Fisher) Say $B$ comes from $A$, deleting rows and columns $i_1,i_2,\dots ,i_{n-r}$.

\lambda_{k}(A) &= \max_{y_{1}, y_{2}, \dots, y_{k-1} \in \mathbb{C}^n} \min_{x \in \mathbb{C^n}\neq 0, x \perp y_{i} \forall i} \frac{x^*Ax}{x^*x} \\ &= \max_{y_{1}, \dots, y_{k-1}} \min_{x \perp y_{i} \forall i, x \perp e_{i_{1}},\dots, e_{i_{n-r}}} \frac{x^*Ax}{x^*x} \\ (*) &= \max_{v_{1}, v_{2}, \dots, v_{k-1} \in \mathbb{C}^r} \min_{z \in \mathbb{C}^r \neq 0, z \perp v_{1}, \dots, v_{k-1}} \frac{z^*Bz}{z^*z} = \lambda_{k}(B) \;\;\;\;\text{ by Courant-Fischer} \end{aligned}$$ - Note that the $e_{i_{k}}$ are the standard basis vectors with the $1$ in the index of each $i_{k}$ - The $z$s are the $x$s without the $i_{k}$ components (since they are orthogonal to those standard basis vectors) - We can then "perform surgery" on the $y$s also to get rid of those coordinates to get $v_{1},\dots,v_{k-1} \in \mathbb{C}^r$

(see inclusion principle)

Corollary

Suppose $A\in M_n$ hermitian and $\hat{A}$ is a $n-1\times n-1$ principal submatrix. Then ${\lambda }_1(A)\le {\lambda }_1(\hat{A})\le {\lambda }_2(A)\le {\lambda }_2(\hat{A})\le {\lambda }_3(A)\le \cdots \le {\lambda }_k(\hat{A})\le {\lambda }_k(A)$

(see the eigenvalues of a hermitian matrix and a principal submatrix one smaller alternate in magnitude)

Corollary

For any $A\in M_n$ hermitian, $\forall i$ we have ${\lambda }_1(A)\le a_{ii}\le {\lambda }_n(A)$.

This follows immediately from the inclusion principle, since each diagonal entry is a $1\times 1$ principal submatrix

(see the diagonal entries of a hermitian matrix are bounded by the eigenvalues)

Majorization

Let $x,y\in {\mathbb{R}}^n$. Say we can order the components of $x$ so that $x_{{\ell }_1}\le \cdots \le x_{{\ell }_n}$ and the same for the components of $y$ so we have $y_{k_1}\le \cdots \le y_{k_n}$. We say $x$ majorizes $y$ precisely when

• For all $m,{\sum }_{i=1}^m{x}_{{\ell }_i}\ge {\sum }_{i=1}^m{y}_{k_i}$ and
• equality holds when $m=n$

(see majorization)

Theorem

Let $A\in M_n$ be hermitian. Then the vector of diagonal elements of $A$, call it $\text{diag}A$ majorizes the vector of ordered eigenvalues of $A$, call it $\Lambda (A)$

Proof via induction on $A$ Any case where $n=1$ is trivially true. Assume the theorem holds for all matrices of size $n$ up to some fixed $k-1$. Consider the case when $n=k$.

Let $B$ be a submatrix of $A$ obtained by deleting one row and corresponding column ${\ell }_k$ where the diagonals of $A$ are ordered $a_{{\ell }_1{\ell }_1}\le a_{{\ell }_2{\ell }_2}\le \cdots \le a_{{\ell }_k{\ell }_k}$.

For all $n=1,\dots ,k-1$, we have ${\sum }_{i=1}^k{\lambda }_i(A)\le {\sum }_{i=1}^k{\lambda }_i(B)$ by interlacing 2. Then by the induction hypothesis, we have that ${\sum }_{i=1}^n{\lambda }_i(B)\le {\sum }_{i=1}^n{a}_{{\ell }_i{\ell }_i}$. But for the case when $n=k$, we have that ${\sum }_{i=1}^n{\lambda }_i(A)=\mathrm{Tr}(A)={\sum }_{i=1}^n{a}_{ᵢ}$ Thus the theorem holds

(see diagonal elements of a hermitian matrix majorize its eigenvalues)

Corollary

Let $A\in M_n$ be hermitian and $r:1\le r\le n\in \mathbb{Z}$. Then ${\sum }_{k=1}^r{\lambda }_k(A)={\min }_{U\in M_{n,r}\text{ orthonormal cols}}\mathrm{Tr}{U}^{\ast }AU$

and also ${\sum }_{k=0}^{r-1}{\lambda }_{n-k}(A)={\max }_{U\in M_{n,r}\text{ orthonormal cols}}\mathrm{Tr}{U}^{\ast }AU$

And claim that this implies the previous result ( just take take $U$ as the identity )

Proof / Intuition $U\in M_n$ with orthonormal columns, extend Gram-Schmidt to get $V=[U|\ast ]\in M_n$ unitary. Then when we do the multiplication $[V^{\ast }AV{]}_{r\times r}=U^{\ast }AU$ So by interlacing 2, we get that ${\lambda }_k(A)={\lambda }_k(V^{\ast }AV)\le {\lambda }_k(U^{\ast }AU)$ So sum over $k=1,2,\dots ,r$ to get ${\sum }_{k=1}^r{\lambda }_k(A)\le \mathrm{Tr}(U^{\ast }AU)$ So we have the desired lower bound, and need to show we can achieve equality.

Given any

Let $W$ be the orthonormalized eigenvectors associated with ${\lambda }_1(A),\dots ,{\lambda }_r(A)$. $\mathrm{Tr}{U}^{\ast }AU=\mathrm{Tr}{W}^{\ast }AW=Tr{D}_r$ (the sum of the first $r$ eigenvalues)

(see the sum of the first least eigenvalues is the minimum of the trace of orthonormal multiplications)