Functional Analysis Lecture 15

[[lecture-data]]

← Lecture 14 | Lecture 16 → Lecture Notes: Rodriguez, page 77

3. Hilbert Spaces

Orthonormal Bases

Recall from last time, we introduced

Maximal orthonormal set

$\{e_{\lambda }{\}}_{\lambda \in \Lambda }$ in a Hilbert space is maximal if $u\in H$ and $⟨u,e_{\lambda }⟩=0$ for all $\lambda \in \Lambda$ implies $u=0$.

see maximal orthonormal set

Theorem

Let $H$ be a separable Hilbert space. Then $H$ has a countable maximal orthonormal set.

See separable hilbert spaces have countable maximal orthonormal sets

Orthonormal Basis

Let $H$ be a Hilbert space. An orthonormal basis of $H$ is a countable maximal orthonormal subset $\{e_n\}$ of $H$.

see orthonormal basis of a hilbert space

Theorem

If $\{e_n\}$ is an orthonormal basis of a hilbert space $H$, then for all $u\in H$ we have ${\lim }_{m\to \infty }{\sum }_{n=1}^m⟨u,e_n⟩e_n={\sum }_{n=1}^{\infty }⟨u,e_n⟩e_n=u$

NOTE

ie like in finite-dimensional linear algebra, we can write any element as a linear combination of the basis elements. But in this space, there might be an infinite number of such elements.

Proof (via Bessel's inequality)

First, we prove ${\left\{{\sum }_{n=1}^m⟨u,e_n⟩e_n\right\}}_m$ is Cauchy. Let $ϵ>0$. Then by Bessel’s inequality, we have ${\sum }_{n=1}^{\infty }|⟨u,e_n⟩{|}^2\le ||u|{|}^2<\infty$ Thus, for all $M\in \mathbb{N}$ such that for all $N\ge M$ we have ${\sum }_{m=N+1}^{\infty }|⟨u,e_n⟩{|}^2<ϵ$

Then for all $m>\ell \ge M$ we compute

\left\lvert \left\lvert \sum_{n=1}^m \langle u, e_{n} \rangle e_{n} - \sum_{n=1}^\ell \langle u, e_{n} \rangle e_{n} \right\rvert \right\rvert &= \sum_{n=\ell+1}^m \lvert \langle u, e_{n} \rangle \rvert ^2 \\ &\leq \sum_{\ell+1}^\infty \lvert \langle u, e_{n} \rangle \rvert ^2 \\ &< \epsilon^2 \end{align}$$ thus the sequence is indeed Cauchy. Since $H$ is complete, there exists some $\bar{u}\in H$ where $$\bar{u} = \lim_{ m \to \infty } \sum_{n=1}^m \langle u, e_{n} \rangle e_{n}$$ By [[continuity of inner product]], we know that for all $\ell \in \mathbb{N}$ $$\begin{align} \langle u-\bar{u}, e_{\ell} \rangle &= \lim_{ m \to \infty } \left\langle u - \sum_{n=1}^m \langle u, e_{n} \rangle e_{n}, e_{\ell} \right\rangle \\ &= \lim_{ m \to \infty } \left[ \langle u, e_{\ell} \rangle - \sum_{n=1}^m \langle u, e_{n} \rangle \langle e_{n}, e_{\ell} \rangle \right] \\ &= \langle u, e_{\ell} \rangle - \langle u, e_{\ell}\cdot 1 \rangle \\ &= 0 \end{align}$$ Thus $\langle u-\bar{u}, e_{\ell} \rangle = 0$ for all $\ell$ if and only if $u - \bar{u} = 0$. $$\tag*{$\blacksquare$}$$

see all elements of hilbert spaces with orthonormal bases can be written as sums of the basis elements

Thus if we have an orthonormal basis, every element can be expanded in this series in terms of the basis elements (Bessel-Fourier series). And thus every separable Hilbert space has an orthonormal basis.

Theorem

If a Hilbert space $H$ has an orthonormal basis, then $H$ is separable.

Proof

Suppose $\{e_n\}$ is an orthonormal basis for $H$. Then $S={\bigcup }_{m\in \mathbb{N}}\left\{{\sum }_{n=1}^m{q}_n{e}_n|q_1,\dots ,q_m\in \mathbb{Q}+i\mathbb{Q}\right\}$ is a countable. This is because elements in each component (index by $m$) have a bijection with ${\mathbb{Q}}^{2m}$, which is countable. Then since the $m$ are countable, the union is countable and therefore yields a countable set.

So by all elements of hilbert spaces with orthonormal bases can be written as sums of the basis elements, $S$ is dense in $H$

• (since every element $u$ can be expanded in the series described above, and so the partial sums converge to $u$) Thus for any $ϵ>0$, we have take a sufficiently long partial sum of length $L$ to get within $\frac{ϵ}{2}$ of $u$ and approximate each coefficient with a rational number.

So the sums will be in one of the parts of $S$ above, ie $S$ is a countable dense subset of $H$, ie $H$ is separable. \tag*{$\blacksquare$}

See Hilbert spaces with orthonormal bases are separable

• and the resulting Hilbert spaces are separable if and only if they have an orthonormal basis

Theorem (Parseval's Identity)

Let $H$ be a Hilbert space and let $\{e_n\}$ be a countable orthonormal basis of $H$. Then for all $u\in H$, ${\sum }_n|⟨u,e_n⟩{|}^2=||u|{|}^2$ (in Bessel’s inequality, we only had $\le$)

Proof

We know that $u={\sum }_n⟨u,e_n⟩e_n$ from all elements of hilbert spaces with orthonormal bases can be written as sums of the basis elements. If the sum over $n$ is a finite sum, then we just expand the inner product $||u|{|}^2=⟨u,u⟩$. Otherwise, by continuity of inner product, we have

\lvert \lvert u \rvert \rvert ^2 &= \lim_{ m \to \infty } \left\langle \sum_{n=1}^m \langle u, e_{n} \rangle e_{n}, \sum_{\ell=1}^m \langle u, e_{\ell} \rangle e_{\ell} \right\rangle \\ &= \lim_{ m \to \infty } \sum_{n, \ell = 1}^m\langle u, e_{n} \rangle \overline{\langle u, e_{\ell} \rangle } \langle e_{n, e_{\ell}} \rangle \\ (e_{n} \perp e_{\ell}, n \neq \ell) \implies&= \lim_{ m \to \infty } \sum_{n=1} \langle u, e_{n} \rangle \overline{\langle u, e_{n} \rangle } \\ &= \lim_{ m \to \infty } \sum_{n=1}^m \lvert \langle u, e_{n} \rangle \rvert ^2 \end{align}$$ $$\tag*{$\blacksquare$}$$

see Parseval’s identity

We now have a way to identify every separable Hilbert space with the one that was introduced at the beginning of the course.

Theorem

If $H$ is an infinite-dimensional separable Hilbert space, then $H$ is isometrically isomorphic to ${\ell }^2$.

ie, there exists a bijective bounded linear operator $T:H\to {\ell }^2$ such that for all $u,v\in H$ we have $||Tu|{|}_{{\ell }^2}=||u|{|}_H\text{ and }⟨Tu,Tv{⟩}_{{\ell }^2}=⟨u,v{⟩}_H$

The finite case is easier to deal with- we can just show that they are isometrically isomorphic to ${\mathbb{C}}^n$ for some $n$

Proof (sketch)

Since $H$ is separable, it has an orthonormal basis (Hilbert spaces are separable if and only if they have an orthonormal basis). And we can write the Fourier-Bessel series for each element $u\in H$:

u &= \sum_{n=1}^\infty \langle u, e_{n} \rangle e_{n} \\ \implies \lvert \lvert u \rvert \rvert &= \left( \sum_{n=1}^\infty \lvert \langle u, e_{n} \rangle \rvert^2 \right)^{1/2} \end{align}$$ By [[Parseval's identity]]. So we can define our map $T$ as $$T\,u := \{ \langle u, e_{n} \rangle \}_{n}$$ ie, $T\,u$ is the sequence of coefficients in the expansion. And this sequence is in $\ell^2$ ([[l-p vector space|ell-2]]). - $T$ is [[linear function|linear]] by construction, - It is [surjective](https://en.wikipedia.org/wiki/Surjective_function) because every sum $\sum_{n=1}^\infty c_{n }e_{n}$ is Cauchy in $H$ - It is [one-to-one](https://en.wikipedia.org/wiki/Injective_function) because every $u$ is expanded in this way, so any two expansions that are the same will be have the same infinite sum.

see separable Hilbert spaces are bijective with ell-2

Fourier Series

Proposition

The subset of functions ${\left\{\frac{e^{inx}}{\sqrt{2\pi }}\right\}}_{n\in \mathbb{Z}}$ is an orthonormal subset of $L^2([-\pi ,\pi ])$

NOTE

if we don’t like working with complex exponentials we can use $e^{ix}=\cos x+i\sin x$ And work out everything we need.

Proof

Note that

\langle e^{inx}, e^{imx} \rangle &= \int _{-\pi}^\pi e^{inx} \overline{e^{imx}} \, dx \\ &=\int _{-\pi}^\pi e^{i(n-m)x} \, dx \\ &= \begin{cases} 2\pi, & m=n \\ \frac{1}{i(n-m)} e^{i(n-m)x} \bigg{|}_{-\pi}^\pi=0 & m \neq n \end{cases} \end{align}$$ Since the exponential is periodic in $2\pi$. Normalizing by $2\pi$ yields $$\left\langle \frac{e^{inx}}{\sqrt{ 2\pi }} ,\frac{e^{imx}}{\sqrt{ 2\pi }} \right \rangle =\begin{cases} 1 & m=n \\ 0 & m \neq n \end{cases}$$ Giving us our orthonormal set.

see the fourier functions form an orthonormal set

Fourier coefficient

For a function $f\in L^2([-\pi ,\pi ])$, the Fourier coefficient $\hat{f}(n)$ of $f$ is given by $\hat{f}(n)=\frac{1}{2\pi }{\int }_{-\pi }^{\pi }f(t)e^{-int}dt$ And the $N$th partial Fourier sum is $S_{N}f(x)={\sum }_{|n|\le N}\hat{f}(n)e^{inx}={\sum }_{|n|\le N}⟨f,\frac{e^{inx}}{\sqrt{2\pi }}⟩\frac{e^{inx}}{2\pi }$

NOTE

this follows from all elements of hilbert spaces with orthonormal bases can be written as sums of the basis elements and Hilbert spaces are bijective with ell-2

see Fourier coefficient

Fourier series

The Fourier series of $f$ is the formal series ${\sum }_{n\in \mathbb{Z}}\hat{f}(n)e^{inx}$

see Fourier series

Question

Does convergence (in $L^2$) ${\sum }_{n=1}^{\infty }\hat{f}{e}^{inx}\to f$ hold for all $f\in L^2([-\pi ,\pi ])$? ie, does $||f-S_{N}f|{|}_2={\left({\int }_{-\pi }^{\pi }|f(x)-S_{N}f(x){|}^{2}dx\right)}^{1/2}\to 0\text{ as }N\to \infty$

• ie, is $\left\{\frac{e^{inx}}{\sqrt{2\pi }}\right\}$ a maximal orthonormal set in $L^2([-\pi ,\pi ])$?
• ie, does $\hat{f}(n)=0\forall n\in \mathbb{Z}⟹f=0$ ?

Answer

It turns out, yes, but we will need to build some more framework to show this.

Theorem

For all $f\in L^2([-\pi ,\pi ])$ and all $N\in \mathbb{N}\cup \{0\}$,

S_{N}f(x) &= \int _{-\pi}^\pi D_{N}(x-t) f(t) \, dx \\ D_{N}(x) &= \begin{cases} \frac{2N+1}{2\pi} & x=0 \\ \frac{\sin\left( N+\frac{1}{2} \right)x}{2\pi \sin \frac{x}{2}} & x \neq 0 \end{cases} \end{align}$$ The function $D_{N}$ is [[continuous map|continuous]] and also smooth and is called the **Dirichlet kernel**.

Proof (warmup for some calculations)

For any $f\in L^2([-\pi ,\pi ])$, we know that

S_{N}f(x) &= \sum_{\lvert n \rvert \leq N} \left( \frac{1}{2\pi} \int _{-\pi}^\pi f(t) e^{- i n t} \, dt \right)e^{inx} \\ (*)&= \int _{-\pi}^\pi f(t) \underbrace{\left( \frac{1}{2\pi} \sum_{\lvert n \rvert \leq N} e^{i n (x-t)}\right)}_{D_{N}(x-t)}\, dt \\ \end{align}$$ Where $(*)$ is by [[lebesgue integral of sum is sum of the integral|linearity of the integral]] $$\begin{align} D_{N}(x) &= \frac{1}{2\pi} \sum_{\lvert n \rvert \leq N} e^{i n x} \\ &= \frac{1}{2\pi} e^{-i N x} \sum_{n=0}^{2N} e^{i n x} \\ &= \begin{cases} \frac{1}{2\pi} e^{-i N x} \left[ \frac{1 - e^{i(2N-1)x}}{1-e^{ix}} \right] ,& e^{ix}\neq 1 \iff x\neq 0 \\ \frac{2N+1}{2\pi}, & x=0 \end{cases} \\ &= \begin{cases} \frac{1}{2\pi} \left[ \frac{e^{i(N + 1/2)x} - e^{i(N+1/2)x}}{e^{ix/2}-e^{-ix/2}} \right] ,& x\neq 0 \\ \frac{2N+1}{2\pi}, & x=0 \end{cases} \\ \left( \sin x = \frac{e^{ix} - e^{-ix}}{2i} \right) \implies&=\begin{cases} \frac{\sin\left( N+\frac{1}{2} \right)x}{2\pi \sin \frac{x}{2}},& x\neq 0 \\ \frac{2N+1}{2\pi}, & x=0 \end{cases} \end{align}$$ Which gives us the desired result $$\tag*{$\blacksquare$}$$

See fourier partial sums are given by the Dirichlet kernel

Cesaro-Fourier mean

Let $f\in L^2([-\pi ,\pi ])$. The $N$th Cesaro-Fourier mean of $f$ is ${\sigma }_{N}f(x)=\frac{1}{N+1}{\sum }_{k=0}^N{S}_{k}f(x)$ Where $S_{k}f(x)$ is the $k$th partial Fourier sum.

see Cesaro-Fourier mean

NOTE

• If the Fourier coefficients are all zero, then the function is zero
• Want to show that if the partial Fourier sums converge, then it must be a sum of zeros. But doing this with the $S_N$ directly is the original question.
• The “averaged” version (the Cesaro-Fourier mean) is easier to work with, so we will try and show convergence here instead

Note

We know from real analysis that the Cesaro means of a sequence behave better but do not lose any information.

$\{-1,1,-1,1,\dots \}$ do not converge, but their Cersaro means do

Sequences like

Next time, we will see more why this convergence works.

const { dateTime } = await cJS()
 
return function View() {
	const file = dc.useCurrentFile();
	return <p class="dv-modified">Created {dateTime.getCreated(file)}     ֍     Last Modified {dateTime.getLastMod(file)}</p>
}