Functional Analysis Lecture 15

[[lecture-data]]

← Lecture 14 | Lecture 16 →
Lecture Notes: Rodriguez, page 77

3. Hilbert Spaces

Orthonormal Bases

Recall from last time, we introduced

Maximal orthonormal set

${e_{λ}}_{λ \in Λ}$ in a [[Hilbert space]] is maximal if $u \in H$ and $⟨ u, e_{λ} ⟩ = 0$ for all $λ \in Λ$ implies $u = 0$ .

see [[maximal orthonormal set]]

Theorem

Let $H$ be a [[separable]] [[Hilbert space]]. Then $H$ has a countable [[maximal orthonormal set]].

See [[separable hilbert spaces have countable maximal orthonormal sets]]

Orthonormal Basis

Let $H$ be a maximal orthonormal subset ${e_{n}}$ of $H$ .

see [[orthonormal basis of a hilbert space]]

Theorem

If ${e_{n}}$ is an [[orthonormal basis of a hilbert space]] $H$ , then for all $u \in H$ we have

lim_{m \to \infty} \sum_{n = 1}^{m} ⟨ u, e_{n} ⟩ e_{n} = \sum_{n = 1}^{\infty} ⟨ u, e_{n} ⟩ e_{n} = u

Note

ie like in finite-dimensional linear algebra, we can write any element as a linear combination of the basis elements. But in this space, there might be an infinite number of such elements.

)

First, we prove ${\sum_{n = 1}^{m} ⟨ u, e_{n} ⟩ e_{n}}_{m}$ is Cauchy. Let $ϵ > 0$ . Then by [[Bessel's inequality]], we have

\sum_{n=1}^\infty \lvert \langle u, e_{n} \rangle \rvert {#2} \leq \lvert \lvert u \rvert \rvert {#2} < \infty

Thus, for all $M \in N$ such that for all $N \geq M$ we have $\sum_{m = N + 1}^{\infty} | ⟨ u, e_{n} ⟩ |^{2} < ϵ$

Then for all $m > ℓ \geq M$ we compute

\begin{align} \left\lvert \left\lvert \sum_{n=1}^m \langle u, e_{n} \rangle e_{n} - \sum_{n=1}^\ell \langle u, e_{n} \rangle e_{n} \right\rvert \right\rvert &= \sum_{n=\ell+1}^m \lvert \langle u, e_{n} \rangle \rvert {#2} \\ &\leq \sum_{\ell+1}^\infty \lvert \langle u, e_{n} \rangle \rvert {#2} \\ &< \epsilon^2 \end{align}

thus the sequence is indeed Cauchy. Since $H$ is complete, there exists some $\bar{u} \in H$ where

\bar{u} = lim_{m \to \infty} \sum_{n = 1}^{m} ⟨ u, e_{n} ⟩ e_{n}

By [[continuity of inner product]], we know that for all $ℓ \in N$

\begin{aligned} ⟨ u - \bar{u}, e_{ℓ} ⟩ & = lim_{m \to \infty} ⟨ u - \sum_{n = 1}^{m} ⟨ u, e_{n} ⟩ e_{n}, e_{ℓ} ⟩ \\ = lim_{m \to \infty} [⟨ u, e_{ℓ} ⟩ - \sum_{n = 1}^{m} ⟨ u, e_{n} ⟩ ⟨ e_{n}, e_{ℓ} ⟩] \\ = ⟨ u, e_{ℓ} ⟩ - ⟨ u, e_{ℓ} \cdot 1 ⟩ \\ = 0 \end{aligned}

Thus $⟨ u - \bar{u}, e_{ℓ} ⟩ = 0$ for all $ℓ$ if and only if $u - \bar{u} = 0$ .

\begin{matrix} ◼ \end{matrix}

see [[all elements of hilbert spaces with orthonormal bases can be written as sums of the basis elements]]

Thus if we have an orthonormal basis, every element can be expanded in this series in terms of the basis elements (Bessel-Fourier series). And thus every separable [[Hilbert space]] has an orthonormal basis.

Theorem

If a orthonormal basis, then $H$ is [[separable]].

Proof

Suppose ${e_{n}}$ is an orthonormal basis for $H$ . Then

S = ⋃_{m \in N} {\sum_{n = 1}^{m} q_{n} e_{n} | q_{1}, \dots, q_{m} \in Q + i Q}

is a countable. This is because elements in each component (index by $m$ ) have a bijection with $Q^{2 m}$ , which is countable. Then since the $m$ are countable, the union is countable and therefore yields a countable set.

So by [[all elements of hilbert spaces with orthonormal bases can be written as sums of the basis elements]], $S$ is dense in $H$

(since every element $u$ can be expanded in the series described above, and so the partial sums converge to $u$ )
Thus for any $ϵ > 0$ , we have take a sufficiently long partial sum of length $L$ to get within $\frac{ϵ}{2}$ of $u$ and approximate each coefficient with a rational number.

So the sums will be in one of the parts of $S$ above, ie $S$ is a countable dense subset of $H$ , ie $H$ is separable.

\begin{matrix} ◼ \end{matrix}

See [[Hilbert spaces with orthonormal bases are separable]]

and the resulting [[Hilbert spaces are separable if and only if they have an orthonormal basis]]

Theorem (Parseval's Identity)

Let $H$ be a orthonormal basis of $H$ . Then for all $u \in H$ ,

\sum_{n} \lvert \langle u, e_{n} \rangle \rvert {#2} = \lvert \lvert u \rvert \rvert {#2}

(in [[Bessel's inequality]], we only had $\leq$ )

[!proof]
We know that

u = \sum_{n} ⟨ u, e_{n} ⟩ e_{n}

from , we have

\begin{align} \lvert \lvert u \rvert \rvert {#2} &= \lim_{ m \to \infty } \left\langle \sum_{n=1}^m \langle u, e_{n} \rangle e_{n}, \sum_{\ell=1}^m \langle u, e_{\ell} \rangle e_{\ell} \right\rangle \\ &= \lim_{ m \to \infty } \sum_{n, \ell = 1}^m\langle u, e_{n} \rangle \overline{\langle u, e_{\ell} \rangle } \langle e_{n, e_{\ell}} \rangle \\ (e_{n} \perp e_{\ell}, n \neq \ell) \implies&= \lim_{ m \to \infty } \sum_{n=1} \langle u, e_{n} \rangle \overline{\langle u, e_{n} \rangle } \\ &= \lim_{ m \to \infty } \sum_{n=1}^m \lvert \langle u, e_{n} \rangle \rvert {#2}

\end{align}$$
$\begin{matrix} ◼ \end{matrix}$

see [[Parseval's identity]]

We now have a way to identify every [[separable]] [[Hilbert space]] with the one that was introduced at the beginning of the course.

Theorem

If $H$ is an infinite-dimensional isometrically isomorphic to $ℓ^{2}$ .

ie, there exists a bijective bounded linear operator $T : H \to ℓ^{2}$ such that for all $u, v \in H$ we have

| | T u | |_{ℓ^{2}} = | | u | |_{H} and ⟨ T u, T v ⟩_{ℓ^{2}} = ⟨ u, v ⟩_{H}

The finite case is easier to deal with- we can just show that they are isometrically isomorphic to $C^{n}$ for some $n$

Proof (sketch)

Since $H$ is Fourier-Bessel series for each element $u \in H$ :

\begin{aligned} u & = \sum_{n = 1}^{\infty} ⟨ u, e_{n} ⟩ e_{n} \\ ⟹ | | u | | & = {(\sum_{n = 1}^{\infty} | ⟨ u, e_{n} ⟩ |^{2})}^{1 / 2} \end{aligned}

By [[Parseval's identity]]. So we can define our map $T$ as

T u := {⟨ u, e_{n} ⟩}_{n}

ie, $T u$ is the sequence of coefficients in the expansion. And this sequence is in $ℓ^{2}$ (ell-2).

$T$ is linear by construction,
It is surjective because every sum $\sum_{n = 1}^{\infty} c_{n} e_{n}$ is Cauchy in $H$
It is one-to-one because every $u$ is expanded in this way, so any two expansions that are the same will be have the same infinite sum.

see [[separable Hilbert spaces are bijective with ell-2]]

Fourier Series

Proposition

The subset of functions ${\frac{e^{i n x}}{\sqrt{2 π}}}_{n \in Z}$ is an orthonormal subset of $L^{2} ([- π, π])$

Note

if we don't like working with complex exponentials we can use

e^{i x} = \cos x + i \sin x

And work out everything we need.

Proof

Note that

\begin{aligned} ⟨ e^{i n x}, e^{i m x} ⟩ & = \int_{- π}^{π} e^{i n x} \overset{―}{e^{i m x}} d x \\ = \int_{- π}^{π} e^{i (n - m) x} d x \\ = {\begin{cases} 2 π, & m = n \\ \frac{1}{i (n - m)} e^{i (n - m) x} |_{- π}^{π} = 0 & m \neq n \end{cases} \end{aligned}

Since the exponential is periodic in $2 π$ . Normalizing by $2 π$ yields

⟨ \frac{e^{i n x}}{\sqrt{2 π}}, \frac{e^{i m x}}{\sqrt{2 π}} ⟩ = {\begin{cases} 1 & m = n \\ 0 & m \neq n \end{cases}

Giving us our orthonormal set.

see the [[fourier functions form an orthonormal set]]

Fourier coefficient

For a function $f \in L^{2} ([- π, π])$ , the Fourier coefficient $\hat{f} (n)$ of $f$ is given by

\hat{f} (n) = \frac{1}{2 π} \int_{- π}^{π} f (t) e^{- i n t} d t

And the $N$ th partial Fourier sum is

S_{N} f (x) = \sum_{| n | \leq N} \hat{f} (n) e^{i n x} = \sum_{| n | \leq N} ⟨ f, \frac{e^{i n x}}{\sqrt{2 π}} ⟩ \frac{e^{i n x}}{2 π}

Note

this follows from [[all elements of hilbert spaces with orthonormal bases can be written as sums of the basis elements]] and [[Hilbert spaces are bijective with ell-2]]

see [[Fourier coefficient]]

Fourier series

The Fourier series of $f$ is the formal series

\sum_{n \in Z} \hat{f} (n) e^{i n x}

see [[Fourier series]]

Question

Does convergence (in $L^{2}$ ) $\sum_{n = 1}^{\infty} \hat{f} e^{i n x} \to f$ hold for all $f \in L^{2} ([- π, π])$ ? ie, does

\lvert \lvert f - S_{N} f\rvert \rvert _{2} = \left( \int _{-\pi}^\pi \lvert f(x) - S_{N}f(x) \rvert {#2} \, dx \right)^{1/2} \to {0} \, \text{ as } N \to \infty

ie, is ${\frac{e^{i n x}}{\sqrt{2 π}}}$ a [[maximal orthonormal set]] in $L^{2} ([- π, π])$ ?
ie, does $\hat{f} (n) = 0 \forall n \in Z ⟹ f = 0$ ?

Answer

It turns out, yes, but we will need to build some more framework to show this.

Theorem

For all $f \in L^{2} ([- π, π])$ and all $N \in N \cup {0}$ ,

\begin{aligned} S_{N} f (x) & = \int_{- π}^{π} D_{N} (x - t) f (t) d x \\ D_{N} (x) & = {\begin{cases} \frac{2 N + 1}{2 π} & x = 0 \\ \frac{\sin (N + \frac{1}{2}) x}{2 π \sin \frac{x}{2}} & x \neq 0 \end{cases} \end{aligned}

The function $D_{N}$ is continuous and also smooth and is called the Dirichlet kernel.

Proof (warmup for some calculations)

For any $f \in L^{2} ([- π, π])$ , we know that

\begin{aligned} S_{N} f (x) & = \sum_{| n | \leq N} (\frac{1}{2 π} \int_{- π}^{π} f (t) e^{- i n t} d t) e^{i n x} \\ (*) & = \int_{- π}^{π} f (t) \underset{D_{N} (x - t)}{\underset{⏟}{(\frac{1}{2 π} \sum_{| n | \leq N} e^{i n (x - t)})}} d t \end{aligned}

Where $(*)$ is by linearity of the integral

\begin{aligned} D_{N} (x) & = \frac{1}{2 π} \sum_{| n | \leq N} e^{i n x} \\ = \frac{1}{2 π} e^{- i N x} \sum_{n = 0}^{2 N} e^{i n x} \\ = {\begin{cases} \frac{1}{2 π} e^{- i N x} [\frac{1 - e^{i (2 N - 1) x}}{1 - e^{i x}}], & e^{i x} \neq 1 ⟺ x \neq 0 \\ \frac{2 N + 1}{2 π}, & x = 0 \end{cases} \\ = {\begin{cases} \frac{1}{2 π} [\frac{e^{i (N + 1 / 2) x} - e^{i (N + 1 / 2) x}}{e^{i x / 2} - e^{- i x / 2}}], & x \neq 0 \\ \frac{2 N + 1}{2 π}, & x = 0 \end{cases} \\ (\sin x = \frac{e^{i x} - e^{- i x}}{2 i}) ⟹ & = {\begin{cases} \frac{\sin (N + \frac{1}{2}) x}{2 π \sin \frac{x}{2}}, & x \neq 0 \\ \frac{2 N + 1}{2 π}, & x = 0 \end{cases} \end{aligned}

Which gives us the desired result

\begin{matrix} ◼ \end{matrix}

See [[fourier partial sums are given by the Dirichlet kernel]]

Cesaro-Fourier mean

Let $f \in L^{2} ([- π, π])$ . The $N$ th Cesaro-Fourier mean of $f$ is

σ_{N} f (x) = \frac{1}{N + 1} \sum_{k = 0}^{N} S_{k} f (x)

Where $S_{k} f (x)$ is the $k$ th partial Fourier sum.

see [[Cesaro-Fourier mean]]

Note

If the Fourier coefficients are all zero, then the function is zero
Want to show that if the partial Fourier sums converge, then it must be a sum of zeros. But doing this with the $S_{N}$ directly is the original question.
The "averaged" version (the [[Cesaro-Fourier mean]]) is easier to work with, so we will try and show convergence here instead

Note

We know from real analysis that the Cesaro means of a sequence behave better but do not lose any information.

Example

Sequences like ${- 1, 1, - 1, 1, \dots}$ do not converge, but their Cersaro means do

Next time, we will see more why this convergence works.

const { dateTime } = await cJS()

return function View() {
	const file = dc.useCurrentFile();
	return <p class="dv-modified">Created {dateTime.getCreated(file)}     ֍     Last Modified {dateTime.getLastMod(file)}</p>
}