Hilbert Space

Definition (Hilbert space)

An F\mathbb{F}-Hilbert Space is an F\mathbb{F}-inner product space that is a Complete.

Remark

Any Hilbert Space is a Banach Space.

Remark (Why Hilbert spaces?)

  1. They allow us geometric insight on a set of signals
  2. If a Hilbert space is Separable then there exists countably (or finitely) many Orthonormal vectors which can be used as a Basis.
  3. This formulation allows us to develop approximations of signals using a finite number of basis signals
  4. We can state a projection theorem which certifies optimality in a wide variety of applications.

Theorem (2.2.3)

Let HH be a Hilbert space and BB a subspace of HH. Consider the problem of approximating some point x∈Hx\in H using a point from BB: inf⁡m∈B∥x−m∥\inf_{m\in B}\lVert x-m \rVert

  1. A necessary and sufficient condition for m∗∈Bm^{*}\in B to be the minimizing element in BB so that inf⁡m∈B∥x−m∥=∥x−m∗∥(1)\inf_{m\in B}\lVert x-m \rVert =\lVert x-m^{*} \rVert \tag{1}or equivalently ∥x−m∗∥≤∥x−y∥∀y∈B\lVert x-m^{*} \rVert \le \lVert x-y \rVert \quad \forall y\in Bis that x−m∗x-m^{*} is orthogonal to BB. If m∗m^{*} exists, then it is also unique.
  2. Let HH be a Hilbert space and BB be a closed subspace of HH. For any vector x∈Hx\in H, there is a unique vector m∗∈Bm^{*}\in B satisfying (1)(1).

\begin{proof} We first show 1. Let x∈Hx\in H, m0∈Bm_{0}\in B. Want to show that if x−m0x-m_{0} is not Orthogonal vector to BB then it cannot be optimal.

Suppose m0∈Bm_{0}\in B is such that ∃m∈B\exists m\in B with <x−m0,m>>0\left< x-m_{0},m \right>>0, i.e., x−m0x-m_{0} is not orthogonal to BB.

WLOG, take ∥m∥=1\lVert m \rVert=1 and <x−m0,m>=δ\left< x-m_{0},m \right>=\delta.

We can show that with m1=m0+δmm_{1}=m_{0}+\delta m, we will have ∥x−m1∥2=⟨x−m1,x−m1⟩2=⟨x−m0−δm,x−m0−δm⟩=⟨x−m0,x−m0⟩−δ⟨m,x−m0⟩−δ⟨x−m0,m⟩+δ2⟨m,m⟩=∥x−m0∥2−2δ2+δ2=∥x−m0∥2−δ2<∥x−m0∥2\begin{align*} \lVert x-m_{1} \rVert ^{2}&= \sqrt{ \langle x-m_{1}, x-m_{1} \rangle }^{2}\\ &= \langle x-m_{0}-\delta m, x-m_{0}-\delta m \rangle \\ &= \langle x-m_{0}, x-m_{0} \rangle -\delta \langle m, x-m_{0}\rangle-\delta \langle x-m_{0}, m \rangle +\delta^{2}\langle m, m \rangle \\ &= \lVert x-m_{0} \rVert ^{2}-2\delta^{2}+\delta^{2}\\ &= \lVert x-m_{0} \rVert ^{2}-\delta^{2}\\ &< \lVert x-m_{0} \rVert ^{2} \end{align*}and thus m0m_{0} cannot be a minimizer of (1)(1).

We now show that if x−m∗x-m^{*} is orthogonal to BB, then it is optimal. If <x−m∗,M>=0\left< x-m^{*},M \right> =0, we have that for any m∈Bm\in B
∥x−m∥2=∥x−m∗+m∗−m∥2=∥x−m∗∥2+∥m∗−m∥2≥∥x−m∗∥2\lVert x-m \rVert ^{2}=\lVert x-m^{*}+m^{*}-m \rVert ^{2}=\lVert x-m^{*} \rVert ^{2}+\lVert m^{*}-m \rVert ^{2}\ge \lVert x-m^{*} \rVert^{2} since for x⊥yx\perp y we have that ∥x+y∥2=∥x∥2+∥y∥2\lVert x+y \rVert^{2}=\lVert x \rVert^{2}+\lVert y \rVert^{2}. This yields a contradiction, proving 1.

We now show 2. Let δ=inf⁡m∈B∥x−m∥\delta=\inf_{m\in B}\lVert x-m \rVert Let {mk}k∈N\{ m_{k} \}_{k\in \mathbb{N}} be s.t. ∥x−mk∥→δ\lVert x-m_{k} \rVert \to\deltaand observe that ⟨x−mk+x−mn,x−mk+x−mn⟩+⟨x−mk−(x−mn),x−mk−(x−mn)⟩=⟨x−mk,x−mk+x−mn⟩+⟨x−mn,x−mk+x−mn⟩+⟨x−mk,x−mk−(x−mn)⟩+⟨−(x−mn),x−mk−(x−mn)⟩=2⟨x−mk,x−mk⟩+2⟨x−mn,x−mn⟩\begin{align*} &\langle x-m_{k}+x-m_{n}, x-m_{k}+x-m_{n} \rangle +\langle x-m_{k}-(x-m_{n}), x-m_{k}-(x-m_{n}) \rangle \\ &= \langle x-m_{k}, x-m_{k} +x-m_{n}\rangle +\langle x-m_{n}, x-m_{k}+x-m_{n} \rangle \\ &\quad\quad\quad+ \langle x-m_{k}, x-m_{k}-(x-m_{n}) \rangle +\langle -(x-m_{n}), x-m_{k}-(x-m_{n}) \rangle \\ &= 2\langle x-m_{k}, x-m_{k} \rangle +2\langle x-m_{n}, x-m_{n} \rangle \end{align*}Write ⟨x−mk+x−mn,x−mk+x−mn⟩=⟨2(x−mk+mn2),2(x−mk+mn2)⟩=4⟨x−mk+mn2,x−mk+mn2⟩\begin{align*} \langle x-m_{k}+x-m_{n}, x-m_{k}+x-m_{n} \rangle &= \left\langle 2\left( x- \frac{m_{k}+m_{n}}{2} \right), 2\left( x- \frac{m_{k}+m_{n}}{2} \right) \right\rangle \\ &= 4\left\langle x- \frac{m_{k}+m_{n}}{2}, x- \frac{m_{k}+m_{n}}{2} \right\rangle \tag{🚧} \end{align*}Since mk+mn2∈B\frac{m_{k}+m_{n}}{2}\in B then by our definition of δ\delta we have that: ∥x−mk+mn2∥≥δ\left\lVert x- \frac{m_{k}+m_{n}}{2} \right\rVert \ge\deltathen we square both sides then multiply by 44 4∥x−mk+mn2∥2≥4δ24\left\lVert x- \frac{m_{k}+m_{n}}{2} \right\rVert ^{2}\ge 4\delta^{2} and since the LHS is exactly (🚧)(🚧) then by applying Parallelogram law we get ∥mk−mn∥2≤2∥x−mn∥2+2∥x−mk∥2−4δ2\lVert m_{k}-m_{n} \rVert ^{2}\le 2\lVert x-m_{n} \rVert ^{2}+2\lVert x-m_{k} \rVert ^{2}-4\delta^{2}As a result, as ∥x−mn∥2→δ2\lVert x-m_{n} \rVert^{2}\to\delta^{2}, we have that mkm_{k} is Cauchy. Since BB is closed, it has a limit, call the limit m~\tilde{m}. We claim that the limit is optimal and hence m~=m∗\tilde{m}=m^{*}. Conside the difference: ⟨x−mn,x−mn⟩−⟨x−m~,x−m~⟩\langle x-m_{n}, x-m_{n} \rangle -\langle x-\tilde{m}, x-\tilde{m} \rangle We claim that the difference goes to zero; this follows from the Continuity of inner product (Yuksel). Indeed ∣⟨x−mn,x−mn⟩−⟨x−m~,x−m~⟩∣=∣⟨x−mn,x−mn⟩−⟨x−mn,x−m~⟩+⟨x−mn,x−m~⟩−⟨x−m~,x−m~⟩∣≤∣⟨x−mn,x−mn⟩−⟨x−mn,x−m~⟩∣+∣⟨x−mn,x−m~⟩−⟨x−m~,x−m~⟩∣=∣⟨x−mn,mn−m~⟩∣+∣⟨mn−m~,x−m~⟩∣≤∥x−mn∥∥mn−m~∥+∥mn−m~∥∥x−m~∥\begin{align*} \left| \langle x-m_{n}, x-m_{n} \rangle -\langle x-\tilde{m}, x-\tilde{m} \rangle \right| &= \left| \langle x-m_{n}, x-m_{n} \rangle -\langle x-m_{n}, x-\tilde{m} \rangle\right.\\ &\quad\quad\quad\left.+ \langle x-m_{n}, x-\tilde{m} \rangle- \langle x-\tilde{m}, x-\tilde{m} \rangle \right| \\ &\le \left| \langle x-m_{n}, x-m_{n} \rangle -\langle x-m_{n}, x-\tilde{m} \rangle \right| \\ &\quad\quad+ \left| \langle x-m_{n}, x-\tilde{m} \rangle -\langle x-\tilde{m}, x-\tilde{m} \rangle \right| \\ &= \left| \langle x-m_{n}, m_{n}-\tilde{m} \rangle \right| +\left| \langle m_{n}-\tilde{m}, x-\tilde{m} \rangle \right| \\ &\le \lVert x-m_{n} \rVert \lVert m_{n} -\tilde{m}\rVert +\lVert m_{n}-\tilde{m} \rVert \lVert x-\tilde{m} \rVert \end{align*}where the final inequality is due to Cauchy-Schwartz. Now, since ∥x−mn∥→δ\lVert x-m_{n} \rVert\to\delta, we have that ∥x−mn∥\lVert x-m_{n} \rVert is bounded. Finally as ∥mn−m~∥→0\lVert m_{n}-\tilde{m} \rVert\to0, both terms in the final line go to zero and we conclude that δ=lim⁡n→∞∥x−mn∥=∥x−m~∥\delta=\lim_{ n \to \infty } \lVert x-m_{n} \rVert =\lVert x-\tilde{m} \rVert and therefore m~\tilde{m} is a minimizing vector. By part 1, this has to be the only minimizing vector in BB.

\end{proof}

Remark

So this is the statement of the theorem in the class notes but I’m ngl, it’s unnecessarily abstract. Pretty much what this says that is that WLOG imagine we’re working in 2D space HH and we take a closed subset BB which we can say lives along some line in HH. This theorem says that if we take any point b∈Hb\in H then the closest point from x0∈Mx_{0}\in M to bb is one such that the difference vector b−x0b-x_{0} is orthogonal to MM: 300 The projection here is the point x0x_{0}. This is our best approximation to bb; we’re essentially “projecting” bb to some lower fidelity space MM and x0x_{0} is chosen as the candidate projection since it best approximates bb with respect to the Metric (or Norm or in our case, Inner Product).

Theorem (2.3.1)

Let {ei}\{ e_{i} \} be a sequence of orthonormal vectors in a Hilbert space HH. Let {xn=∑i=1nϵiei}n∈N\left\{ x_{n}=\sum_{i=1}^{n}\epsilon_{i}e_{i} \right\}_{n\in \mathbb{N}}be a sequence of vectors in HH. The sequence converges to some vector xˉ∈H\bar{x}\in H if and only if ∑i=1∞∣ϵi∣2<∞\sum_{i=1}^{\infty}\left| \epsilon_{i} \right| ^{2}<\infty In this case, we have that ⟨xˉ,ei⟩=ϵi\langle \bar{x}, e_{i} \rangle=\epsilon_{i}.

Theorem (2.3.3a)

For any two orthogonal vectors v,wv,w we have ∥v+w∥2=∥v∥2+∥w∥2\lVert v+w \rVert ^{2}=\lVert v \rVert ^{2}+\lVert w \rVert ^{2}

Theorem (2.3.3)

Let HH be a separable Hilbert space. Then, every orthonormal system of vectors in HH has a finite or countably infinite number of elements.

\begin{proof} Let D={x1,x2,…,}D=\{ x_{1},x_{2},\dots ,\} be a countable set dense in HH. Let {eα}α∈A⊂H\{ e_{\alpha} \}_{\alpha \in \mathcal{A}}\subset H be a set of orthonormal vectors. Observe that by , for α≠β\alpha \neq\beta:∥eα−eβ∥2=∥eα∥2+∥eβ∥2=2\lVert e_{\alpha}-e_{\beta} \rVert^{2}=\lVert e_{\alpha} \rVert^{2}+\lVert e_{\beta} \rVert^{2} =2 hence ∥eα−eβ∥=2\lVert e_{\alpha}-e_{\beta} \rVert =\sqrt{ 2 } By denseness of DD, for every eαe_{\alpha}, there exists some element xkα∈Dx_{k_{\alpha}}\in D, such that ∥eα−xkα∥<12.\lVert e_{\alpha}-x_{k_{\alpha}} \rVert < \frac{1}{\sqrt{ 2 }}.this is because we want to select xkαx_{k_{\alpha}} such that it is injective wrt eαe_{\alpha} and hence we need to restrict the radius using the following relation 2ϵ<2  ⟺  ϵ<12.2\epsilon< \sqrt{ 2 }\iff\epsilon< \frac{1}{\sqrt{ 2 }}. Let MM be the collection of such vectors {xkα}α∈A⊂D\{ x_{k_{\alpha}} \}_{\alpha \in \mathcal{A}}\subset D and observe that since it is a subset of DD, this set is countable as well. Now, for a given xkαx_{k_{\alpha}}, for any eβe_{\beta} with β≠α\beta \neq\alpha, we have, by the relation, ∥x∥−∥y∥≤∥x−y∥\lVert x \rVert-\lVert y \rVert\le \lVert x-y \rVert ∥eβ−eα∥−∥−(eα−xkα)∥≤∥eβ−eα−(−(eα−xkα))∥=∥eβ−xkα∥\lVert e_{\beta}-e_{\alpha} \rVert -\lVert -(e_{\alpha}-x_{k_{\alpha}}) \rVert \le \lVert e_{\beta}-e_{\alpha}-(-(e_{\alpha}-x_{k_{\alpha}})) \rVert =\lVert e_{\beta}-x_{k_{\alpha}} \rVert and thus 2−12=12≤∥eβ−xkα∥\sqrt{ 2 }- \frac{1}{\sqrt{ 2 }}= \frac{1}{\sqrt{ 2 }}\le \lVert e_{\beta}-x_{k_{\alpha}} \rVert Therefore, for every xkα∈M,!∃eα:d(xkα,eα)<12x_{k_{\alpha}}\in M, !\exists e_{\alpha}:d(x_{k_{\alpha}},e_{\alpha})< \frac{1}{\sqrt{ 2 }}. Thus, we can associate with every element in {eα}α∈A\{ e_{\alpha} \}_{\alpha \in \mathcal{A}} a unique element in the countable set M⊂DM\subset D. Thus, the set {eα}α∈A\{ e_{\alpha} \}_{\alpha \in \mathcal{A}} is countable. \end{proof} >[!def|2.3.5] Complete orthonormal sequence >An orthonormal sequence in a Hilbert space HH is complete if the only vector in HH which is orthogonal to each of the vectors is the null vector.

Lemma (2.3.4)

Let HH be a Hilbert space and let {ei}i∈N\{ e_{i} \}_{i\in \mathbb{N}} be an orthonormal sequence in HH. Then for any vector x∈Hx\in H ∑i=1∞∣⟨x,ei⟩∣2≤∥x∥2\sum_{i=1}^{\infty}|\langle x, e_{i} \rangle |^{2}\le \lVert x \rVert ^{2}

\begin{proof} For n∈Nn\in \mathbb{N} let xn=∑i=1n⟨x,ei⟩ei.x_{n}=\sum_{i=1}^{n}\langle x, e_{i} \rangle e_{i}.This vector xnx_{n} is the projection of xx onto the subspace spanned by the first nn basis vectors. The vector x−xnx-x_{n} is orthogonal to this subspace, and thus orthogonal to xnx_{n} itself. By the we have ∥x∥2=∥xn∥2+∥x−xn∥2.\lVert x \rVert ^{2}=\lVert x_{n} \rVert ^{2}+\lVert x-x_{n} \rVert ^{2}.The proof proceeds by computing ∥x−xn∥2\lVert x-x_{n} \rVert^{2}: 0≤∥x−∑i=1n⟨x,ei⟩ei∥2=⟨x−∑i=1n⟨x,ei⟩ei,x−∑j=1n⟨x,ej⟩ej⟩=∥x∥2−∑i=1n⟨x,ei⟩‾⟨x,ei⟩−∑j=1n⟨x,ej⟩⟨ej,x⟩+∑i=1n∑j=1n⟨x,ei⟩‾⟨x,ej⟩⟨ei,ej⟩=∥x∥2−∑i=1n∣⟨x,ei⟩∣2−∑j=1n∣⟨x,ej⟩∣2+∑i=1n∣⟨x,ei⟩∣2=∥x∥2−∑i=1n∣⟨x,ei⟩∣2.\begin{align*} 0&\le \left\lVert x-\sum_{i=1}^{n}\langle x, e_{i} \rangle e_{i} \right\rVert ^{2}\\ &= \left\langle x-\sum_{i=1}^{n}\langle x, e_{i} \rangle e_{i}, x-\sum_{j=1}^{n}\langle x, e_{j} \rangle e_{j} \right\rangle \\ &= \lVert x \rVert ^{2}-\sum_{i=1}^{n} \overline{\langle x, e_{i} \rangle }\langle x, e_{i} \rangle -\sum_{j=1}^{n}\langle x, e_{j} \rangle \langle e_{j}, x \rangle +\sum_{i=1}^{n}\sum_{j=1}^{n}\overline{\langle x, e_{i} \rangle }\langle x, e_{j} \rangle \langle e_{i}, e_{j} \rangle \\ &= \lVert x \rVert ^{2}-\sum_{i=1}^{n}\left| \langle x, e_{i} \rangle \right| ^{2}-\sum_{j=1}^{n}\left| \langle x, e_{j} \rangle \right| ^{2}+\sum_{i=1}^{n}\left| \langle x, e_{i} \rangle \right| ^{2}\\ &= \lVert x \rVert ^{2}-\sum_{i=1}^{n}\left| \langle x, e_{i} \rangle \right| ^{2} .\end{align*}This holds for any n≥1n\ge 1. Since the partial sums are non-negative and bounded above by ∥x∥2\lVert x \rVert^{2}, the series ∑i=1∞∣⟨x,ei⟩∣2\sum_{i=1}^{\infty}\left| \langle x, e_{i} \rangle \right|^{2} converges and its sum is less than or equal to ∥x∥2\lVert x \rVert^{2}. \end{proof}

Theorem (2.3.4)

A Hilbert space contains a complete orthonormal sequence if and only if it is separable.

\begin{proof}   ⟸  \impliedby: Let HH be separable. Then, there exists a countable dense subset D={x1,x2,… }D=\{ x_{1},x_{2},\dots \}. Apply the Gran-Schmidt process (212) to obtain {e1,e2,…,}\{ e_{1},e_{2},\dots, \}, an orthonormal basis. We claim that this set is a complete orthonormal sequence.

Suppose not; that is, let h∈Hh\in H be so that ∥h∥≠0\lVert h \rVert\neq 0 and let ⟨h,ek⟩=0,∀k∈N.(⭐®)\langle h, e_{k} \rangle=0,\forall k\in \mathbb{N}\tag{⭐️}.i.e., let hh be orthogonal to each eie_{i} and not the null vector. Now, for every ϵ>0\epsilon>0, there exists xm∈Dx_{m}\in D with ∥h−xm∥≤ϵ(3)\lVert h-x_{m} \rVert\le \epsilon\tag{3} and observe that xm=∑k=1mαiei(4)x_{m}=\sum_{k=1}^{m}\alpha_{i}e_{i}\tag{4}since the span of vectors {e1,…,em}\{ e_{1},\dots,e_{m} \} contains xmx_{m}. Then, ∥h∥2=⟨h,h⟩=⟨h−∑k=1mαiei,h⟩=⟨h−xm,h⟩≤∥h−xm∥∥h∥≤ϵ∥h∥\lVert h \rVert ^{2}=\langle h, h \rangle =\left\langle h-\sum_{k=1}^{m}\alpha_{i}e_{i}, h \right\rangle =\langle h-x_{m}, h \rangle \le \lVert h-x_{m} \rVert \lVert h \rVert \le \epsilon \lVert h \rVert which is pretty much: Inner product defines a norm (Yuksel), add subtract summation term and apply linearity of inner product then apply (⭐®)(⭐️), (4)(4), Cauchy-Schwarz Inequality (Yuksel), then (3)(3). This implies that ∥h∥≤ϵ\lVert h \rVert\le \epsilon. Since ϵ>0\epsilon>0 is arbitrary; then we have ∥h∥=0\lVert h \rVert=0, a contradiction hence ∥h∥=0\lVert h \rVert=0 and hh is the null element.

  ⟹  \implies Now, let HH have a complete orthonormal sequence {e1,e2,… }\{ e_{1},e_{2},\dots \}. We will show that D=⋃n∈N{x∈H:x=∑i=1nαiei,αi∈Q},D=\bigcup_{n\in \mathbb{N}}\left\{ x\in H: x=\sum_{i=1}^{n}\alpha_{i}e_{i},\alpha_{i}\in \mathbb{Q} \right\},is a

  1. countable
  2. dense subset in HH, and separability of HH follows by the definition.
  1. That DD is countable follows from the fact that for every nn, the set {x∈H:x=∑i=1nαiei,αi∈Q}\left\{ x\in H: x=\sum_{i=1}^{n}\alpha_{i}e_{i},\alpha_{i}\in \mathbb{Q} \right\} is countable as a Cartesian product of finitely many countable sets, and thus the countable union over n∈Nn\in \mathbb{N} leads to a countable set.

  2. We now show that this set is dense in HH. i.e., that ∀x∈H,∀ϵ>0,∃e∈D:∥x−e∥<ϵ\forall x\in H, \forall\epsilon>0, \exists e\in D:\lVert x-e\rVert<\epsilon where e=∑i=1Mαiei,αi∈Qe=\sum_{i=1}^{M}\alpha_{i}e_{i},\alpha_{i}\in \mathbb{Q}.

Consider the sequence of vectors xn:=∑i=1n⟨x,ei⟩eix_{n}:=\sum_{i=1}^{n}\langle x, e_{i} \rangle e_{i}. Observe that xnx_{n} converges to a limit by Theorem 2.3.1. To see that the sequence ∑i=1∞∣⟨x,ei⟩∣2\sum_{i=1}^{\infty}\left| \langle x, e_{i} \rangle \right|^{2} is summable, use to conclude ∑i=1∞∣⟨x,ei⟩∣2≤∥x∥2.\sum_{i=1}^{\infty}\left| \langle x, e_{i} \rangle \right| ^{2}\le \lVert x \rVert ^{2}.Therefore, the limit ∑i=1∞⟨x,ei⟩ei\sum_{i=1}^{\infty}\langle x, e_{i} \rangle e_{i} is well-defined and hence xn→xx_{n}\to x for some x∈Hx\in H.

Now, let h=x−∑i=1∞⟨x,ei⟩eih=x-\sum_{i=1}^{\infty} \langle x, e_{i} \rangle e_{i}. we claim that ∥h∥=0\lVert h \rVert=0. For any m∈Nm\in \mathbb{N}, ⟨h,em⟩=⟨x−∑i=1∞⟨x,ei⟩ei,em⟩=⟨x,em⟩−⟨lim⁡n→∞∑i=1n⟨x,ei⟩ei,em⟩=⟨x,em⟩−lim⁡n→∞⟨∑i=1n⟨x,ei⟩ei,em⟩=⟨x,em⟩−⟨x,em⟩=0\begin{align*}\langle h, e_{m} \rangle &= \left\langle x-\sum_{i=1}^{\infty}\langle x, e_{i} \rangle e_{i}, e_{m} \right\rangle \\ &= \langle x, e_{m} \rangle -\left\langle \lim_{ n \to \infty } \sum_{i=1}^{n}\langle x, e_{i} \rangle e_{i}, e_{m} \right\rangle \\ &= \langle x, e_{m} \rangle -\lim_{ n \to \infty } \left\langle \sum_{i=1}^{n}\langle x, e_{i} \rangle e_{i}, e_{m} \right\rangle \\ &= \langle x, e_{m} \rangle -\langle x, e_{m} \rangle \\ &= 0 \end{align*} But since {e1,e2,… }\{ e_{1},e_{2},\dots \} is a complete orthonormal sequence, it must be that ∥h∥=0\lVert h \rVert=0. Hence, x=∑i=1∞⟨x,ei⟩eix=\sum_{i=1}^{\infty}\langle x, e_{i} \rangle e_{i}. Now approximate this vector, by first truncating the sum so that ∀ϵ>0\forall\epsilon>0, ∥x−∑i=1N⟨x,ei⟩ei∥≤ϵ2\left\lVert x-\sum_{i=1}^{N}\langle x, e_{i} \rangle e_{i} \right\rVert \le \frac{\epsilon}{2}and then approximating the coefficients by rational numbers so that ∥∑i=1N⟨x,ei⟩ei−∑i=1Nαiei∥≤ϵ2,αi∈Q.\left\lVert \sum_{i=1}^{N}\langle x, e_{i} \rangle e_{i}-\sum_{i=1}^{N}\alpha_{i}e_{i} \right\rVert \le \frac{\epsilon}{2},\quad\alpha_{i}\in \mathbb{Q}.This implies ∥x−∑i=1Nαiei∥≤∥x−∑i=1N⟨x,ei⟩ei∥+∥∑i=1N⟨x,ei⟩ei−∑i=1Nαiei∥≤ϵ\begin{align*} \left\lVert x-\sum_{i=1}^{N}\alpha_{i}e_{i} \right\rVert &\le \left\lVert x-\sum_{i=1}^{N}\langle x, e_{i} \rangle e_{i} \right\rVert +\left\lVert \sum_{i=1}^{N}\langle x, e_{i} \rangle e_{i}-\sum_{i=1}^{N}\alpha_{i}e_{i} \right\rVert \\ &\le \epsilon \end{align*}Since for every ϵ\epsilon such an approximation can be made by some element in DD, this completes the proof.

\end{proof}

Theorem (2.3.5)

In a Hilbert space HH, a complete orthonormal sequence {en}n∈N\{ e_{n} \}_{n\in \mathbb{N}} defines a basis in the sense that for any x∈Hx\in H, we have x=lim⁡N→∞∑k=1N⟨x,ek⟩ekx=\lim_{ N \to \infty } \sum_{k=1}^{N}\langle x, e_{k} \rangle e_{k}

Linked from