Chapter 26

A Glimpse at Hilbert Spaces

26.1 The Projection Lemma, Duality

Given a Hermitian space !E, ϕ", we showed in Section 10.1 that the function # # : E → R defined such that
#u# = ϕ(u, u), is a norm on E. Thus, E is a normed vector space. If E is also complete, then it is a very
interesting space.
Recall that completeness has to do with the convergence of Cauchy sequences. A normed vector space
!E, # #" is automatically a metric space under the metric d defined such that d(u, v) = #v − u# (see the
Appendix in Chapter 17 Section 17.2, for the definition of a normed vector space and of a metric space, or
Lang [109, 110], or Dixmier [50]). Given a metric space E with metric d, a sequence (an )n≥1 of elements
an ∈ E is a Cauchy sequence iff for every " > 0, there is some N ≥ 1 such that

d(am , an ) < " for all m, n ≥ N.

We say that E is complete iff every Cauchy sequence converges to a limit (which is unique, since a metric
space is Hausdorff).
Every finite dimensional vector space over R or C is complete. For example, one can show by induction
that given any basis (e1 , . . . , en ) of E, the linear map h: Cn → E defined such that

h((z1 , . . . , zn )) = z1 e1 + · · · + zn en

is a homeomorphism (using the sup-norm on Cn ). One can also use the fact that any two norms on a finite
dimensional vector space over R or C are equivalent (see Lang [110], Dixmier [50], or Schwartz [149]).
However, if E has infinite dimension, it may not be complete. When a Hermitian space is complete, a
number of the properties that hold for finite dimensional Hermitian spaces also hold for infinite dimensional
spaces. For example, any closed subspace has an orthogonal complement, and in particular, a finite dimen-
sional subspace has an orthogonal complement. Hermitian spaces that are also complete play an important
role in analysis. Since they were first studied by Hilbert, they are called Hilbert spaces.

Definition 26.1.1 A (complex) Hermitian space !E, ϕ" which is a complete normed vector space under the
norm # # induced by ϕ is called a Hilbert space. A real Euclidean space !E, ϕ" which is complete under the
norm # # induced by ϕ is called a real Hilbert space.

All the results in this section hold for complex Hilbert spaces as well as for real Hilbert spaces. We
state all results for the complex case only, since they also apply to the real case, and since the proofs in the
complex case need a little more care.

765
766 CHAPTER 26. A GLIMPSE AT HILBERT SPACES

!∞Example 1. The space l2 of all countably infinite sequences x = (xi )i∈N of complex numbers such that
2 2 2
i=0 |xi | < ∞ is a Hilbert space. It will be shown later that the map ϕ: l × l → C defined such that

∞
"
ϕ ((xi )i∈N , (yi )i∈N ) = xi yi
i=0

is well defined, and that l2 is a Hilbert space under ϕ. In fact, we will prove a more general result (Lemma
26.3.2).
Example 2. The set C ∞ [a, b] of smooth functions f : [a, b] → C is a Hermitian space under the Hermitian
form # b
!f, g" = f (x)g(x)dx,
a

but it is not a Hilbert space because it is not complete. It is possible to construct its completion L2 ([a, b]),
which turns out to be the space of Lebesgue integrable functions on [a, b].

Remark: Given a Hermitian space E (with Hermitian product !−, −"), it is possible to construct a Hilbert
space Eh (with Hermitian product !−, −"h ) and a linear map j: E → Eh , such that

!u, v" = !j(u), j(v)"h

for all u, v ∈ E, and j(E) is dense in Eh . Furthermore, Eh is unique up to isomorphism. For details, see
Bourbaki [21].
One of the most important facts about finite-dimensional Hermitian (and Euclidean) spaces is that they
have orthonormal bases. This implies that, up to isomorphism, every finite-dimensional Hermitian space is
isomorphic to Cn (for some n ∈ N) and that the inner product is given by
n
"
!(x1 , . . . , xn ), (y1 , . . . , yn )" = xi yi .
i=1

Furthermore, every subspace W has an orthogonal complement W ⊥ , and the inner product induces a natural
duality between E and E ∗ , where E ∗ is the space of linear forms on E.
When E is a Hilbert space, E may be infinite dimensional, often of uncountable dimension. Thus, we
can’t expect that E always have an orthonormal basis. However, if we modify the notion of basis so that a
“Hilbert basis” is an orthogonal family that is also dense in E, i.e., every v ∈ E is the limit of a sequence
of finite combinations of vectors from the Hilbert basis, then we can recover most of the “nice” properties
of finite-dimensional Hermitian spaces. For instance, if (uk )k∈K is a Hilbert basis, for every v ∈!E, we can
define the Fourier coefficients ck = !v, uk "/#uk #, and then, v is the “sum” of its Fourier series k∈K ck uk .
However, the cardinality of the index set K can be very large, and it is necessary to define what it means
for a family of vectors indexed by K to be summable. We will do this in Section 26.2. It turns out that
every Hilbert space is isomorphic to a space of the form l2 (K), where l2 (K) is a generalization of the space
of Example 1 (see Theorem 26.3.7, usually called the Riesz-Fischer theorem).
Our first goal is to prove that a closed subspace of a Hilbert space has an orthogonal complement. We
also show that duality holds if we redefine the dual E & of E to be the space of continuous linear maps on
E. Our presentation closely follows Bourbaki [21]. We also were inspired by Rudin [145], Lang [109, 110],
Schwartz [149, 148], and Dixmier [50]. In fact, we highly recommend Dixmier [50] as a clear and simple text
on the basics of topology and analysis. We first prove the so-called projection lemma.
Recall that in a metric space E, a subset X of E is closed iff for every convergent sequence (xn ) of points
xn ∈ X, the limit x = limn→ ∞ xn also belongs to X. The closure X of X is the set of all limits of convergent
26.1. THE PROJECTION LEMMA, DUALITY 767

sequences (xn ) of points xn ∈ X. Obviously, X ⊆ X. We say that the subset X of E is dense in E iff
E = X, the closure of X, which means that every a ∈ E is the limit of some sequence (xn ) of points xn ∈ X.
Convex sets will again play a crucial role.
First, we state the following easy “parallelogram inequality”, whose proof is left as an exercise.
Lemma 26.1.2 If E is a Hermitian space, for any two vectors u, v ∈ E, we have
2 2 2 2
#u + v# + #u − v# = 2(#u# + #v# ).

From the above, we get the following lemma:

Lemma 26.1.3 If E is a Hermitian space, given any d, δ ∈ R such that 0 ≤ δ < d, let
B = {u ∈ E | #u# < d} and C = {u ∈ E | #u# ≤ d + δ}.
For any convex set such A that A ⊆ C − B, we have
√
#v − u# ≤ 12dδ,
for all u, v ∈ A (see Figure 26.1).

A
u

C B
v

Figure 26.1: Inequality of Lemma 26.1.3

$ $
Proof . Since A is convex, 12 (u + v) ∈ A if u, v ∈ A, and thus, $ 12 (u + v)$ ≥ d. From the parallelogram
inequality written in the form
$ $2 $ $2 % &
$1 $ $ $
$ (u + v)$ + $ 1 (u − v)$ = 1 #u#2 + #v#2 ,
$2 $ $2 $ 2
since δ < d, we get
$ $2 % & $ $2
$1 $ $ $
$ (u − v)$ = 1 #u#2 + #v#2 − $ 1 (u + v)$ ≤ (d + δ)2 − d2 = 2dδ + δ 2 ≤ 3dδ,
$2 $ 2 $ 2 $

from which √
#v − u# ≤ 12dδ.

If X is a nonempty subset of a metric space (E, d), for any a ∈ E, recall that we define the distance
d(a, X) of a to X as
d(a, X) = inf d(a, b).
b∈X
Also, the diameter δ(X) of X is defined by
δ(X) = sup{d(a, b) | a, b ∈ X}.
It is possible that δ(X) = ∞. We leave the following standard two facts as an exercise (see Dixmier [50]):
768 CHAPTER 26. A GLIMPSE AT HILBERT SPACES

Lemma 26.1.4 Let E be a metric space.

(1) For every subset X ⊆ E, δ(X) = δ(X).

(2) If E is a complete metric space, for every

' sequence (Fn ) of closed nonempty subsets of E such that
Fn+1 ⊆ Fn , if limn→∞ δ(Fn ) = 0, then ∞ n=1 Fn consists of a single point.

We are now ready to prove the crucial projection lemma.

Lemma 26.1.5 (Projection lemma) Let E be a Hilbert space.

(1) For any nonempty convex and closed subset X ⊆ E, for any u ∈ E, there is a unique vector pX (u) ∈ X
such that
#u − pX (u)# = inf #u − v# = d(u, X).
v∈X

(2) The vector pX (u) is the unique vector w ∈ E satisfying the following property (see Figure 26.2):

w∈X and - !u − w, z − w" ≤ 0 for all z ∈ X. (∗)

w
u X

Figure 26.2: Inequality of Lemma 26.1.5

Proof . (1) Let d = inf v∈X #u − v# = d(u, X). We define a sequence Xn of subsets of X as follows: for every
n ≥ 1, ( )
1
Xn = v ∈ X | #u − v# ≤ d + .
n
It is immediately verified that each Xn is nonempty (by definition of d), convex, and that Xn+1 ⊆ Xn . Also,
by Lemma 26.1.3, we have *
sup{#w − v# | v, w ∈ Xn } ≤ 12d/n,
' '
and thus, n≥1 Xn contains at most one point. We will prove that n≥1 Xn contains exactly one point,
namely, pX (u). For this, define a sequence (wn )n≥1 by picking some wn ∈ Xn for every n ≥ 1. We claim
that (wn )n≥1 is a Cauchy sequence. Given any " > 0, if we pick N such that

12d
N> ,
"2
since (Xn )n≥1 is a monotonic decreasing sequence, for all m, n ≥ N , we have
*
#wm − wn # ≤ 12d/N < ",
26.1. THE PROJECTION LEMMA, DUALITY 769

as desired. Since E is complete, the sequence (wn )n≥1 has a limit w, and since wn ∈ X and X is closed, we
must have w ∈ X. Also observe that

#u − w# ≤ #u − wn # + #wn − w# ,

and since w is the limit of (wn )n≥1 and

1
#u − wn # ≤ d + ,
n
given any " > 0, there is some n large enough so that
1 " "
< and #wn − w# ≤ ,
n 2 2
and thus
#u − w# ≤ d + ".
Since'the above holds for every " > 0, we have #u − w# = d. Thus, w ∈ Xn for all n ≥ 1, which proves
that n≥1 Xn = {w}. Now, any z ∈ X such that #u − z# = d(u, X) = d also belongs to every Xn , and thus
z = w, proving the uniqueness of w, which we denote as pX (u).
(2) Let w ∈ X. Since X is convex, z = (1 − λ)pX (u) + λw ∈ X for every λ, 0 ≤ λ ≤ 1. Then, we have

#u − z# ≥ #u − pX (u)#

for all λ, 0 ≤ λ ≤ 1, and since

2 2
#u − z# = #u − pX (u) − λ(w − pX (u))#
2 2
= #u − pX (u)# + λ2 #w − pX (u)# − 2λ- !u − pX (u), w − pX (u)" ,

for all λ, 0 < λ ≤ 1, we get

1 % 2 2
& λ
2
- !u − pX (u), w − pX (u)" = #u − pX (u)# − #u − z# + #w − pX (u)# ,
2λ 2
and since this holds for every λ, 0 < λ ≤ 1 and

#u − z# ≥ #u − pX (u)# ,

we have
- !u − pX (u), w − pX (u)" ≤ 0.

Conversely, assume that w ∈ X satisfies the condition

- !u − w, z − w" ≤ 0

for all z ∈ X. For all z ∈ X, we have

2 2 2 2
#u − z# = #u − w# + #z − w# − 2- !u − w, z − w" ≥ #u − w# ,

which implies that #u − w# = d(u, X) = d, and from (1), that w = pX (u).

The vector pX (u) is called the projection of u onto X, and the map pX : E → X is called the projection of
E onto X. In the case of a real Hilbert space, there is an intuitive geometric interpretation of the condition

!u − pX (u), z − pX (u)" ≤ 0
770 CHAPTER 26. A GLIMPSE AT HILBERT SPACES

for all z ∈ X. If we restate the condition as

!u − pX (u), pX (u) − z" ≥ 0

for all z ∈ X, this says that the absolute value of the measure of the angle between the vectors u − pX (u)
and pX (u) − z is at most π/2. This makes sense, since X is convex, and points in X must be on the side
opposite to the “tangent space” to X at pX (u), which is orthogonal to u − pX (u). Of course, this is only an
intuitive description, since the notion of tangent space has not been defined!
The map pX : E → X is continuous, as shown below.

Lemma 26.1.6 Let E be a Hilbert space. For any nonempty convex and closed subset X ⊆ E, the map
pX : E → X is continuous.

Proof . For any two vectors u, v ∈ E, let x = pX (u) − u, y = pX (v) − pX (u), and z = v − pX (v). Clearly,

v − u = x + y + z,

and from Lemma 26.1.5 (2), we also have

- !x, y" ≥ 0 and - !z, y" ≥ 0,

from which we get

2 2 2
#v − u# = #x + y + z# = #x + z + y#
2 2
= #x + z# + #y# + 2- !x, y" + 2- !z, y"
2 2
≥ #y# = #pX (v) − pX (u)# .

However, #pX (v) − pX (u)# ≤ #v − u# obviously implies that pX is continuous.

We can now prove the following important lemma.

Lemma 26.1.7 Let E be a Hilbert space.

(1) For any closed subspace V ⊆ E, we have E = V ⊕V ⊥ , and the map pV : E → V is linear and continuous.
(2) The projection pV (u) is the unique vector w ∈ E such that

w∈V and !u − w, z" = 0 for all z ∈ V .

Proof . (1) First, we prove that u − pV (u) ∈ V ⊥ for all u ∈ E. For any v ∈ V , since V is a subspace,
z = pV (u) + λv ∈ V for all λ ∈ C, and since V is convex and nonempty (since it is a subspace), and closed
by hypothesis, by Lemma 26.1.5 (2), we have

-(λ !u − pV (u), v") = -(!u − pV (u), λv" = - !u − pV (u), z − pV (u)" ≤ 0

for all λ ∈ C. In particular, the above holds for λ = !u − pV (u), v", which yields

| !u − pV (u), v" | ≤ 0,

and thus, !u − pV (u), v" = 0. As a consequence, u − pV (u) ∈ V ⊥ for all u ∈ E. Since u = pV (u) + u − pV (u)
for every u ∈ E, we have E = V + V ⊥ . On the other hand, since !−, −" is positive definite, V ∩ V ⊥ = {0},
and thus E = V ⊕ V ⊥ .
We already proved in Lemma 26.1.6 that pV : E → V is continuous. Also, since

pV (λu + µv) − (λpV (u) + µpV (v)) = pV (λu + µv) − (λu + µv) + λ(u − pV (u)) + µ(v − pV (v)),
26.1. THE PROJECTION LEMMA, DUALITY 771

for all u, v ∈ E, and since the left-hand side term belongs to V , and from what we just showed, the right-hand
side term belongs to V ⊥ , we have

pV (λu + µv) − (λpV (u) + µpV (v)) = 0,

showing that pV is linear.

(2) This is basically obvious from (1). We proved in (1) that u − pV (u) ∈ V ⊥ , which is exactly the
condition
!u − pV (u), z" = 0
for all z ∈ V . Conversely, if w ∈ V satisfies the condition

!u − w, z" = 0

for all z ∈ V , since w ∈ V , every vector z ∈ V is of the form y − w, with y = z + w ∈ V , and thus, we have

!u − w, y − w" = 0

for all y ∈ V , which implies the condition of Lemma 26.1.5 (2):

- !u − w, y − w" ≤ 0

for all y ∈ V . By Lemma 26.1.5, w = pV (u) is the projection of u onto V .

Let us illustrate the power of Lemma 26.1.7 on the following “least squares” problem. Given a real
m × n-matrix A and some vector b ∈ Rm , we would like to solve the linear system

Ax = b

in the least-squares sense, which means that we would like to find some solution x ∈ Rn that minimizes the
Euclidean norm #Ax − b# of the error Ax − b. It is actually not clear that the problem has a solution, but
it does! The problem can be restated as follows: Is there some x ∈ Rn such that

#Ax − b# = infn #Ay − b# ,

y∈R

or equivalently, is there some z ∈ Im (A) such that

#z − b# = d(b, Im (A)),

where Im (A) = {Ay ∈ Rm | y ∈ Rn }, the image of the linear map induced by A. Since Im (A) is a closed
subspace of Rm , because we are in finite dimension, Lemma 26.1.7 tells us that there is a unique z ∈ Im (A)
such that
#z − b# = infn #Ay − b# ,
y∈R

and thus, the problem always has a solution since z ∈ Im (A), and since there is at least some x ∈ Rn such
that Ax = z (by definition of Im (A)). Note that such an x is not necessarily unique. Furthermore, Lemma
26.1.7 also tells us that z ∈ Im (A) is the solution of the equation

!z − b, w" = 0 for all w ∈ Im (A),

or equivalently, that x ∈ Rn is the solution of

!Ax − b, Ay" = 0 for all y ∈ Rn ,

which is equivalent to
!A( (Ax − b), y" = 0 for all y ∈ Rn ,
772 CHAPTER 26. A GLIMPSE AT HILBERT SPACES

and thus, since the inner product is positive definite, to A( (Ax − b) = 0, i.e.,

A( Ax = A( b.

Therefore, the solutions of the original least-squares problem are precisely the solutions of the the so-called
normal equations
A( Ax = A( b,
discovered by Gauss and Legendre around 1800. We also proved that the normal equations always have a
solution.
Computationally, it is best not to solve the normal equations directly, and instead, to use methods such
as the QR-decomposition (applied to A) or the SVD-decomposition (in the form of the pseudo-inverse). We
will come back to this point later on.
As an other corollary of Lemma 26.1.7, for any continuous nonnull linear map h: E → C, the null space

H = Ker h = {u ∈ E | h(u) = 0} = h−1 (0)

is a closed hyperplane H, and thus, H ⊥ is a subspace of dimension one such that E = H ⊕H ⊥ . This suggests
defining the dual space of E as the set of all continuous maps h: E → C.

Remark: If h: E → C is a linear map which is not continuous, then it can be shown that the hyperplane
H = Ker h is dense in E! Thus, H ⊥ is reduced to the trivial subspace {0}. This goes against our intuition
of what a hyperplane in Rn (or Cn ) is, and warns us not to trust our “physical” intuition too much when
dealing with infinite dimensions. As a consequence, the map &r : E → E ∗ defined in Section 10.2 is not
surjective, since the linear forms of the form u 1→ !u, v" (for some fixed vector v ∈ E) are continuous (the
inner product is continuous).
We now show that by redefining the dual space of a Hilbert space as the set of continuous linear forms
on E, we recover Lemma 10.2.1.

Definition 26.1.8 Given a Hilbert space E, we define the dual space E & of E as the vector space of all
continuous linear forms h: E → C. Maps in E & are also called bounded linear operators, bounded linear
functionals, or simply, operators or functionals.

As in Section 10.2, for all u, v ∈ E, we define the maps ϕlu : E → C and ϕrv : E → C such that

ϕlu (v) = !u, v" ,

and
ϕrv (u) = !u, v" .
Because the inner product !−, −" is continuous, it is obvious that ϕlu is continuous and semilinear, so that
&
ϕlu ∈ E , and ϕrv is continuous and linear, so that ϕrv ∈ E & .
Lemma 10.2.1 is generalized to Hilbert spaces as follows.

Lemma 26.1.9 (Riesz representation theorem) Let E be a Hilbert space. Then, the following properties
hold:
&
(1) The map &l : E → E defined such that
&l (u) = ϕlu ,
is linear, continuous, and bijective.
26.2. TOTAL ORTHOGONAL FAMILIES (HILBERT BASES), FOURIER COEFFICIENTS 773

(2) The map &r : E → E & defined such that

&r (v) = ϕrv ,
is semilinear, continuous, and bijective.

Proof . The proof is basically identical to the proof of Lemma 10.2.1, except that a different argument is
&
required for the surjectivity of &r : E → E & and &l : E → E , since E may not be finite dimensional. For any
nonnull linear operator h ∈ E & , the hyperplane H = Ker h = h−1 (0) is a closed subspace of E, and by
Lemma 26.1.7, H ⊥ is a subspace of dimension one such that E = H ⊕ H ⊥ . Then, picking any nonnull vector
w ∈ H ⊥ , observe that H is also the kernel of the linear operator ϕrw , with

ϕrw (u) = !u, w" ,

and thus, by Lemma 17.1.1, there is some nonnull scalar λ ∈ C such that h = λϕrw . But then, h = ϕrλw ,
&
proving that &r : E → E & is surjective. A similar argument proves that &l : E → E is surjective.

Lemma 26.1.9 is known as the Riesz representation theorem, or “Little Riesz Theorem”. It shows that
the inner product on a Hilbert space induces a natural linear isomorphism between E and its dual E & .

Remarks:
(1) Actually, the map &r : E → E & turns out to be an isometry. To show this, we need to recall the notion
of norm of a linear map, which we do not want to do right now.
(2) Many books on quantum mechanics use the so-called Dirac notation to denote objects in the Hilbert
space E and operators in its dual space E & . In the Dirac notation, an element of E is denoted as
|x", and an element of E & is denoted as !t|. The scalar product is denoted as !t| · |x". This uses the
isomorphism between E and E & , except that the inner product is assumed to be semi-linear on the left,
rather than on the right.

Lemma 26.1.9 allows us to define the adjoint of a linear map, as in the Hermitian case (see Lemma
10.2.2). The proof is unchanged.

Lemma 26.1.10 Given a Hilbert space E, for every linear map f : E → E, there is a unique linear map
f ∗ : E → E, such that
f ∗ (u) · v = u · f (v),
for all u, v ∈ E. The map f ∗ is called the adjoint of f .

It is easy to show that if f is continuous, then f ∗ is also continuous. As in the Hermitian case, given two
Hilbert spaces E and F , for any linear map f : E → F , such that

!f (u), v"2 = !u, f ∗ (v)"1

for all u ∈ E and all v ∈ F . The linear map f ∗ is also called the adjoint of f .

26.2 Total Orthogonal Families (Hilbert Bases),

Fourier Coefficients
We conclude our quick tour of Hilbert spaces by showing that the notion of orthogonal basis can be generalized
to Hilbert spaces. However, the useful notion is not the usual notion of a basis, but a notion which is an
abstraction of the concept of Fourier series. Every element of a Hilbert space is the “sum” of its Fourier
series.
774 CHAPTER 26. A GLIMPSE AT HILBERT SPACES

Definition 26.2.1 Given a Hilbert space E, a family (uk )k∈K of nonnull vectors is an orthogonal family
iff the uk are pairwise orthogonal, i.e., !ui , uj " = 0 for all i 2= j (i, j ∈ K), and an orthonormal family iff
!ui , uj " = δi, j , for all i, j ∈ K. A total orthogonal family (or system) or Hilbert basis is an orthogonal family
that is dense in E. This means that for every v ∈ E, for every " > 0, there is some finite subset I ⊆ K and
some family (λi )i∈I of complex numbers, such that
$ $
$ " $
$ $
$v − λi ui $ < ".
$ $
i∈I

2
Given an orthogonal family (uk )k∈K , for every v ∈ E, for every k ∈ K, the scalar ck = !v, uk " / #uk # is
called the k-th Fourier coefficient of v over (uk )k∈K .

Remark: The terminology Hilbert basis is misleading, because a Hilbert basis (uk )k∈K is not necessarily
a basis in the algebraic sense. Indeed, in general, (uk )k∈K does not span E. Intuitively, it takes linear
combinations of the uk ’s with infinitely many nonnull coefficients to span E. Technically, this is achieved
in terms of limits. In order to avoid the confusion between bases in the algebraic sense and Hilbert bases,
some authors refer to algebraic bases as Hamel bases and to total orthogonal families (or Hilbert bases) as
Schauder bases.
!
the form i∈I λi ui
Given an orthogonal family (uk )k∈K , for any finite subset I of K, we often call sums of!
partial sums of Fourier series, and if these partial sums converge to a limit denoted as k∈K ck uk , we call
!
k∈K ck uk a Fourier series.

However, we have to make sense of such sums! Indeed, when K is unordered or uncountable, the notion
of limit or sum has not been defined. This can be done as follows (for more details, see Dixmier [50]):

Definition 26.2.2 Given a normed vector space E (say, a Hilbert space), for any nonempty index set K,
we say that a family (uk )k∈K of vectors in E is summable with sum v ∈ E iff for every " > 0, there is some
finite subset I of K, such that, $ $
$ $
$ " $
$v − uj $
$ $<"
$ j∈J $

for every finite subset J with I ⊆ J ⊆ K. We say that the family (uk )k∈K is summable iff there is some
v ∈ E such that (uk )k∈K is summable with sum v. A family (uk )k∈K is a Cauchy family iff for every " > 0,
there is a finite subset I of K, such that, $ $
$" $
$ $
$ uj $
$ $<"
$j∈J $

for every finite subset J of K with I ∩ J = ∅,

!
If (uk )k∈K is summable with sum v, we usually denote v as k∈K uk . The following technical lemma
will be needed:

Lemma 26.2.3 Let E be a complete normed vector space (say, a Hilbert space).

(1) For any nonempty index set K, a family (uk )k∈K is summable iff it is a Cauchy family.

(2) Given
! a family (rk )k∈K of nonnegative reals rk ≥ 0, if there is some real number
! B > 0 such that
i∈I ri < B for every finite subset I of K,
! then (rk )k∈K is summable and k∈K rk = r, where r is
least upper bound of the set of finite sums i∈I ri (I ⊆ K).
26.2. TOTAL ORTHOGONAL FAMILIES (HILBERT BASES), FOURIER COEFFICIENTS 775

Proof . (1) If (uk )k∈K is summable, for every finite subset I of K, let
" "
uI = ui and u = uk
i∈I k∈K

For every " > 0, there is some finite subset I of K such that

#u − uL # < "/2

for all finite subsets L such that I ⊆ L ⊆ K. For every finite subset J of K such that I ∩ J = ∅, since
I ⊆ I ∪ J ⊆ K and I ∪ J is finite, we have

#u − uI∪J # < "/2 and #u − uI # < "/2,

and since
#uI∪J − uI # ≤ #uI∪J − u# + #u − uI #
and uI∪J − uI = uJ since I ∩ J = ∅, we get

#uJ # = #uI∪J − uI # < ",

which is the condition for (uk )k∈K to be a Cauchy family.

Conversely, assume that (uk )k∈K is a Cauchy family. We define inductively a decreasing sequence (Xn )
of subsets of E, each of diameter at most 1/n, as follows: For n = 1, since (uk )k∈K is a Cauchy family, there
is some finite subset J1 of K such that
#uJ # < 1/2
for every finite subset J of K with J1 ∩ J = ∅. We pick some finite subset J1 with the above property, and
we let I1 = J1 and
X1 = {uI | I1 ⊆ I ⊆ K, I finite}.
For n ≥ 1, there is some finite subset Jn+1 of K such that

#uJ # < 1/(2n + 2)

for every finite subset J of K with Jn+1 ∩ J = ∅. We pick some finite subset Jn+1 with the above property,
and we let In+1 = In ∪ Jn+1 and

Xn+1 = {uI | In+1 ⊆ I ⊆ K, I finite}.

Since In ⊆ In+1 , it is obvious that Xn+1 ⊆ Xn for all n ≥ 1. We need to prove that each Xn has diameter
at most 1/n. Since Jn was chosen such that

#uJ # < 1/(2n)

for every finite subset J of K with Jn ∩ J = ∅, and since Jn ⊆ In , it is also true that

#uJ # < 1/(2n)

for every finite subset J of K with In ∩ J = ∅ (since In ∩ J = ∅ and Jn ⊆ In implies that Jn ∩ J = ∅). Then,
for every two finite subsets J, L such that In ⊆ J, L ⊆ K, we have

#uJ−In # < 1/(2n) and #uL−In # < 1/(2n),

and since
#uJ − uL # ≤ #uJ − uIn # + #uIn − uL # = #uJ−In # + #uL−In # ,
776 CHAPTER 26. A GLIMPSE AT HILBERT SPACES

we get
#uJ − uL # < 1/n,
which proves that δ(Xn ) ≤ 1/n. Now, if we consider the sequence of closed sets (Xn ), we still have Xn+1 ⊆
Xn , and
'∞by Lemma 26.1.4, δ(Xn ) = δ(Xn ) ≤ 1/n, which means that limn→∞ δ(Xn ) = 0, and by Lemma
26.1.4, n=1 Xn consists of a single element u. We claim that u is the sum of the family (uk )k∈K .
For every " > 0, there is some n ≥ 1 such that n > 2/", and since u ∈ Xm for all m ≥ 1, there is some
finite subset J0 of K such that In ⊆ J0 and

#u − uJ0 # < "/2,

where In is the finite subset of K involved in the definition of Xn . However, since δ(Xn ) ≤ 1/n, for every
finite subset J of K such that In ⊆ J, we have

#uJ − uJ0 # ≤ 1/n < "/2,

and since
#u − uJ # ≤ #u − uJ0 # + #uJ0 − uJ # ,
we get
#u − uJ # < "
for every finite subset J of K with In ⊆ J, which proves that u is the sum of the family (uk )k∈K .
!
(2) Since every finite sum i∈I ri is bounded by the uniform bound B, the set of these finite
! sums has a
least upper bound r ≤ B. For every " > 0, since r is the least upper bound of the finite sums i∈I ri (where
I finite, I ⊆ K), there is some finite I ⊆ K such that
+ +
+ " ++
+
+r − ri + < ",
+ +
i∈I

and since rk ≥ 0 for all k ∈ K, we have " "

ri ≤ rj
i∈I j∈J

whenever I ⊆ J, which shows that + + +

+ + +
+ " + ++ " ++
+r − rj ++ ≤ +r − ri + < "
+
+ j∈J + + i∈I
+
!
for every finite subset J such that I ⊆ J ⊆ K, proving that (rk )k∈K is summable with sum k∈K rk = r.

Remark: The notion of summability implies that the sum of a family (uk )k∈K is independent of any order
on K. In this sense, it is a kind of “commutative summability”. More precisely, it is easy to show that for
every bijection ϕ: K → K (intuitively, a reordering of K), the family (uk )k∈K is summable iff the family
(ul )l∈ϕ(K) is summable, and if so, they have the same sum.
The following lemma gives some of the main properties of Fourier coefficients. Among other things, at
most countably many of the Fourier coefficient may be nonnull, and the partial sums of a Fourier series
converge. Given an orthogonal family (uk )k∈K , we let Uk = Cuk , and pUk : E → Uk is the projection of E
onto Uk .

Lemma 26.2.4 Let E be a Hilbert space, (uk )k∈K an orthogonal family in E, and V the closure of the
subspace generated by (uk )k∈K . The following properties hold:
26.2. TOTAL ORTHOGONAL FAMILIES (HILBERT BASES), FOURIER COEFFICIENTS 777

(1) For every v ∈ E, for every finite subset I ⊆ K, we have

" 2
|ci |2 ≤ #v# ,
i∈I

where the ck are the Fourier coefficients of v. As a consequence, at most countably many of the ck may
be nonnull.
(2) For every vector v ∈ E, if (ck )k∈K are the Fourier coefficients of v, the following conditions are
equivalent:
(2a) v ∈ V
!
(2b) The family (ck uk )k∈K is summable and v = k∈K ck u k .
2 !
(2c) The family (|ck |2 )k∈K is summable and #v# = k∈K |ck |2 ;
(3) The family (|ck |2 )k∈K is summable, and we have the Bessel inequality:
"
|ck |2 ≤ #v#2 .
k∈K
!
The family (ck uk )k∈K
! forms a Cauchy family, and thus, the Fourier series k∈K ck uk converges in E
to some vector u = k∈K ck uk . Furthermore, u = pV (v).

Proof . (1) Let "

uI = ci u i
i∈I

for any finite subset I of K. We claim that v − uI is orthogonal to ui for every i ∈ I. Indeed,
, -
"
!v − uI , ui " = v − cj u j , u i
j∈I
"
= !v, ui " − cj !uj , ui "
j∈I

= !v, ui " − ci #ui #2

= !v, ui " − !v, ui " = 0,

since !uj , ui " = 0 for all i 2= j and ci = !v, ui " / #ui #2 . As a consequence, we have
$ $2
$ " " $
2 $ $
#v# = $v − ci u i + ci u i $
$ $
i∈I i∈I
$ $2 $ $2
$ " $ $" $
$ $ $ $
= $v − ci u i $ + $ ci u i $
$ $ $ $
i∈I i∈I
$ $2
$ " $ "
$ $
= $v − ci u i $ + |ci |2 ,
$ $
i∈I i∈I

since the ui are pairwise orthogonal, that is,

$ $2
$ " $ "
$2 $
#v# = $v − ci u i $ + |ci |2 .
$ $
i∈I i∈I
778 CHAPTER 26. A GLIMPSE AT HILBERT SPACES

Thus, " 2
|ci |2 ≤ #v# ,
i∈I

as claimed. Now, for every natural number n ≥ 1, if Kn is the subset of K consisting of all ck such that
|ck | ≥ 1/n, the number of elements in Kn is at most
" " 2
|nck |2 ≤ n2 |ck |2 ≤ n2 #v# ,
k∈Kn k∈Kn

which is finite, and thus, at most a countable number of the ck may be nonnull.
(2) We prove the chain of implications (a) ⇒ (b) ⇒ (c) ⇒ (a).
(a) ⇒ (b): If v ∈ V , since V is the closure of the subspace spanned by (uk )k∈K , for every " > 0, there is
some finite subset I of K and some family (λi )i∈I of complex numbers, such that
$ $
$ " $
$ $
$v − λi ui $ < ".
$ $
i∈I

Now, for every finite subset J of K such that I ⊆ J, we have

$ $2 $$
$2
$
$ " $ $ " " " $
$ $ $
$v − λi ui $ = $v − cj u j + cj u j − λi ui $
$
$ $ $ $
i∈I j∈J j∈J i∈I
$ $2 $ $2
$ " $ $" " $
$ $ $ $
=$
$v − c u $
j j$ + $
$ c u
j j − λ u $
i i$ ,
$ j∈J $ $j∈J i∈I $
!
since I ⊆ J and the uj (with j ∈ J) are orthogonal to v − j∈J cj uj by the argument in (1), which shows
that $ $ $ $
$ " $ $ " $
$ $ $ $
$v − c u $ ≤ $v − λ u $ < ",
$ j j $ $ i i
$
$ j∈J $ i∈I

and thus, that the family (uk )k∈K is summable with sum v, so that
"
v= ck u k .
k∈K

!
(b) ⇒ (c): If v = k∈K ck uk , then for every " > 0, there some finite subset I of K, such that
$ $
$ $
$ " $ √
$v − cj u j $
$ $ < ",
$ j∈J $

for every finite subset J of K such that I ⊆ J, and since we proved in (1) that
$ $2
$ $
2 $ " $ "
$
#v# = $v − cj u j $ |cj |2 ,
$ +
$ j∈I $ j∈J

we get "
2
#v# − |cj |2 < ",
j∈J
26.2. TOTAL ORTHOGONAL FAMILIES (HILBERT BASES), FOURIER COEFFICIENTS 779

which proves that (|ck |2 )k∈K is summable with sum #v#2 .

2
(c) ⇒ (a): Finally, if (|ck |2 )k∈K is summable with sum #v# , for every " > 0, there is some finite subset
I of K such that "
2
#v# − |cj |2 < "2
j∈J

for every finite subset J of K such that I ⊆ J, and again, using the fact that
$ $2
$ " $ "
2 $ $
$
#v# = $v − cj u j $ |cj |2 ,
$ +
$ j∈J $ j∈J

we get $ $
$ $
$ " $
$v − c $
j j $ < ",
u
$
$ j∈J $
!
which proves that (ck uk )k∈K is summable with sum k∈K ck uk = v, and v ∈ V .
! 2
(3) Since i∈I |ci |2 ≤ #v# for every finite subset I of K, by Lemma 26.2.3, the family (|ck |2 )k∈K is
summable. The Bessel inequality "
|ck |2 ≤ #v#2
k∈K
!
is an obvious consequence of the inequality i∈I |ci |2 ≤ #v#2 (for every finite I ⊆ K). Since (|ck |2 )k∈K is
summable with sum c, for every " > 0, there is some finite subset I of K such that
"
|cj |2 < "2
j∈J

for every finite subset J of K such that I ∩ J = ∅. Since

$ $2
$" $ "
$ $
$ cj u j $ |cj |2 ,
$ $ =
$j∈J $ j∈J

we get $ $
$ $
$" $
$ c u $
j j $ < ".
$
$j∈J $

This proves that (ck uk )k∈K is a Cauchy family,!

which, by Lemma 26.2.3, implies that (ck uk )k∈K is summable,
since E is complete. Thus, the Fourier series k∈K ck uk is summable, with its sum denoted u ∈ V .
!
Since k∈K ck uk is summable with sum u, for every " > 0, there is some finite subset I1 of K such that
$ $
$ $
$ " $
$u − c $
j j$ < "
u
$
$ j∈J $

for every finite subset J of K such that I1 ⊆ J. By the triangle inequality, for every finite subset I of K,
$ $ $ $
$ " $ $" $
$ $ $ $
#u − v# ≤ $u − ci u i $ + $ ci u i − v $ .
$ $ $ $
i∈I i∈I
780 CHAPTER 26. A GLIMPSE AT HILBERT SPACES

!
By (2), every w ∈ V is the sum of its Fourier series k∈K λk uk , and for every " > 0, there is some finite
subset I2 of K such that $ $
$ $
$ " $
$w − λj uj $
$ $<"
$ j∈J $
for every finite subset J of K such that I2 ⊆ J. By the triangle inequality, for every finite subset I of K,
$ $ $ $
$ " $ $ " $
$ $ $ $
$v − λi ui $ ≤ #v − w# + $w − λi ui $ .
$ $ $ $
i∈I i∈I

Letting I = I1 ∪ I2 , since we showed in (2) that

$ $ $ $
$ " $ $ " $
$ $ $ $
$v − ci ui $ ≤ $v − λi ui $
$ $ $ $
i∈I i∈I

for every finite subset I of K, we get

$ $ $ $
$ " $ $" $
$ $ $ $
#u − v# ≤ $u − ci u i $ + $ ci u i − v $
$ $ $ $
i∈I i∈I
$ $ $ $
$ " $ $" $
$ $ $ $
≤ $u − ci u i $ + $ λi ui − v $
$ $ $ $
i∈I i∈I
$ $ $ $
$ " $ $ " $
$ $ $ $
≤ $u − ci ui $ + #v − w# + $w − λi ui $ ,
$ $ $ $
i∈I i∈I

and thus
#u − v# ≤ #v − w# + 2".
Since this holds for every " > 0, we have

#u − v# ≤ #v − w#

for all w ∈ V , i.e. #v − u# = d(v, V ), with u ∈ V , which proves that u = pV (v).

26.3 The Hilbert Space l2(K) and the Riesz-Fischer Theorem

Lemma 26.2.4 suggests looking at the space of sequences (zk )k∈K (where zk ∈ C) such that (|zk |2 )k∈K is
summable. Indeed, such spaces are Hilbert spaces, and it turns out that every Hilbert space is isomorphic
to one of those. Such spaces are the infinite-dimensional version of the spaces Cn under the usual Euclidean
norm.

Definition 26.3.1 Given any nonempty index set K,!the space l2 (K) is the set of all sequences (zk )k∈K ,
where zk ∈ C, such that (|zk |2 )k∈K is summable, i.e., k∈K |zk |2 < ∞.

Remarks:
(1) When K is a finite set of cardinality n, l2 (K) is isomorphic to Cn .
(2) When K = N, the space l2 (N) corresponds to the space l2 of example 2 in Section 10.1. In that
example, we claimed that l2 was a Hermitian space, and in fact, a Hilbert space. We now prove this
fact for any index set K.
26.3. THE HILBERT SPACE L2 (K) AND THE RIESZ-FISCHER THEOREM 781

Lemma 26.3.2 Given any nonempty index set K, the space l2 (K) is a Hilbert space under the Hermitian
product "
!(xk )k∈K , (yk )k∈K " = xk yk .
k∈K

The subspace consisting of sequences (zk )k∈K such that zk = 0, except perhaps for finitely many k, is a dense
subspace of l2 (K).

Proof . First, we need to prove that l2 (K) is a vector space. Assume that (xk )k∈K and (yk )k∈K are in l2 (K).
This means that (|xk |2 )k∈K and (|yk |2 )k∈K are summable, !which, in view of Lemma
! 26.2.3, is equivalent to
the existence of some positive bounds A and B such that i∈I |xi |2 < A and i∈I |yi |2 < B, for every finite
subset I of!K. To prove that (|xk + yk |2 )k∈K is summable, it is sufficient to prove that there is some C > 0
such that i∈I |xi + yi |2 < C for every finite subset I of K. However, the parallelogram inequality implies
that " "
|xi + yi |2 ≤ 2(|xi |2 + |yi |2 ) ≤ 2(A + B),
i∈I i∈I

for every finite subset I of K, and we conclude by Lemma 26.2.3. Similarly, for every λ ∈ C,
" "
|λxi |2 ≤ |λ|2 |xi |2 ≤ |λ|2 A,
i∈I i∈I

and (λk xk )k∈K is summable. Therefore, l2 (K) is a vector space.

By the Cauchy-Schwarz inequality,
" " "
|xi yi | ≤ |xi ||yi | ≤ (|xi |2 + |yi |2 )/2 ≤ (A + B)/2,
i∈I i∈I i∈I

for every finite subset I of K. Thus, by Lemma 26.2.3, (|xk yk |)k∈K is summable (We say that (xk yk )k∈K
is absolutely summable). However, it is a standard fact that this implies that (xk yk )k∈K is summable (For
every " > 0, there is some finite subset I of K such that
"
|xj yj | < "
j∈J

for every finite subset J of K such that I ∩ J = ∅, and thus

" "
| xj yj | ≤ |xj yj | < ",
j∈J i∈J

proving that (xk yk )k∈K is a Cauchy family, and thus summable). We still have to prove that l2 (K) is
complete.
Consider a sequence ((λnk )k∈K )n≥1 of sequences (λnk )k∈K ∈ l2 (K), and assume that it is a Cauchy
sequence. This means that for every " > 0, there is some N ≥ 1 such that
"
|λm n 2
k − λk | < "
2

k∈K

for all m, n ≥ N . For every fixed k ∈ K, this implies that

|λm n
k − λk | < "

for all m, n ≥ N , which shows that (λnk )n≥1 is a Cauchy sequence in C. Since C is complete, the sequence
(λnk )n≥1 has a limit λk ∈ C. We claim that (λk )k∈K ∈ l2 (K) and that this is the limit of ((λnk )k∈K )n≥1 .
782 CHAPTER 26. A GLIMPSE AT HILBERT SPACES

Given any " > 0, the fact that ((λnk )k∈K )n≥1 is a Cauchy sequence implies that there is some N ≥ 1 such
that for every finite subset I of K, we have
"
|λm n 2
i − λi | < "/4
i∈I

for all m, n ≥ N . Let p = |I|. Then, √

"
|λm
i − λn
i | < √
2 p
for every i ∈ I. Since λi is the limit of (λni )n≥1 , we can find some n large enough so that
√
"
|λni − λi | < √
2 p
for every i ∈ I. Since
|λm m n n
i − λi | ≤ |λi − λi | + |λi − λi |,
we get √
"
|λm
i − λi | < √ ,
p
and thus, "
|λm 2
i − λi | < ",
i∈I
for all m ≥ N . Since the above holds for every finite subset I of K, by Lemma 26.2.3, we get
"
|λm 2
k − λk | < ",
k∈K

for all m ≥ N . This proves that (λm

k− λk )k∈K ∈ l2 (K) for all m ≥ N , and since l2 (K) is a vector space and
(λk )k∈K ∈ l (K) for all m ≥ 1, we get (λk )k∈K ∈ l2 (K). However,
m 2

"
|λm 2
k − λk | < "
k∈K

for all m ≥ N , means that the sequence (λm converges to (λk )k∈K ∈ l2 (K). The fact that the subspace
k )k∈K
consisting of sequences (zk )k∈K such that zk = 0 except perhaps for finitely many k is a dense suspace of
l2 (K) is left as an easy exercise.

Remark: The subspace consisting of all sequences (zk )k∈K such that zk = 0, except perhaps for finitely
many k, provides an example of a subspace which is not closed in l2 (K). Indeed, this space is strictly
contained in l2 (K), since there are countable sequences of nonnull elements in l2 (K) (why?).
We just need two more lemmas before being able to prove that every Hilbert space is isomorphic to some
l2 (K).
Lemma 26.3.3 Let E be a Hilbert space, and (uk )k∈K an orthogonal family in E. The following properties
hold:
!
(1) For every family (λk )k∈K ∈ l2 (K), the family (λk uk )k∈K is summable, and v = k∈K λk uk is the only
vector such that ck = λk for all k ∈ K, where the ck are the Fourier coefficients of v.
! !
(2) For any two families (λk )k∈K ∈ l2 (K) and (µk )k∈K ∈ l2 (K), if v = k∈K λk uk and w = k∈K µk uk ,
we have the following equation, also called Parseval identity:
"
!v, w" = λk µk .
k∈K
26.3. THE HILBERT SPACE L2 (K) AND THE RIESZ-FISCHER THEOREM 783

Proof . (1) The fact that (λk )k∈K ∈ l2 (K) means that (|λk |2 )k∈K is summable. The proof given in Lemma
26.2.4 (3) applies to the family (|λk |2 )k∈K (instead of (|ck |2 )k∈K ), and yields the fact that (λk uk )k∈K is
! 2
summable. Letting v = k∈K λk uk , recall that ck = !v, uk " / #uk # . Pick some k ∈ K. Since !−, −" is
continuous, for every " > 0, there is some η > 0 such that

| !v, uk " − !w, uk " | < " #uk #2

whenever
#v − w# < η.
However, since for every η > 0, there is some finite subset I of K such that
$ $
$ " $
$ $
$v − λj uj $
$ $<η
$ j∈J $
!
for every finite subset J of K such that I ⊆ J, we can pick J = I ∪ {k}, and letting w = j∈J λj uj , we get
+ , -++
+ "
+ + 2
+!v, uk " − λj uj , uk ++ < " #uk # .
+
+ j∈J +

However, , -
2
"
!v, uk " = ck #uk # and λj uj , uk = λk #uk #2 ,
j∈J

and thus, the above proves that |ck − λk | < " for every " > 0, and thus, that ck = λk .
(2) Since !−, −" is continuous, for every " > 0, there are some η1 > 0 and η2 > 0, such that

| !x, y" | < "

! !
whenever #x# < η1 and #y# < η2 . Since v = k∈K λk uk and w = k∈K µk uk , there is some finite subset I1
of K such that $ $
$ " $
$ $
$v − λj uj $
$ $ < η1
$ j∈J $
for every finite subset J of K such that I1 ⊆ J, and there is some finite subset I2 of K such that
$ $
$ $
$ " $
$w − µj u j $
$ $ < η2
$ j∈J $

for every finite subset J of K such that I2 ⊆ J. Letting I = I1 ∪ I2 , we get

+, -+
+ " " +
+ +
+ v− λi ui , w − µi ui + < ".
+ +
i∈I i∈I

Furthermore,
, -
" " " "
!v, w" = v− λi ui + λi ui , w − µi u i + µi u i
i∈I i∈I i∈I i∈I
, -
" " "
= v− λi ui , w − µi u i + λi µi ,
i∈I i∈I i∈I
784 CHAPTER 26. A GLIMPSE AT HILBERT SPACES

! !
since the ui are orthogonal to v − i∈I λi ui and w − i∈I µi ui for all i ∈ I. This proves that for every
" > 0, there is some finite subset I of K such that
+ +
+ " +
+ +
+!v, w" − λi µi + < ".
+ +
i∈I

We already know from Lemma 26.3.2 that (λk µk )k∈K is summable, and since " > 0 is arbitrary, we get
"
!v, w" = λk µk .
k∈K

The next Lemma states properties characterizing Hilbert bases (total orthogonal families).

Lemma 26.3.4 Let E be a Hilbert space, and let (uk )k∈K be an orthogonal family in E. The following
properties are equivalent:

(1) The family (uk )k∈K is a total orthogonal family.

∈ E, if (ck )k∈K are the Fourier coefficients of v, then the family (ck uk )k∈K is
(2) For every vector v !
summable and v = k∈K ck uk .

(3) For every vector v ∈ E, we have the Parseval identity:

2
"
#v# = |ck |2 .
k∈K

(4) For every vector u ∈ E, if !u, uk " = 0 for all k ∈ K, then u = 0.

Proof . The equivalence of (1), (2), and (3), is an immediate consequence of Lemma 26.2.4 and Lemma
26.3.3.
!
(4) If (uk )k∈K is a total orthogonal family and !u, uk " = 0 for all k ∈ K, since u = k∈K ck uk where
2
ck = !u, uk "/ #uk # , we have ck = 0 for all k ∈ K, and u = 0.
Conversely, assume that the closure V of (uk )k∈K is different from E. Then, by Lemma 26.1.7, we have
E = V ⊕ V ⊥ , where V ⊥ is the orthogonal complement of V , and V ⊥ is nontrivial since V 2= E. As a
consequence, there is some nonnull vector u ∈ V ⊥ . But then, u is orthogonal to every vector in V , and in
particular,
!u, uk " = 0
for all k ∈ K, which, by assumption, implies that u = 0, contradicting the fact that u 2= 0.

Remarks:

(1) If E is a Hilbert space and (uk )k∈K is a total orthogonal family in E, there is a simpler argument to
prove that u = 0 if !u, uk " = 0 for all k ∈ K, based on the continuity of !−, −". The argument is to
prove that the assumption implies that !v, u" = 0 for all v ∈ E. Since !−, −" is positive definite, this
implies that u = 0. By continuity of !−, −", for every " > 0, there is some η > 0 such that for every
finite subset I of K, for every family (λi )i∈I , for every v ∈ E,
+ , -+
+ " +
+ +
+!v, u" − λi ui , u + < "
+ +
i∈I
26.3. THE HILBERT SPACE L2 (K) AND THE RIESZ-FISCHER THEOREM 785

whenever $ $
$ " $
$ $
$v − λi ui $ < η.
$ $
i∈I

Since (uk )k∈K is dense in E, for every v ∈ E, there is some finite subset I of K and some family (λi )i∈I
such that $ $
$ " $
$ $
$v − λi ui $ < η,
$ $
i∈I
.! /
and since by assumption, i∈I λi ui , u = 0, we get

|!v, u"| < ".

Since this holds for every " > 0, we must have !v, u" = 0
(2) If V is any nonempty subset of E, the kind of argument used in the previous remark can be used to
prove that V ⊥ is closed (even if V is not), and that V ⊥⊥ is the closure of V .

We will now prove that every Hilbert space has some Hilbert basis. This requires using a fundamental
theorem from set theory known as Zorn’s Lemma, which we quickly review.
Given any set X with a partial ordering ≤, recall that a nonempty subset C of X is a chain if it is totally
ordered (i.e., for all x, y ∈ C, either x ≤ y or y ≤ x). A nonempty subset Y of X is bounded iff there is
some b ∈ X such that y ≤ b for all y ∈ Y . Some m ∈ X is maximal iff for every x ∈ X, m ≤ x implies that
x = m. We can now state Zorn’s Lemma. For more details, see Rudin [145], Lang [107], or Artin [5].
Lemma 26.3.5 Given any nonempty partially ordered set X, if every (nonempty) chain in X is bounded,
then X has some maximal element.

We can now prove the existence of Hilbert bases. We define a partial order on families (uk )k∈K as follows:
For any two families (uk )k∈K1 and (vk )k∈K2 , we say that
(uk )k∈K1 ≤ (vk )k∈K2
iff K1 ⊆ K2 and uk = vk for all k ∈ K1 . This is clearly a partial order.
Lemma 26.3.6 Let E be a Hilbert space. Given any orthogonal family (uk )k∈K in E, there is a total
orthogonal family (ul )l∈L containing (uk )k∈K .
Proof . Consider the set S of all orthogonal families greater than or equal to the family B = (uk )k∈K . We
claim that every chain in S is bounded. Indeed, if C = (Cl )l∈L is a chain in S, where Cl = (uk,l )k∈Kl , the
union family
(uk )k∈!l∈L Kl , where uk = uk,l whenever k ∈ Kl ,
is clearly an upper bound for C, and it is immediately verified that it is an orthogonal family. By Zorn’s
Lemma 26.3.5, there is a maximal family (ul )l∈L containing (uk )k∈K . If (ul )l∈L is not dense in E, then its
closure V is strictly contained in E, and by Lemma 26.1.7, the orthogonal complement V ⊥ of V is nontrivial
since V 2= E. As a consequence, there is some nonnull vector u ∈ V ⊥ . But then, u is orthogonal to every
vector in (ul )l∈L , and we can form an orthogonal family strictly greater than (ul )l∈L by adding u to this
family, contradicting the maximality of (ul )l∈L . Therefore, (ul )l∈L is dense in E, and thus, it is a Hilbert
basis.

Remark: It is possible to prove that all Hilbert bases for a Hilbert space E have index sets K of the same
cardinality. For a proof, see Bourbaki [21].
At last, we can prove that every Hilbert space is isomorphic to some Hilbert space l2 (K) for some suitable
K.
786 CHAPTER 26. A GLIMPSE AT HILBERT SPACES

Theorem 26.3.7 (Riesz-Fischer) For every Hilbert space E, there is some nonempty set K such that E
is isomorphic to the Hilbert space l2 (K). More specifically, for any Hilbert basis (uk )k∈K of E, the maps
f : l2 (K) → E and g: E → l2 (K) defined such that
" % &
f ((λk )k∈K ) = λk uk and g(u) = !u, uk "/ #uk #2 = (ck )k∈K ,
k∈K
k∈K

are bijective linear isometries such that g ◦ f = id and f ◦ g = id.

Proof . By Lemma 26.3.3 (1), the map f is well defined, and it it clearly linear. By Lemma 26.2.4 (3), the
map g is well defined, and it is also clearly linear. By Lemma 26.2.4 (2b), we have
"
f (g(u)) = u = ck u k ,
k∈K

and by Lemma 26.3.3 (1), we have

g(f ((λk )k∈K )) = (λk )k∈K ,
and thus g ◦ f = id and f ◦ g = id. By Lemma 26.3.3 (2), the linear map g is an isometry. Therefore, f is a
linear bijection and an isometry between l2 (K) and E, with inverse g.

Remark: The surjectivity of the map g: E → l2 (K) is known as the Riesz-Fischer theorem.
Having done all this hard work, we sketch how these results apply to Fourier series. Again, we refer the
readers to Rudin [145] or Lang [109, 110] for a comprehensive exposition.
Let C(T ) denote the set of all periodic continuous functions f : [−π, π] → C with period 2π. There is a
Hilbert space L2 (T ) containing C(T ) and such that C(T ) is dense in L2 (T ), whose inner product is given by
# π
!f, g" = f (x)g(x)dx.
−π

The Hilbert space L2 (T ) is the space of Lebesgue square-integrable periodic functions (of period 2π).
It turns out that the family (eikx )k∈Z is a total orthogonal family in L2 (T ), because it is already dense
in C(T ) (for instance, see Rudin [145]). Then, the Riesz-Fischer theorem says that for every family (ck )k∈Z
of complex numbers such that "
|ck |2 < ∞,
k∈Z
2
there is a unique function f ∈ L (T ) such that f is equal to its Fourier series
"
f (x) = ck eikx ,
k∈Z

where the Fourier coefficients ck of f are given by the formula

# π
1
ck = f (t)e−ikt dt.
2π −π

The Parseval theorem says that

+∞
" # π
1
ck dk = f (t)g(t)dt
2π −π
k=−∞

for all f, g ∈ L2 (T ), where ck and dk are the Fourier coefficients of f and g.

26.4. PROBLEMS 787

Thus, there is an isomorphism between the two Hilbert spaces L2 (T ) and l2 (Z),!which is the deep reason
why the Fourier coefficients “work”. Theorem 26.3.7 implies that the Fourier series k∈Z ck eikx of a function
f ∈ L2 (T ) converges to f in the L2 -sense, i.e., in the mean-square sense. This does not necessarily imply
that the Fourier series converges to f pointwise! This is a subtle issue, and for more on this subject, the
reader is referred to Lang [109, 110] or Schwartz [151, 152].
We can also consider the set C([−1, 1]) of continuous functions f : [−1, 1] → C. There is a Hilbert space
L2 ([−1, 1]) containing C([−1, 1]) and such that C([−1, 1]) is dense in L2 ([−1, 1]), whose inner product is given
by
# 1
!f, g" = f (x)g(x)dx.
−1
2
The Hilbert space L ([−1, 1]) is the space of Lebesgue square-integrable functions over [−1, 1]. The Legendre
polynomials Pn (x) defined in Example 5 of Section 6.2 (Chapter 6) form a Hilbert basis of L2 ([−1, 1]). Recall
that if we let fn be the function
fn (x) = (x2 − 1)n ,
Pn (x) is defined as follows:
1
P0 (x) = 1, and Pn (x) = fn(n) (x),
2n n!
(n)
where fn is the nth derivative of fn . The reason for the leading coefficient is to get Pn (1) = 1. It can be
shown with much efforts that
" (2(n − k))!
Pn (x) = (−1)k xn−2k .
2n (n − k)!k!(n − 2k)!
0≤k≤n/2

26.4 Problems

Problem 1.
Prove that the subgroup of U(n) generated by the hyperplane reflections is equal to the group

SU(n)± = {f ∈ U(n) | det(f ) = ±1}.

Problem 2.
Give the details of the proof of Theorem 25.1.7.
Problem 3.
Give the details of the proof of Theorem 25.1.8.
Problem 4.
Give the details of the proof of Lemma 25.1.9.
Problem 5.
Give the details of the proof of Lemma 25.2.2.
Problem 6.
Give the details of the proof of Theorem 25.2.7,
Problem 7.
Give the details of the proof of Theorem 25.2.8.
Problem 8.
788 CHAPTER 26. A GLIMPSE AT HILBERT SPACES

Given a Hermitian space E, Prove the “parallelogram inequality”,

2 2 2 2
#u + v# + #u − v# = 2(#u# + #v# ).

Problem 9.
Prove Lemma 26.1.4.
Problem 10.
Prove that the subspace of l2 (K) consisting of sequences (zk )k∈K such that zk = 0 except perhaps for
finitely many k is a dense suspace of l2 (K).
Problem 11.
Prove that the subspace consisting of all sequences (zk )k∈K such that zk = 0, except perhaps for finitely
many k, provides an example of a subspace which is not closed in l2 (K).
Problem 12.
Given a Hilbert space E, prove that for any nonempty subset V of E, the subspace V ⊥ is closed (even
if V is not), and V ⊥⊥ is the closure of V .