4Hilbert spaces
II Linear Analysis
4.1 Inner product spaces
We have just looked at continuous functions on some compact space
K
. Another
important space is the space of square-integrable functions. Consider the space
L
2
(R) =
f : f is Lebesgue integrable :
Z
|f|
2
< ∞
/∼,
where f ∼ g if f = g is Lebesgue almost everywhere.
One thing we like to think about is the Fourier series. Recall that for
f ∈ C(S
1
), we have defined, for each k ∈ Z,
ˆ
f(k) =
1
2π
Z
π
−π
e
−ikx
f(x) dx,
and we have defined the partial sum
S
N
(f)(x) =
N
X
n=−N
e
inx
ˆ
f(k).
We have previously seen that even if
f
is continuous, it is possible that the partial
sums
S
N
do not converge, even pointwise. However, we can ask for something
weaker:
Proposition. Let f ∈ C(S
1
). Then
lim
N→∞
1
2π
Z
π
−π
|f(x) − S
N
(f)(x)|
2
dx = 0.
We will prove this later. However, the key points of the proof is the “orthog-
onality” of {e
inx
}. More precisely, we have
1
2π
Z
π
−π
e
inx
e
−imx
dx = 0 if m 6= n.
The introduction of Hilbert spaces is in particular a way to put this in a
general framework. We want to introduce an extra structure that gives rise to
“orthogonality”.
Definition
(Inner product)
.
Let
V
be a vector space over
R
or
C
. We say
p : V × V → R or C is an inner product on V it satisfies
(i) p(v, w) = p(w, v) for all v, w ∈ V . (antisymmetry)
(ii) p
(
λ
1
v
1
+
λ
2
v
2
, u
) =
λ
1
p
(
v
1
, w
) +
λ
2
p
(
v
2
, w
). (linearity in first argument)
(iii) p(v, v) ≥ 0 for all v ∈ V and equality holds iff v = 0. (non-negativity)
We will often denote an inner product by
p
(
v, w
) =
hv, wi
. We call (
V, h·, ·i
)
an inner product space.
Definition
(Orthogonality)
.
In an inner product space,
v
and
w
are orthogonal
if hv, wi = 0.
Orthogonality and the inner product is important when dealing with vector
spaces. For example, recall that when working with finite-dimensional spaces, we
had things like Hermitian matrices, orthogonal matrices and normal matrices. All
these are in some sense defined in terms of the inner product and orthogonality.
More fundamentally, when we have a finite-dimensional vector spaces, we often
write the vectors as a set of
n
coordinates. To define this coordinate system, we
start by picking
n
orthogonal vectors (which are in fact orthonormal), and then
the coordinates are just the projections onto these orthogonal vectors.
Hopefully, you are convinced that inner products are important. So let’s see
what we can get if we put in inner products to arbitrary vector spaces.
We will look at some easy properties of the inner product.
Proposition
(Cauchy-Schwarz inequality)
.
Let (
V, h·, ·i
) be an inner product
space. Then for all v, w ∈ V ,
|hv, wi| ≤
p
hv, vihw, wi,
with equality iff there is some λ ∈ R or C such that v = λw or w = λv.
Proof.
wlog, we can assume
w 6
= 0. Otherwise, this is trivial. Moreover, assume
hv, wi ∈ R. Otherwise, we can just multiply w by some e
iα
.
By non-negativity, we know that for all t, we have
0 ≤ hv + tw, v + twi
= hv, vi + 2thv, wi+ t
2
hw, wi.
Therefore, the discriminant of this quadratic polynomial in
t
is non-positive, i.e.
4(hv, wi)
2
− 4hv, vihw, wi ≤ 0,
from which the result follows.
Finally, note that if equality holds, then the discriminant is 0. So the
quadratic has exactly one root. So there exists
t
such that
v
+
tw
= 0, which of
course implies v = −tw.
Proposition. Let (V, h·, ·i) be an inner product space. Then
kvk =
p
hv, vi
defines a norm.
Proof.
The first two axioms of the norm are easy to check, since it follows directly
from definition of the inner product that
kvk ≥
0 with equality iff
v
=
0
, and
kλvk = |λ|kvk.
The only non-trivial thing to check is the triangle inequality. We have
kv + wk
2
= hv + w, v + wi
= kvk
2
+ kwk
2
+ |hv, wi| + |hw, vi|
≤ kvk
2
+ kwk
2
+ 2kvkkwk
= (kvk + kwk)
2
Hence we know that kv + wk ≤ kvk + kwk.
This motivates the following definition:
Definition
(Euclidean space)
.
A normed vector space (
V, k · k
) is a Euclidean
space if there exists an inner product h·, ·i such that
kvk =
p
hv, vi.
Proposition.
Let (
E, k · k
) be a Euclidean space. Then there is a unique inner
product h·, ·i such that kvk =
p
hv, vi.
Proof. The real and complex cases are slightly different.
First suppose
E
is a vector space over
R
, and suppose also that we have an
inner product h·, ·i such that kv k =
p
hv, vi. Then
hv + w, v + wi = kvk
2
+ 2hv, wi + kwk
2
.
So we get
hv, wi =
1
2
(kv + wk
2
− kvk
2
− kwk
2
). (∗)
In particular, the inner product is completely determined by the norm. So this
must be unique.
Now suppose E is a vector space over C. We have
hv + w, v + wi = kvk
2
+ kwk
2
+ hv, wi + hw, vi (1)
hv − w, v − wi = kvk
2
+ kwk
2
− hv, wi − hw, vi (2)
hv + iw, v + iwi = kvk
2
+ kwk
2
− ihv, wi + ihw, vi (3)
hv − iw, v − iwi = kvk
2
+ kwk
2
+ ihv, wi − ihw, vi (4)
Now consider (1) − (2) + i(3) − i(4). Then we obtain
kv + wk
2
− kv − wk
2
+ ikv + iwk
2
− ikv − iwk
2
= 4hv, wi. (†)
So again hv, wi is again determined by the norm.
The identities (
∗
) and (
†
) are sometimes known as the polarization identities.
Definition
(Hilbert space)
.
A Euclidean space (
E, k · k
) is a Hilbert space if it
is complete.
We will prove certain properties of the inner product.
Proposition
(Parallelogram law)
.
Let (
E, k · k
) be a Euclidean space. Then
for v, w ∈ E, we have
kv − wk
2
+ kv + wk
2
= 2kvk
2
+ 2kwk
2
.
This is called the parallelogram law because it says that for any parallelogram,
the sum of the square of the lengths of the diagonals is the sum of square of the
lengths of the two sides.
v
w
v + w
v − w
Proof. This is just simple algebraic manipulation. We have
kv − wk
2
+ kv + wk
2
= hv − w, v − wi+ hv + w, v + wi
= hv, vi − hv, wi−hw, vi + hw, wi
+ hv, vi + hv, wi+ hw, vi + hw, wi
= 2hv, vi + 2hw, wi.
Proposition
(Pythagoras theorem)
.
Let (
E, k · k
) be a Euclidean space, and
let v, w ∈ E be orthogonal. Then
kv + wk
2
= kvk
2
+ kwk
2
.
Proof.
kv + wk
2
= hv + w, v + wi
= hv, vi + hv, wi+ hw, vi + hw, wi
= hv, vi + 0 + 0 + hw, wi
= kvk
2
+ kwk
2
.
By induction, if
v
i
∈ E
for
i
= 1
, ··· , n
such that
hv
i
, v
j
i
= 0 for
i 6
=
j
, i.e.
they are mutually orthogonal, then
n
X
i=1
v
i
2
=
n
X
i=1
kv
i
k
2
.
Proposition.
Let (
E, k · k
) be a Euclidean space. Then
h·, ·i
:
E × E → C
is
continuous.
Proof. Let (v, w) ∈ E × E, and (
˜
v,
˜
w) ∈ E × E. We have
khv, wi − h
˜
v,
˜
wik = khv, wi − hv,
˜
wi + hv,
˜
wi − h
˜
v,
˜
wik
≤ khv, wi − hv,
˜
wik + khv,
˜
wi − h
˜
v,
˜
wik
= khv, w −
˜
wik + khv −
˜
v,
˜
wik
≤ kvkkw −
˜
wk + kv −
˜
vkk
˜
wk
Hence for
v, w
sufficiently closed to
˜
v,
˜
w
, we can get
khv, wi−h
˜
v,
˜
wik
arbitrarily
small. So it is continuous.
When we have an incomplete Euclidean space, we can of course take the
completion of it to form a complete extension of the original normed vector
space. However, it is not immediately obvious that the inner product can also
be extended to the completion to give a Hilbert space. The following proposition
tells us we can do so.
Proposition.
Let (
E, k · k
) denote a Euclidean space, and
¯
E
its completion.
Then the inner product extends to an inner product on
¯
E
, turning
¯
E
into a
Hilbert space.
Proof.
Recall we constructed the completion of a space as the equivalence classes
of Cauchy sequences (where two Cauchy sequences (
x
n
) and (
x
0
n
) are equivalent
if
|x
n
−x
0
n
| →
0). Let (
x
n
)
,
(
y
n
) be two Cauchy sequences in
E
, and let
˜x, ˜y ∈
¯
E
denote their equivalence classes. We define the inner product as
h
˜
x,
˜
yi = lim
n→∞
hx
n
, y
n
i. (∗)
We want to show this is well-defined. Firstly, we need to make sure the limit
exists. We can show this by showing that this is a Cauchy sequence. We have
khx
n
, y
n
i − hx
m
, y
m
ik = khx
n
, y
n
i − hx
m
, y
n
i + hx
m
, y
n
i − hx
m
, y
m
ik
≤ khx
n
, y
n
i − hx
m
, y
n
ik + khx
m
, y
n
i − hx
m
, y
m
ik
≤ khx
n
, x
m
, y
n
ik + khx
m
, y
n
− y
m
ik
≤ kx
n
− x
m
kky
n
k + kxkky
n
− y
n
k
So hx
n
, y
n
i is a Cauchy sequence since (x
n
) and (y
n
) are.
We also need to show that (
∗
) does not depend on the representatives for
˜
x
and
˜
y. This is left as an exercise for the reader
We also need to show that
h·, ·i
¯
E
define the norm of
k · k
¯
E
, which is yet
another exercise.
Example. Consider the space
`
2
=
(
(x
1
, x
2
, ···) : x
i
∈ C,
∞
X
i=1
|x
i
|
2
< ∞
)
.
We already know that this is a complete Banach space. We can also define an
inner product on this space by
ha, bi
`
2
=
∞
X
i=1
a
i
¯
b
i
.
We need to check that this actually converges. We prove this by showing absolute
convergence. For each n, we can use Cauchy-Schwarz to obtain
n
X
i=1
|a
i
¯
b
i
| ≤
n
X
i=1
|a|
2
!
1
2
n
X
i=1
|b|
2
!
1
2
≤ kak
`
2
kbk
`
2
.
So it converges. Now notice that the
`
2
norm is indeed induced by this inner
product.
This is a significant example since we will later show that every separable (i.e.
has countable basis) infinite dimensional Hilbert space is isometric isomorphic
to `
2
.
Definition
(Orthogonal space)
.
Let
E
be a Euclidean space and
S ⊆ E
an
arbitrary subset. Then the orthogonal space of S, denoted by S
⊥
is given by
S
⊥
= {v ∈ E : ∀w ∈ S, hv, wi = 0}.
Proposition.
Let
E
be a Euclidean space and
S ⊆ E
. Then
S
⊥
is a closed
subspace of E, and moreover
S
⊥
= (span S)
⊥
.
Proof.
We first show it is a subspace. Let
u, v ∈ S
⊥
and
λ, µ ∈ C
. We want to
show λu + µv ∈ S
⊥
. Let w ∈ S. Then
hλu + µv, wi = λhu, wi + µhv, wi = 0.
To show it is closed, let
u
n
∈ S
⊥
be a sequence such that
u
n
→ u ∈ E
. Let
w ∈ S. Then we know that
hu
n
, wi = 0.
Hence, by the continuity of the inner product, we have
0 = lim
n→∞
hu
n
, wi = hlim u
n
, wi = hu, wi.
The remaining part is left as an exercise.
Note that if
V
is a linear subspace, then
V ∩V
⊥
=
{
0
}
, since any
v ∈ V ∩V
⊥
has to satisfy hv, vi = 0. So V + V
⊥
is a direct sum.
Theorem.
Let (
E, k · k
) be a Euclidean space, and
F ⊆ E
a complete subspace.
Then F ⊕ F
⊥
= E.
Hence, by definition of the direct sum, for
x ∈ E
, we can write
x
=
x
1
+
x
2
,
where x
1
∈ F and x
2
∈ F
⊥
. Moreover, x
1
is uniquely characterized by
kx
1
− xk = inf
y∈F
ky − xk.
F
x
x
1
Note that this is not necessarily true if F is not complete.
Proof.
We already know that
F ⊕ F
⊥
is a direct sum. It thus suffices to show
that the sum is the whole of E.
Let y
i
∈ F be a sequence with
lim
i→∞
ky
i
− xk = inf
y∈F
ky − xk = d.
We want to show that
y
is a Cauchy sequence. Let
ε >
0 be given. Let
n
0
∈ N
such that for all i ≥ n
0
, we have
ky
i
− xk
2
≤ d
2
+ ε.
We now use the parallelogram law for
v
=
x − y
i
,
w
=
x − y
j
with
i, j ≥ n
0
.
Then the parallelogram law says:
kv + wk
2
+ kv − wk
2
= 2kvk
2
+ 2kwk
2
,
or
ky
j
− y
i
k
2
+ k2x − y
i
− y
j
k
2
= 2ky
i
− xk
2
+ 2ky
j
− xk
2
.
Hence we know that
ky
i
− y
j
k
2
≤ 2ky
i
− xk
2
+ 2ky
j
− xk
2
− 4
x −
y
i
+ y
j
2
2
≤ 2(d
2
+ ε) + 2(d
2
+ ε) − 4d
2
≤ 4ε.
So
y
i
is a Cauchy sequence. Since
F
is complete,
y
i
→ y ∈ F
for some
F
.
Moreover, by continuity, of k · k, we know that
d = lim
i→∞
ky
i
− xk = ky − xk.
Now let
x
1
=
y
and
x
2
=
x − y
. The only thing left over is to show
x
2
∈ F
⊥
.
Suppose not. Then there is some
˜
y ∈ F such that
h
˜
y, x
2
i 6= 0.
The idea is that we can perturbe
y
by a little bit to get a point even closer to
x
.
By multiplying
˜
y with a scalar, we can assume
h
˜
y, x
2
i > 0.
Then for t > 0, we have
k(y + t
˜
y) − xk
2
= hy + t
˜
y − x, y + t
˜
y − xi
= hy − x, y − xi + ht
˜
y, y − xi + hy − x, t
˜
yi + t
2
h
˜
y,
˜
yi
= d
2
− 2th
˜
y, x
2
i + t
2
k
˜
yk
2
.
Hence for sufficiently small
t
, the
t
2
term is negligible, and we can make this
less that d
2
. This is a contradiction since y + t
˜
y ∈ F .
As a corollary, we can define the projection map as follows:
Corollary.
Let
E
be a Euclidean space and
F ⊆ E
a complete subspace. Then
there exists a projection map
P
:
E → E
defined by
P
(
x
) =
x
1
, where
x
1
∈ F
is
as defined in the theorem above. Moreover,
P
satisfies the following properties:
(i) P (E) = F and P (F
⊥
) = {0}, and P
2
= P . In other words, F
⊥
≤ ker P .
(ii) (I − P )(E) = F
⊥
, (I − P )(F ) = {0}, (I − P )
2
= (I − P ).
(iii) kP k
B(E,E)
≤
1 and
kI −P k
B(E,E)
≤
1, with equality if and only if
F 6
=
{
0
}
and F
⊥
6= {0} respectively.
Here P projects our space to F , where I − P projects our space to F
⊥
.