8Inner product spaces
IB Linear Algebra
8.1 Definitions and basic properties
Definition
(Inner product space)
.
Let
V
be a vector space. An inner product
on
V
is a positive-definite symmetric bilinear/hermitian form. We usually write
(x, y) instead of φ(x, y).
A vector space equipped with an inner product is an inner product space.
We will see that if we have an inner product, then we can define lengths and
distances in a sensible way.
Example.
(i) R
n
or C
n
with the usual inner product
(x, y) =
n
X
i=1
¯x
i
y
i
forms an inner product space.
In some sense, this is the only inner product on finite-dimensional spaces,
by Sylvester’s law of inertia. However, we would not like to think so, and
instead work with general inner products.
(ii)
Let
C
([0
,
1]
, F
) be the vector space of real/complex valued continuous
functions on [0, 1]. Then the following is an inner product:
(f, g) =
Z
1
0
¯
f(t)g(t) dt.
(iii)
More generally, for any
w
: [0
,
1]
→ R
+
continuous, we can define the inner
product on C([0, 1], F) as
(f, g) =
Z
1
0
w(t)
¯
f(t)g(t) dt.
If V is an inner product space, we can define a norm on V by
kvk =
p
(v, v).
This is just the usual notion of norm on
R
n
and
C
n
. This gives the notion of
length in inner product spaces. Note that
kvk >
0 with equality if and only if
v = 0.
Note also that the norm
k · k
determines the inner product by the polarization
identity.
We want to see that this indeed satisfies the definition of a norm, as you might
have seen from Analysis II. To prove this, we need to prove the Cauchy-Schwarz
inequality.
Theorem
(Cauchy-Schwarz inequality)
.
Let
V
be an inner product space and
v, w ∈ V . Then
|(v, w)| ≤ kvkkwk.
Proof.
If
w
= 0, then this is trivial. Otherwise, since the norm is positive
definite, for any λ, we get
0 ≤ (v −λw, v − λw) = (v, v ) −
¯
λ(w, v) − λ(v, w) + |λ|
2
(w, w).
We now pick a clever value of λ. We let
λ =
(w, v)
(w, w)
.
Then we get
0 ≤ (v, v) −
|(w, v)|
2
(w, w)
−
|(w, v)|
2
(w, w)
+
|(w, v)|
2
(w, w)
.
So we get
|(w, v)|
2
≤ (v, v)(w, w ).
So done.
With this, we can prove the triangle inequality.
Corollary
(Triangle inequality)
.
Let
V
be an inner product space and
v, w ∈ V
.
Then
kv + wk ≤ kvk + kw k.
Proof. We compute
kv + wk
2
= (v + w, v + w)
= (v, v) + (v, w) + (w, v) + (w, w)
≤ kvk
2
+ 2kvkkwk + kwk
2
= (kvk + kwk)
2
.
So done.
The next thing we do is to define orthogonality. This generalizes the notion
of being “perpendicular”.
Definition
(Orthogonal vectors)
.
Let
V
be an inner product space. Then
v, w ∈ V are orthogonal if (v, w) = 0.
Definition
(Orthonormal set)
.
Let
V
be an inner product space. A set
{v
i
:
i ∈ I} is an orthonormal set if for any i, j ∈ I, we have
(v
i
, v
j
) = δ
ij
It should be clear that an orthonormal set must be linearly independent.
Definition
(Orthonormal basis)
.
Let
V
be an inner product space. A subset of
V is an orthonormal basis if it is an orthonormal set and is a basis.
In an inner product space, we almost always want orthonormal basis only. If
we pick a basis, we should pick an orthonormal one.
However, we do not know there is always an orthonormal basis, even in the
finite-dimensional case. Also, given an orthonormal set, we would like to extend
it to an orthonormal basis. This is what we will do later.
Before that, we first note that given an orthonormal basis, it is easy to find
the coordinates of any vector in this basis. Suppose
V
is a finite-dimensional
inner product space with an orthonormal basis v
1
, ··· , v
n
. Given
v =
n
X
i=1
λ
i
v
i
,
we have
(v
j
, v) =
n
X
i=1
λ
i
(v
j
, v
i
) = λ
j
.
So v ∈ V can always be written as
n
X
i=1
(v
i
, v)v
i
.
Lemma
(Parseval’s identity)
.
Let
V
be a finite-dimensional inner product space
with an orthonormal basis v
1
, ··· , v
n
, and v, w ∈ V . Then
(v, w) =
n
X
i=1
(v
i
, v)(v
i
, w).
In particular,
kvk
2
=
n
X
i=1
|(v
i
, v)|
2
.
This is something we’ve seen in IB Methods, for infinite dimensional spaces.
However, we will only care about finite-dimensional ones now.
Proof.
(v, w) =
n
X
i=1
(v
i
, v)v
i
,
n
X
j=1
(v
j
, w)v
j
=
n
X
i,j=1
(v
i
, v)(v
j
, w)(v
i
, v
j
)
=
n
X
i,j=1
(v
i
, v)(v
j
, w)δ
ij
=
n
X
i=1
(v
i
, v)(v
i
, w).