8Inner product spaces

IB Linear Algebra

8.1 Definitions and basic properties

Definition

(Inner product space)

.

Let

V

be a vector space. An inner product

on

V

is a positive-definite symmetric bilinear/hermitian form. We usually write

(x, y) instead of φ(x, y).

A vector space equipped with an inner product is an inner product space.

We will see that if we have an inner product, then we can define lengths and

distances in a sensible way.

Example.

(i) R

n

or C

n

with the usual inner product

(x, y) =

n

X

i=1

¯x

i

y

i

forms an inner product space.

In some sense, this is the only inner product on finite-dimensional spaces,

by Sylvester’s law of inertia. However, we would not like to think so, and

instead work with general inner products.

(ii)

Let

C

([0

,

1]

, F

) be the vector space of real/complex valued continuous

functions on [0, 1]. Then the following is an inner product:

(f, g) =

Z

1

0

¯

f(t)g(t) dt.

(iii)

More generally, for any

w

: [0

,

1]

→ R

+

continuous, we can define the inner

product on C([0, 1], F) as

(f, g) =

Z

1

0

w(t)

¯

f(t)g(t) dt.

If V is an inner product space, we can define a norm on V by

kvk =

p

(v, v).

This is just the usual notion of norm on

R

n

and

C

n

. This gives the notion of

length in inner product spaces. Note that

kvk >

0 with equality if and only if

v = 0.

Note also that the norm

k · k

determines the inner product by the polarization

identity.

We want to see that this indeed satisfies the definition of a norm, as you might

have seen from Analysis II. To prove this, we need to prove the Cauchy-Schwarz

inequality.

Theorem

(Cauchy-Schwarz inequality)

.

Let

V

be an inner product space and

v, w ∈ V . Then

|(v, w)| ≤ kvkkwk.

Proof.

If

w

= 0, then this is trivial. Otherwise, since the norm is positive

definite, for any λ, we get

0 ≤ (v −λw, v − λw) = (v, v ) −

¯

λ(w, v) − λ(v, w) + |λ|

2

(w, w).

We now pick a clever value of λ. We let

λ =

(w, v)

(w, w)

.

Then we get

0 ≤ (v, v) −

|(w, v)|

2

(w, w)

−

|(w, v)|

2

(w, w)

+

|(w, v)|

2

(w, w)

.

So we get

|(w, v)|

2

≤ (v, v)(w, w ).

So done.

With this, we can prove the triangle inequality.

Corollary

(Triangle inequality)

.

Let

V

be an inner product space and

v, w ∈ V

.

Then

kv + wk ≤ kvk + kw k.

Proof. We compute

kv + wk

2

= (v + w, v + w)

= (v, v) + (v, w) + (w, v) + (w, w)

≤ kvk

2

+ 2kvkkwk + kwk

2

= (kvk + kwk)

2

.

So done.

The next thing we do is to define orthogonality. This generalizes the notion

of being “perpendicular”.

Definition

(Orthogonal vectors)

.

Let

V

be an inner product space. Then

v, w ∈ V are orthogonal if (v, w) = 0.

Definition

(Orthonormal set)

.

Let

V

be an inner product space. A set

{v

i

:

i ∈ I} is an orthonormal set if for any i, j ∈ I, we have

(v

i

, v

j

) = δ

ij

It should be clear that an orthonormal set must be linearly independent.

Definition

(Orthonormal basis)

.

Let

V

be an inner product space. A subset of

V is an orthonormal basis if it is an orthonormal set and is a basis.

In an inner product space, we almost always want orthonormal basis only. If

we pick a basis, we should pick an orthonormal one.

However, we do not know there is always an orthonormal basis, even in the

finite-dimensional case. Also, given an orthonormal set, we would like to extend

it to an orthonormal basis. This is what we will do later.

Before that, we first note that given an orthonormal basis, it is easy to find

the coordinates of any vector in this basis. Suppose

V

is a finite-dimensional

inner product space with an orthonormal basis v

1

, ··· , v

n

. Given

v =

n

X

i=1

λ

i

v

i

,

we have

(v

j

, v) =

n

X

i=1

λ

i

(v

j

, v

i

) = λ

j

.

So v ∈ V can always be written as

n

X

i=1

(v

i

, v)v

i

.

Lemma

(Parseval’s identity)

.

Let

V

be a finite-dimensional inner product space

with an orthonormal basis v

1

, ··· , v

n

, and v, w ∈ V . Then

(v, w) =

n

X

i=1

(v

i

, v)(v

i

, w).

In particular,

kvk

2

=

n

X

i=1

|(v

i

, v)|

2

.

This is something we’ve seen in IB Methods, for infinite dimensional spaces.

However, we will only care about finite-dimensional ones now.

Proof.

(v, w) =

n

X

i=1

(v

i

, v)v

i

,

n

X

j=1

(v

j

, w)v

j

=

n

X

i,j=1

(v

i

, v)(v

j

, w)(v

i

, v

j

)

=

n

X

i,j=1

(v

i

, v)(v

j

, w)δ

ij

=

n

X

i=1

(v

i

, v)(v

i

, w).