II Linear Analysis (Full)

Part II — Linear Analysis

Based on lectures by J. W. Luk

Notes taken by Dexter Chua

Michaelmas 2015

These notes are not endorsed by the lecturers, and I have modified them (often

significantly) after lectures. They are nowhere near accurate representations of what

was actually lectured, and in particular, all errors are almost surely mine.

Part IB Linear Algebra, Analysis II and Metric and Topological Spaces are essential

Normed and Banach spaces. Linear mappings, continuity, boundedness, and norms.

Finite-dimensional normed spaces. [4]

The Baire category theorem. The principle of uniform boundedness, the closed graph

theorem and the inversion theorem; other applications. [5]

The normality of compact Hausdorff spaces. Urysohn’s lemma and Tiezte’s exten-

sion theorem. Spaces of continuous functions. The Stone-Weierstrass theorem and

applications. Equicontinuity: the Ascoli-Arzel`a theorem. [5]

Inner product spaces and Hilbert spaces; examples and elementary properties. Or-

thonormal systems, and the orthogonalization process. Bessel’s inequality, the Parseval

equation, and the Riesz-Fischer theorem. Duality; the self duality of Hilbert space. [5]

Bounded linear operations, invariant subspaces, eigenvectors; the spectrum and resolvent

set. Compact operators on Hilbert space; discreteness of spectrum. Spectral theorem

for compact Hermitian operators. [5]

Contents

0 Introduction

1 Normed vector spaces

1.1 Bounded linear maps

1.2 Dual spaces

1.3 Adjoint

1.4 The double dual

1.5 Isomorphism

1.6 Finite-dimensional normed vector spaces

1.7 Hahn–Banach Theorem

2 Baire category theorem

2.1 The Baire category theorem

2.2 Some applications

3 The topology of C(K)

3.1 Normality of compact Hausdorff spaces

3.2 Tietze-Urysohn extension theorem

3.3 Arzel`a-Ascoli theorem

3.4 Stone–Weierstrass theorem

4 Hilbert spaces

4.1 Inner product spaces

4.2 Riesz representation theorem

4.3 Orthonormal systems and basis

4.4 The isomorphism with `

4.5 Operators

4.6 Self-adjoint operators

0 Introduction

In IB Linear Algebra, we studied vector spaces in general. Most of the time, we

concentrated on finite-dimensional vector spaces, since these are easy to reason

about. For example, we know that every finite-dimensional vector space (by

definition) has a basis. Using the basis, we can represent vectors and linear maps

concretely as column vectors (in F

) and matrices.

However, in real life, often we have to work with infinite-dimensional vector

spaces instead. For example, we might want to consider the vector space of

all continuous (real) functions, or the vector space of infinite sequences. It is

difficult to analyse these spaces using the tools from IB Linear Algebra, since

many of those assume the vector space is finite-dimensional. Moreover, in these

cases, we often are not interested in the vector space structure itself. It’s just

that the objects we are interested in happen to have a vector space structure.

Instead, we want to look at notions like continuity and convergence. We want to

do analysis on vector spaces. These are not something the vector space structure

itself provides.

In this course, we are going to give our vector spaces some additional structure.

For the first half of the course, we will grant our vector space a norm. This

allows us to assign a “length” to each vector. With this, we can easily define

convergence and continuity. It turns out this allows us to understand a lot about,

say, function spaces and sequence spaces.

In the second half, we will grant a stronger notion, called the inner product.

Among many things, this allows us to define orthogonality of the elements of a

vector space, which is something we are familiar with from, say, IB Methods.

Most of the time, we will be focusing on infinite-dimensional vector spaces,

since finite-dimensional spaces are boring. In fact, we have a section dedicated

to proving that finite-dimensional vector spaces are boring. In particular, they

are all isomorphic to

, and most of our theorems can be proved trivially for

finite-dimensional spaces using what we already know from IB Linear Algebra.

So we will not care much about them.

1 Normed vector spaces

In IB Linear Algebra, we have studied vector spaces in quite a lot of detail.

However, just knowing something is a vector space usually isn’t too helpful.

Often, we would want the vector space to have some additional structure. The

first structure we will study is a norm.

Definition

(Normed vector space)

A normed vector space is a pair (

V, k · k

where

is a vector space over a field

and

k · k

is a function

k · k

V 7→ R

known as the norm, satisfying

(i) kvk ≥ 0 for all v ∈ V , with equality iff v = 0.

(ii) kλvk = |λ|kvk for all λ ∈ F, v ∈ V .

(iii) kv + wk ≤ kvk+ kwk for all v, w ∈ V .

Intuitively, we think of kvk as the “length” or “magnitude” of the vector.

Example.

Let

be a finite dimensional vector space, and

, ··· , e

}

a basis.

Then, for any v =

i=1

, we can define a norm as

kvk =

i=1

If we are given a norm of a vector space

, we immediately obtain two more

structures on V for free, namely a metric and a topology.

Recall from IB Metric and Topological Spaces that (

V, d

) is a metric space if

the metric d : V × V → R satisfies

(i) d(x, x) = 0 for all x ∈ V .

(ii) d(x, y) = d(y, x) for all x, y ∈ V .

(iii) d(x, y) ≤ d(x, z) + d(z, y) for all x, y, z ∈ V .

Also, a topological spaces is a set

together with a topology (a collection of

open subsets) such that

(i) ∅ and V are open subsets.

(ii) The union of open subsets is open.

(iii) The finite intersection of open subsets is open.

As we have seen in IB Metric and Topological Spaces, a norm on a vector space

induces a metric by

(

v, w

) =

kv − wk

. This metric in terms defines a topology

where the open sets are given by “

U ⊆ V

is open iff for any

x ∈ U

, there

exists some ε > 0 such that B(x, ε) = {y ∈ V : d(x, y) < ε} ⊆ U”.

This induced topology is not just a random topology on the vector space.

They have the nice property that the vector space operators behave well under

this topology.

Proposition.

Addition + :

V ×V → V

, and scalar multiplication

F×V → V

are continuous with respect to the topology induced by the norm (and the usual

product topology).

Proof.

Let

be open in

. We want to show that (+)

−1

(

) is open. Let

(

, v

)

∈

(+)

−1

(

), i.e.

∈ U

. Since

∈ U

, there exists

such that

(

, ε

)

⊆ U

. By the triangle inequality, we know that

(

)

⊆

U. Hence we have (v

, v

) ∈ B



, v



⊆ (+)

−1

(U). So (+)

−1

(U) is open.

Scalar multiplication can be done in a very similar way.

This motivates the following definition — we can do without the norm, and

just require a topology in which addition and scalar multiplication are continuous.

Definition

(Topological vector space)

A topological vector space (

V, U

) is

a vector space

together with a topology

such that addition and scalar

multiplication are continuous maps, and moreover singleton points

{v}

are

closed sets.

The requirement that points are closed is just a rather technical requirement

needed in certain proofs. We should, however, not pay too much attention to

this when trying to understand it intuitively.

A natural question to ask is: when is a topological vector space normable? i.e.

Given a topological vector space, can we find a norm that induces the topology?

To answer this question, we will first need a few definitions.

Definition

(Absolute convexity)

Let

be a vector space. Then

C ⊆ V

absolutely convex (or balanced convex ) if for any

λ, µ ∈ F

such that

|λ|

|µ| ≤

we have λC + µC ⊆ C. In other words, if c

, c

∈ C, we have λc

+ µc

∈ C.

Proposition.

If (

V, k · k

) is a normed vector space, then

(

) =

(

0, t

) =

kvk < t} is absolutely convex.

Proof. By triangle inequality.

Definition

(Bounded subset)

Let

be a topological vector space. Then

B ⊆ V

is bounded if for every open neighbourhood

U ⊆ V

, there is some

s >

0 such

that B ⊆ tU for all t > s.

At first sight, this might seem like a rather weird definition. Intuitively, this

just means that

is bounded if, whenever we take any open set

, by enlarging

it by a scalar multiple, we can make it fully contain B.

Example. B(t) in a normed vector space is bounded.

Proposition.

A topological vector space (

V, U

) is normable if and only if there

exists an absolutely convex, bounded open neighbourhood of 0.

Proof.

One direction is obvious — if

is normable, then

(

) is an absolutely

convex, bounded open neighbourhood of 0.

The other direction is not too difficult as well. We define the Minkowski

functional µ : V → R by

(v) = inf{t > 0 : v ∈ tC},

where C is our absolutely convex, bounded open neighbourhood.

Note that by definition, for any

t < µ

(

v 6∈ tC

. On the other hand, by

absolute convexity, for any t > µ

(v), we have v ∈ tC.

We now show that this is a norm on V :

(i) If v = 0, then v ∈ 0C. So µ

(0) = 0. On the other hand, suppose v 6= 0.

Since a singleton point is closed,

V \{v}

is an open neighbourhood of

0. Hence there is some

such that

C ⊆ tU

. Alternatively,

C ⊆ U

. Hence,

v 6∈

C. So µ

(v) ≥

> 0. So µ

(v) = 0 iff v = 0.

(ii) We have

(λv) = inf{t > 0 : λv ∈ tC} = λ inf{t > 0 : v ∈ tC} = λµ

(v).

(iii) We want to show that

(v + w) ≤ µ

(v) + µ

(w).

This is equivalent to showing that

inf{t > 0 : v + w ∈ tC} ≤ inf{t > 0 : v ∈ tC} + inf{r > 0 : w ∈ rC}.

This is, in turn equivalent to proving that if

v ∈ tC

and

w ∈ rC

, then

(v + w) ∈ (t + r)C.

Let

v/t, w

w/r

. Then we want to show that if

∈ C

and

∈ C

then

(t+r)

(

)

∈ C

. This is exactly what is required by convexity.

So done.

In fact, the condition of absolute convexity can be replaced “convex”, where

“convex” means for every

t ∈

1],

+ (1

− t

)

C ⊆ C

. This is since for every

convex bounded

, we can find always find a absolutely convex bounded

C ⊆ C

which is something not hard to prove.

Among all normed spaces, some are particularly nice, known as Banach

spaces.

Definition

(Banach spaces)

A normed vector space is a Banach space if it is

complete as a metric space, i.e. every Cauchy sequence converges.

Example.

(i)

A finite dimensional vector space (which is isomorphic to

for some

)

is Banach.

(ii) Let X be a compact Hausdorff space. Then let

B(X) = {f : X → R such that f is bounded}.

This is obviously a vector space, and we can define the norm be

kfk

sup

x∈X

(

). It is easy to show that this is a norm. It is less trivial to

show that this is a Banach space.

Let

} ⊆ B

(

) be a Cauchy sequence. Then for any

(

)

} ⊆ R

also Cauchy. So we can define f(x) = lim

n→∞

(x).

To show that

→ f

, let

ε >

0. By definition of

being Cauchy,

there is some

such that for any

n, m > N

and any fixed

, we have

(

)

− f

(

)

| < ε

. Take the limit as

m → ∞

. Then

(

)

→ f

(

). So

(

)

− f

(

)

| ≤ ε

. Since this is true for all

, for any

n > N

, we must

have kf

− fk ≤ ε. So f

→ f.

(iii) Define X as before, and let

C(X) = {f : X → R such that f is continuous}.

Since any continuous

is bounded, so

(

)

⊆ B

(

). We define the norm

as before.

Since we know that

(

)

⊆ B

(

), to show that

(

) is Banach, it suffices

to show that

(

)

⊆ B

(

) is closed, i.e. if

→ f

∈ C

(

), then

f ∈ C

(

), i.e. the uniform limit of a continuous function is continuous.

Proof can be found in IB Analysis II.

(iv) For 1 ≤ p < ∞, define

([0, 1]) = {f : [0, 1] → R such that f is continuous}.

We define the norm k · k

kfk



|f|



1/p

It is easy to show that

is indeed a vector space, and we now check that

this is a norm.

(a) kfk

≥

0 is obvious. Also, suppose that

kfk

= 0. Then we must

have

= 0. Otherwise, if

f 6

= 0, say

(

) =

for some

. Then there

is some

such that for any

y ∈

(

x − δ, x

), we have

(

)

k ≥

Hence

kfk



|f|



1/p

≥

2δ





1/p

> 0.

(b) kλfk = |λ|kfk is obvious

(c)

The triangle inequality is the exactly what the Minkowski inequality

says, which is in the example sheet.

It turns out that

is not a Banach space. We can brute-force a hard

proof here, but we will later develop some tools that allow us to prove this

much more easily.

Hence, we define

([0

1]) to be the completion of

([0

1]). In IID

Probability and Measure, we will show that L

([0, 1]) is in fact the space

([0, 1]) =



f : [0, 1] → R such that

|f|

dx < ∞



/∼,

where the integral is the Lebesgue integral, and we are quotienting by the

relation

f ∼ g

Lebesgue almost everywhere. You will understand

what these terms mean in the IID Probability and Measure course.

(v) `

spaces: for p ∈ [1, ∞), define

(F) =

(

, x

, ···) : x

∈ F,

∞

i=1

< ∞

)

with the norm

kxk

∞

i=1

1/p

It should be easy to check that this is a normed vector space. Moreover,

this is a Banach space. Proof is in example sheet.

(vi) `

∞

space: we define

∞



, x

, ···) : x

∈ F, sup

i∈N

| < ∞



with norm

kxk

∞

= sup

i∈N

Again, this is a Banach space.

(vii)

Let

(1) be the unit open ball in

. Define

(

) to be the set of

continuous functions

B → R

. Note that unlike in our previous example,

these functions need not be bounded. So our previous norm cannot be

applied. However, we can still define a topology as follows:

Let

}

∞

i=1

be a sequence of compact subsets of

such that

⊆ K

i+1

and

∞

i=1

= B. We define the basis to include



f ∈ C(B) : sup

x∈K

|f(x)| <



for each m, i = 1, 2, ···, as well as the translations of these sets.

This weird basis is chosen such that

→ f

in this topology iff

→ f

uniformly in every compact set. It can be showed that this is not normable.

1.1 Bounded linear maps

With vector spaces, we studied linear maps. These are maps that respect the

linear structure of a vector space. With normed vector spaces, the right kind of

maps to study is the bounded linear maps.

Definition

(Bounded linear map)

. T

X → Y

is a bounded linear map if there

is a constant

C >

0 such that

kT xk

≤ Ckxk

for all

x ∈ X

. We write

(

X, Y

)

for the set of bounded linear maps from X to Y .

This is equivalent to saying

(

(1))

⊆ B

(

) for some

C >

0. This also

equivalent to saying that

(

) is bounded for every bounded subset

Note that this final characterization is also valid when we just have a topological

vector space.

How does boundedness relate to the topological structure of the vector spaces?

It turns out that boundedness is the same as continuity, which is another reason

why we like bounded linear maps.

Proposition.

Let

be normed vector spaces,

X → Y

a linear map.

Then the following are equivalent:

(i) T is continuous.

(ii) T is continuous at 0.

(iii) T is bounded.

Proof. (i) ⇒ (ii) is obvious.

(ii)

⇒

(iii): Consider

(1)

⊆ Y

, the unit open ball. Since

is continuous

at 0,

−1

(

(1))

⊆ X

is open. Hence there exists

ε >

0 such that

(

)

⊆

−1

(1)). So T (B

(ε)) ⊆ B

(1). So T (B

(1)) ⊆ B





. So T is bounded.

(iii)

⇒

(i): Let

ε >

0. Then

kT x

−T x

(

−x

)

≤ Ckx

−x

This is less than ε if kx

− x

k < C

−1

ε. So done.

Using the obvious operations,

(

X, Y

) can be made a vector space. What

about a norm?

Definition

(Norm on

(

X, Y

))

Let

X → Y

be a bounded linear map.

Define kT k

B(X,Y )

kT k

B(X,Y )

= sup

kxk≤1

kT xk

Alternatively, this is the minimum

such that

kT xk

≤ Ckxk

for all

In particular, we have

kT xk

≤ kT k

B(X,Y )

kxk

1.2 Dual spaces

We will frequently be interested in one particular case of B(X, Y ).

Definition (Dual space). Let V be a normed vector space. The dual space is

∗

= B(V, F).

We call the elements of V

∗

functionals. The algebraic dual of V is

= L(V, F),

where we do not require boundedness.

One particularly nice property of the dual is that

∗

is always a Banach

space.

Proposition. Let V be a normed vector space. Then V

∗

is a Banach space.

Proof.

Suppose

} ∈ V

∗

is a Cauchy sequence. We define

as follows: for

any

v ∈ V

(

)

} ⊆ F

is Cauchy sequence. Since

is complete (it is either

or C), we can define T : V → R by

T (v) = lim

n→∞

(v).

Our objective is to show that

→ T

. The first step is to show that we indeed

have T ∈ V

∗

, i.e. T is a bounded map.

Let

kvk ≤

1. Pick

= 1. Then there is some

such that for all

i > N

, we

have

(v) −T (v)| < 1.

Then we have

|T (v)| ≤ |T

(v) −T (v)| + |T

(v)|

< 1 + kT

∗

kvk

≤ 1 + kT

∗

≤ 1 + sup

∗

Since

is Cauchy,

sup

∗

is bounded. Since this bound does not depend

on v (and N), we get that T is bounded.

Now we want to show that kT

− T k

∗

→ 0 as n → ∞.

For arbitrary ε > 0, there is some N such that for all i, j > N , we have

− T

∗

< ε.

In particular, for any v such that kvk ≤ 1, we have

(v) −T

(v)| < ε.

Taking the limit as j → ∞, we obtain

(v) −T (v)| ≤ ε.

Since this is true for any v, we have

− T k

∗

≤ ε.

for all i > N. So T

→ T .

Exercise: in general, for

X, Y

normed vector spaces, what condition on

and Y guarantees that B(X, Y ) is a Banach space?

1.3 Adjoint

The idea of the adjoint is given a

T ∈ B

(

X, Y

), produce a “dual map”, or an

adjoint T

∗

∈ B(Y

∗

, X

∗

There is really only one (non-trivial) natural way of doing this. First we can

think about what

∗

should do. It takes in something from

∗

and produces

something in

∗

. By the definition of the dual space, this is equivalent to taking

in a function g : Y → F and returning a function T

∗

(g) : X → F.

To produce this

∗

(

), the only things we have on our hands to use are

X → Y

and

Y → F

. Thus the only option we have is to define

∗

(

)

as the composition g ◦ T , or T

∗

(g)(x) = g(T(x)) (we also have a silly option of

producing the zero map regardless of input, but this is silly). Indeed, this is the

definition of the adjoint.

Definition

(Adjoint)

Let

X, Y

be normal vector spaces. Given

T ∈ B

(

X, Y

we define the adjoint of T , denoted T

∗

, as a map T

∗

∈ B(Y

∗

, X

∗

) given by

∗

(g)(x) = g(T(x))

for x ∈ X, y ∈ Y

∗

. Alternatively, we can write

∗

(g) = g ◦ T.

It is easy to show that our T

∗

is indeed linear. We now show it is bounded.

Proposition. T

∗

is bounded.

Proof.

We want to show that

∗

B(Y

∗

)

is finite. For simplicity of notation,

the supremum is assumed to be taken over non-zero elements of the space. We

have

∗

B(Y

∗

)

= sup

g∈Y

∗

(g)k

∗

kgk

∗

= sup

g∈Y

∗

sup

x∈X

∗

(g)(x)|/kxk

kgk

∗

= sup

g∈Y

∗

sup

x∈X

|g(T x)|

kgk

∗

kxk

≤ sup

g∈Y

∗

sup

x∈X

kgk

∗

kT xk

kgk

∗

kxk

≤ sup

x∈X

kT k

B(X,Y )

kxk

= kT k

B(X,Y )

So it is finite.

1.4 The double dual

Definition

(Double dual)

Let

be a normed vector space. Define

∗∗

= (

∗

)

∗

We want to define a map

V → V

∗∗

. Again, we can reason about what

we expect this function to do. It takes in a

v ∈ V

, and produces a

(

)

∈ V

∗∗

Expanding the definition, this gives a

(

) :

∗

→ F

. Hence this

(

) takes in

a g ∈ V

∗

, and returns a φ(v)(g) ∈ F.

This is easy. Since

g ∈ V

∗

, we know that

is a function

V → F

. Given

this function

and a

v ∈ V

, it is easy to produce a

(

)(

)

∈ F

. Just apply

on v:

φ(v)(g) = g(v).

Proposition.

Let

V → V

∗∗

be defined by

(

)(

) =

(

). Then

is a

bounded linear map and kφk

B(V,V

∗

)

≤ 1

Proof. Again, we are taking supremum over non-zero elements. We have

kφk

B(V,V

∗

)

= sup

v∈V

kφ(v)k

∗∗

kvk

= sup

v∈V

sup

g∈V

∗

|φ(v)(g)|

kvk

kgk

∗

= sup

v∈V

sup

g∈V

∗

|g(v)|

kvk

kgk

∗

≤ 1.

In fact, we will later show that kφk

B(V,V

∗

)

= 1.

1.5 Isomorphism

So far, we have discussed a lot about bounded linear maps, which are “morphisms”

between normed vector spaces. It is thus natural to come up with the notion of

isomorphism.

Definition

(Isomorphism)

Let

X, Y

be normed vector spaces. Then

X → Y

is an isomorphism if it is a bounded linear map with a bounded linear inverse

(i.e. it is a homeomorphism).

We say X and Y are isomorphic if there is an isomorphism T : X → Y .

We say that

X → Y

is an isometric isomorphism if

is an isomorphism

and kT xk

= kxk

for all x ∈ X.

and

are isometrically isomorphic if there is an isometric isomorphism

between them.

Example.

Consider a finite-dimensional space

with the standard basis

, ··· , e

}. For any v =

, the norm is defined by

kvk =





1/2

Then any

g ∈ V

∗

is determined by

(

) for

= 1

, ··· , n

. We want to show that

there are no restrictions on what

(

) can be, i.e. whatever values I assign to

them, g will still be bounded. We have

kgk

∗

= sup

v∈V

|g(v)|

kvk

≤ sup

v∈V

||g(e

(

)

1/2

≤ C sup

v∈V

(

)

(

)



sup

|g(e



= C sup

|g(e

for some

, where the second-to-last line is due to the Cauchy-Schwarz inequality.

The supremum is finite since F

is finite dimensional.

Since

is uniquely determined by a list of values (

(

)

, g

(

)

, ··· , g

(

)),

it has dimension

. Therefore,

∗

is isomorphic to

. By the same lines of

argument, V

∗∗

is isomorphic to F

In fact, we can show that

V → V

∗∗

(

)(

) =

(

) is an isometric

isomorphism (this is not true for general normed vector spaces. Just pick

be incomplete, then V and V

∗∗

cannot be isomorphic since V

∗∗

is complete).

Example. Consider `

for p ∈ [1, ∞). What is `

∗

Suppose q is the conjugate exponent of p, i.e.

= 1.

(if p = 1, define q = ∞) It is easy to see that `

⊆ `

∗

by the following:

Suppose (

, x

, ···

)

∈ `

, and (

, y

, ···

)

∈ `

. Define

(

) =

∞

i=1

We will show that

defined this way is a bounded linear map. Linearity is easy

to see, and boundedness comes from the fact that

kyk

∗

= sup

ky(x)k

kxk

= sup

kxk

≤ sup

kxk

kyk

kxk

= kyk

by the H¨older’s inequality. So every (

)

∈ `

determines a bounded linear map.

In fact, we can show `

∗

is isomorphic to `

1.6 Finite-dimensional normed vector spaces

We are now going to look at a special case of normed vector spaces, where the

vector space is finite dimensional.

It turns out that finite-dimensional vector spaces are rather boring. In

particular, we have

(i) All norms are equivalent.

(ii) The closed unit ball is compact.

(iii) They are Banach spaces.

(iv) All linear maps whose domain is finite dimensional are bounded.

These are what we are going to show in this section.

First of all, we need to say what we mean when we say all norms are

“equivalent”

Definition

(Equivalent norms)

Let

be a vector space, and

k · k

norms on

. We say that these are equivalent if there exists a constant

C >

such that for any v ∈ V , we have

−1

kvk

≤ kvk

≤ Ckvk

It is an exercise to show that equivalent norms induce the same topology,

and hence agree on continuity and convergence. Also, equivalence of norms is an

equivalence relation (as the name suggests).

Now let

be an

-dimensional vector space with basis

, ··· , e

}

. We

can define the `

norm by

kvk

i=1

1/p

where

v =

i=1

Proposition.

Let

be an

-dimensional vector space. Then all norms on

are equivalent to the norm k · k

Corollary. All norms on a finite-dimensional vector space are equivalent.

Proof. Let k · k be a norm on V .

Let v = (v

, ··· , v

) =

∈ V . Then we have

kvk =



≤

i=1

|ke

≤



sup



i=1

≤ Ckvk

where C = sup ke

k < ∞ since we are taking a finite supremum.

For the other way round, let

{v ∈ V

kvk

= 1

}

. We will show the

two following results:

(i) k · k : (S

, k · k

) → R is continuous.

(ii) S

is a compact set.

We first see why this gives what we want. We know that for any continuous map

from a compact set to

, the image is bounded and the infimum is achieved. So

there is some v

∗

∈ S

such that

∗

k = inf

v∈S

kvk.

Since v

∗

6= 0, there is some c

such that kvk ≥ c

for all v ∈ S

Now take an arbitrary non-zero v ∈ V , since

kvk

∈ S

, we know that



kvk



≥ c

which is to say that

kvk ≥ c

kvk

Since we have found c, c

> 0 such that

kvk

≤ kvk ≤ ckvk

now let C = max





> 0. Then

−1

kvk

≤ kvk

≤ Ckvk

So the norms are equivalent. Now we can start to prove (i) and (ii).

First, let v, w ∈ V . We have



kvk −kwk



≤ kv − wk ≤ Ckv − wk

Hence when

is close to

under

, then

kvk

is close to

kwk

. So it is continuous.

To show (ii), it suffices to show that the unit ball

{v ∈ V

kvk

≤

}

is compact, since

is a closed subset of

. We will do so by showing it is

sequentially compact.

Let {v

(k)

}

∞

k=1

be a sequence in B. Write

(k)

i=1

(k)

Since v

(k)

∈ B, we have

i=1

|λ

(k)

| ≤ 1.

Consider the sequence λ

(k)

, which is a sequence in F.

We know that

|λ

(k)

| ≤

1. So by Bolzano-Weierstrass, there is a convergent

subsequence λ

)

Now look at

)

. Since this is bounded, there is a convergent subsequence

)

Iterate this for all

to obtain a sequence

such that

)

is convergent

for all i. So v

)

is a convergent subsequence.

Proposition.

Let

be a finite-dimensional normed vector space. Then the

closed unit ball

B(1) = {v ∈ V : kvk ≤ 1}

is compact.

Proof. This follows from the proof above.

Proposition.

Let

be a finite-dimensional normed vector space. Then

is a

Banach space.

Proof.

Let

} ∈ V

be a Cauchy sequence. Since

}

is Cauchy, it is bounded,

i.e.

} ⊆

(

) for some

R >

0. By above,

(

) is compact. So

}

has a

convergent subsequence

→ v

. Since

}

is Cauchy, we must have

→ v

So v

converges.

Proposition.

Let

V, W

be normed vector spaces,

be finite-dimensional. Also,

let T : V → W be a linear map. Then T is bounded.

Proof.

Recall discussions last time about

∗

for finite-dimensional

. We will

do a similar proof.

Note that since

is finite-dimensional,

im T

finite dimensional. So wlog

is finite-dimensional. Since all norms are equivalent, it suffices to consider the

case where the vector spaces have

and

norm. This can be represented by

a matrix T

such that

T (x

, ··· , x

) =



, ··· ,



We can bound this by

kT (x

, ··· , x

)k ≤

j=1

i=1

||x

| ≤ m



sup

i,j



i=1

| ≤ Ckxk

for some

C >

0, since we are taking the supremum over a finite set. This implies

that kT k

B(`

)

≤ C.

There is another way to prove this statement.

Proof.

(alternative) Let

V → W

be a linear map. We define a norm on

by kvk

= kvk

+ kT vk

. It is easy to show that this is a norm.

Since

is finite dimensional, all norms are equivalent. So there is a constant

C > 0 such that for all v, we have

kvk

≤ Ckvk

In particular, we have

kT vk ≤ Ckvk

So done.

Among all these properties, compactness of

(1) characterizes finite dimen-

sionality.

Proposition.

Let

be a normed vector space. Suppose that the closed unit

ball

B(1) is compact. Then V is finite dimensional.

Proof. Consider the following open cover of

B(1):

B(1) ⊆

[

y∈

B(1)





Since

(1) is compact, this has a finite subcover. So there is some

, ··· , y

such that

B(1) ⊆

[

i=1





Now let

span{y

, ··· , y

}

, which is a finite-dimensional subspace of

. We

want to show that in fact we have Y = V .

Clearly, by definition of Y , the unit ball

B(1) ⊆ Y + B





i.e. for every

v ∈ B

(1), there is some

y ∈ Y, w ∈ B

(

) such that

Multiplying everything by

, we get





⊆ Y + B





Hence we also have

B(1) ⊆ Y + B





By induction, for every n, we have

B(1) ⊆ Y + B





As a consequence,

B(1) ⊆

Y .

Since

is finite-dimensional, we know that

is complete. So

is a closed

subspace of V . So

Y = Y . So in fact

B(1) ⊆ Y.

Since every element in

can be rescaled to an element of

(1), we know that

V = Y . Hence V is finite dimensional.

This concludes our discussion on finite-dimensional vector spaces. We’ll end

with an example that shows these are not true for infinite dimensional vector

spaces.

Example.

Consider

, and

= (0

, ··· ,

, ···

), where

is 1 on the

entry, 0 elsewhere.

Note that if i 6= j, then

− e

k = 2.

Since

∈

(1), we see that

(1) cannot be covered by finitely many open balls

of radius

, since each open ball can contain at most one of {e

1.7 Hahn–Banach Theorem

Let

be a real normed vector space. What can we say about

∗

(

V, R

For instance, If V is non-trivial, must V

∗

be non-trivial?

The main goal of this section is to prove the Hahn–Banach theorem (surprise),

which allows us to produce a lot of elements in

∗

. Moreover, it doesn’t just tell

us that

∗

is non-empty (this is rather dull), but provides a tool to craft (or at

least prove existence of) elements of V

∗

that satisfy some property we want.

Proposition.

Let

be a real normed vector space, and

W ⊆ V

has co-

dimension 1. Assume we have the following two items:

– p : V → R (not necessarily linear), which is positive homogeneous, i.e.

p(λv) = λp(v)

for all v ∈ V, λ > 0, and subadditive, i.e.

p(v

+ v

) ≤ p(v

) + p(v

)

for all

, v

∈ V

. We can think of something like a norm, but more

general.

– f : W → R a linear map such that f(w) ≤ p(w) for all w ∈ W .

Then there exists an extension

V → R

which is linear such that

and

f(v) ≤ p(v) for all v ∈ V .

Why do we want this weird theorem? Our objective is to find something in

∗

. This theorem tells us that to find a bounded linear map in

, we just need

something in

bounded by a norm-like object, and then we can extend it to

Proof.

Let

∈ V \ W

. Since

has co-dimension 1, every element

v ∈ V

can be written uniquely as

, for some

w ∈ W, a ∈ R

. Therefore it

suffices to define

f(v

) and then extend linearly to V .

The condition we want to meet is

f(w + av

) ≤ p(w + av

) (∗)

for all

w ∈ W, a ∈ R

. If

= 0, then this is satisfied since

restricts to

If a > 0 then (∗) is equivalent to

f(w) + a

f(v

) ≤ p(w + av

We can divide by a to obtain

f(a

−1

w) +

f(v

) ≤ p(a

−1

w + v

We let w

= a

−1

w. So we can write this as

f(v

) ≤ p(w

+ v

) − f (w

for all w

∈ W .

If a < 0, then (∗) is equivalent to

f(w) + a

f(v

) ≤ p(w + av

We now divide by a and flip the sign of the equality. So we have

f(a

−1

w) +

f(v

) ≥ −(−a

−1

)p(w + av

In other words, we want

f(v

) ≥ −p(−a

−1

w −v

) − f (a

−1

w).

We let w

= −a

−1

w. Then we are left with

f(v

) ≥ −p(w

− v

) + f (w

for all w

∈ W .

Hence we are done if we can define a

(

) that satisfies these two conditions.

This is possible if and only if

−p(w

− v

) + f (w

) ≤ p(w

+ v

) − f (w

)

for all w

, w

. This holds since

f(w

) + f (w

) = f(w

+ w

)

≤ p(w

+ w

)

= p(w

− v

+ w

+ v

)

≤ p(w

− v

) + p(w

+ v

So the result follows.

The goal is to “iterate” this to get a similar result without the co-dimension

1 assumption. While we can do this directly for finitely many times, this isn’t

helpful (since we already know a lot about finite dimensional normed spaces). To

perform an “infinite iteration”, we need the mysterious result known as Zorn’s

lemma.

Digression on Zorn’s lemma

We first need a few definitions before we can come to Zorn’s lemma.

Definition

(Partial order)

A relation

≤

on a set

is a partial order if it

satisfies

(i) x ≤ x (reflexivity)

(ii) x ≤ y and y ≤ x implies x = y (antisymmetry)

(iii) x ≤ y and y ≤ z implies x ≤ z (transitivity)

Definition

(Total order)

Let (

S, ≤

) be a partial order.

T ⊆ S

is totally ordered

if for all x, y ∈ T , either x ≤ y or y ≤ x, i.e. every two things are related.

Definition

(Upper bound)

Let (

S, ≤

) be a partial order.

⊆ S

subset. We

say b ∈ S is an upper bound of this subset if x ≤ b for all x ∈ S

Definition

(Maximal element)

Let (

S, ≤

) be a partial order. Then

m ∈ S

is a

maximal element if x ≥ m implies x = m.

The glorious Zorn’s lemma tells us that:

Lemma

(Zorn’s lemma)

Let (

S, ≤

) be a non-empty partially ordered set such

that every totally-ordered subset

has an upper bound in

. Then

has a

maximal element.

We will not give a proof of this lemma here, but can explain why it should

be true.

We start by picking one element

. If it is maximal, then done.

Otherwise, there is some

> x

. If this is not maximal, then pick

> x

. We

do this to infinity “and beyond” — after picking infinitely many

, if we have

not yet reached a maximal element, we take an upper bound of this set, and call

it x

. If this is not maximal, we can continue picking a larger element.

We can do this forever, but if this process never stops, even after infinite

time, we would have picked out more elements than there are in

, which is

clearly nonsense. Of course, this is hardly a formal proof. The proper proof can

be found in the IID Logic and Set Theory course.

Back to vector spaces

The Hahn–Banach theorem is just our previous proposition without the constraint

that W has co-dimension 1.

Theorem

(Hahn–Banach theorem*)

Let

be a real normed vector space, and

W ⊆ V a subspace. Assume we have the following two items:

– p

V → R

(not necessarily linear), which is positive homogeneous and

subadditive;

– f : W → R a linear map such that f(w) ≤ p(w) for all w ∈ W .

Then there exists an extension

V → R

which is linear such that

and

f(v) ≤ p(v) for all v ∈ V .

Proof. Let S be the set of all pairs (

V ,

f) such that

(i) W ⊆

V ⊆ V

(ii)

f :

V → R is linear

(iii)

= f

(iv)

v) ≤ p(

v) for all

v ∈ V

We introduce a partial order

≤

by (

)

≤

(

) if

⊆

and

. It is easy to see that this is indeed a partial order.

We now check that this satisfies the assumptions of Zorn’s lemma. Let

{(

)}

α∈A

⊆ S be a totally ordered set. Define (

V ,

f) by

V =

[

α∈A

f(x) =

(x) for x ∈

This is well-defined because

{

(

V ,

)

}

α∈A

is totally ordered. So if

x ∈

and

x ∈

, wlog assume (

)

≤

(

). So

. So

(x) =

(x).

It should be clear that (

V ,

)

∈ S

and (

V ,

) is indeed an upper bound of

{(

)}

α∈A

. So the conditions of Zorn’s lemma are satisfied.

Hence by Zorn’s lemma, there is an maximal element (

W ,

)

∈ S

. Then by

definition,

is linear, restricts to

, and bounded by

. We now show

that

W = V .

Suppose not. Then there is some

∈ V \

. Define

span{

W , v

}

Now

is a co-dimensional 1 subspace of

. By our previous result, we know

that there is some

V → R

linear such that

and

(

)

≤ p

(

) for all

v ∈

V .

Hence we have (

W ,

)

∈ S

but (

W ,

)

(

V ,

). This contradicts the

maximality of (

W ,

f).

There is a particularly important special case of this, which is also known as

Hahn-Banach theorem sometimes.

Corollary

(Hahn-Banach theorem 2.0)

Let

W ⊆ V

be real normed vector

spaces. Given

f ∈ W

∗

, there exists a

f ∈ V

∗

such that

and

∗

kfk

∗

Proof.

Use the Hahn-Banach theorem with

(

) =

kfk

∗

kxk

for all

x ∈ V

Positive homogeneity and subadditivity follow directly from the axioms of the

norm. Then by definition

(

)

≤ p

(

) for all

w ∈ W

. So Hahn-Banach theorem

says that there is

V → R

linear such that

and

(

)

≤ p

(

) =

kfk

∗

kvk

Now notice that

f(v) ≤ kfk

∗

kvk

, −

f(v) =

f(−v) ≤ kfk

∗

kvk

implies that |

f(v)| ≤ kfk

∗

kvk

for all v ∈ V .

On the other hand, we have (again taking supremum over non-zero v)

∗

= sup

v∈V

f(v)|

kvk

≥ sup

w∈W

|f(w)|

kwk

= kfk

∗

So indeed we have k

∗

= kfk

∗

We’ll have some quick corollaries of these theorems.

Proposition.

Let

be a real normed vector space. For every

v ∈ V \ {

}

there is some f

∈ V

∗

such that f

(v) = kvk

and kf

∗

= 1.

Proof.

Apply Hahn-Banach theorem (2.0) with

span{v}

(

) =

kvk

Corollary.

Let

be a real normed vector space. Then

if and only if

f(v) = 0 for all f ∈ V

∗

Corollary.

Let

be a non-trivial real normed vector space,

v, w ∈ V

with

v 6= w. Then there is some f ∈ V

∗

such that f (v) 6= f(w).

Corollary.

is a non-trivial real normed vector space, then

∗

is non-trivial.

We now want to restrict the discussion to double duals. We define

V →

∗∗

as before by φ(v)(f) = f(v) for v ∈ V, f ∈ V

∗

Proposition. The map φ : V → V

∗∗

is an isometry, i.e. kφ(v)k

∗∗

= kvk

Proof. We have previously shown that

kφk

B(V,V

∗∗

)

≤ 1.

It thus suffices to show that the norm is greater than 1, or that

kφ(v)k

∗∗

≥ kvk

We can assume v 6= 0, for which the inequality is trivial. We have

kφ(v)k

∗∗

= sup

f∈V

∗

|φ(v)(f)|

kfk

∗

≥

|φ(v)(f

∗

= |f

(v)| = kvk

where

is the function such that

(

) =

kvk

, kf

∗

= 1 as we have

previously defined.

So done.

In particular,

is injective and one can view

as an isometric embedding of

V into V

∗∗

Definition (Reflexive). We say V is reflexive if φ(V ) = V

∗∗

Note that any reflexive space is Banach, since V

∗∗

, being the dual of V

∗

, is

reflexive.

You might have heard that for any infinite dimensional vector space V , the

dual of

is always strictly larger than

. This does not prevent an infinite

dimensional vector space from being reflexive. When we said the dual of

always strictly larger than

, we are referring to the algebraic dual, i.e. the set of

all linear maps from

. In the definition of reflexive (and everywhere else

where we mention “dual” in this course), we mean the continuous dual, where

we look at the set of all bounded linear maps from

. It is indeed possible

for the continuous dual to be isomorphic to the original space, even in infinite

dimensional spaces, as we will see later.

Example.

Finite-dimensional normed vector spaces are reflexive. Also

reflexive for p ∈ (1, ∞).

Recall that given T ∈ B(V, W ), we defined T

∗

∈ B(W

∗

, V

∗

) by

∗

(f)(v) = f(T v)

for v ∈ V, f ∈ W

∗

We have previously shown that

∗

B(W

∗

)

≤ kT k

B(V,W )

We will now show that in fact equality holds.

Proposition.

∗

B(W

∗

)

= kT k

B(V,W )

Proof. We have already shown that

∗

B(W

∗

)

≤ kT k

B(V,W )

For the other inequality, first let ε > 0. Since

kT k

B(V,W )

= sup

v∈V

kT vk

kvk

by definition, there is some

v ∈ V

such that

kT vk

≥ kTk

B(V,W )

kvk

− ε

wlog, assume kvk

= 1. So

kT vk

≥ kT k

B(V,W )

− ε.

Therefore, we get that

∗

B(W

∗

)

= sup

f∈W

∗

(f)k

∗

kfk

∗

≥ kT

∗

T v

∗

≥ |T

∗

T v

)(v)|

= |f

T v

(T v)|

= kT vk

≥ kT k

B(V,W )

− ε,

where we used the fact that

T v

∗

and

kvk

are both 1. Since

is arbitrary,

we are done.

2 Baire category theorem

2.1 The Baire category theorem

When we first write the Baire category theorem down, it might seem a bit

pointless. However, it turns out to be a really useful result, and we will be able

to prove surprisingly many results from it.

In fact, the Baire category theorem itself does not involve normed vector

spaces. It works on any metric space. However, most of the applications we have

here are about normed vector spaces.

To specify the theorem, we will need some terminology.

Definition

(Nowhere dense set)

Let

be a topological space. A subset

E ⊆ X

is nowhere dense if

E has empty interior.

Usually, we will pick