4ℝn as a normed space
IB Analysis II
4.1 Normed spaces
Our objective is to extend most of the notions we had about functions of a
single variable
f
:
R → R
to functions of multiple variables
f
:
R
n
→ R
. More
generally, we want to study functions
f
: Ω
→ R
m
, where Ω
⊆ R
n
. We wish to
define analytic notions such as continuity, differentiability and even integrability
(even though we are not doing integrability in this course).
In order to do this, we need more structure on
R
n
. We already know that
R
n
is a vector space, which means that we can add, subtract and multiply by
scalars. But to do analysis, we need something to replace our notion of
|x − y|
in R. This is known as a norm.
It is useful to define and study this structure in an abstract setting, as
opposed to thinking about
R
n
specifically. This leads to the general notion of
normed spaces.
Definition (Normed space). Let
V
be a real vector space. A norm on
V
is a
function ∥ · ∥ : V → R satisfying
(i) ∥x∥ ≥ 0 with equality iff x = 0 (non-negativity)
(ii) ∥λx∥ = |λ|∥x∥ (linearity in scalar multiplication)
(iii) ∥x + y∥ ≤ ∥x∥ + ∥y∥ (triangle inequality)
A normed space is a pair (
V, ∥ · ∥
). If the norm is understood, we just say
V
is
a normed space. We do have to be slightly careful since there can be multiple
norms on a vector space.
Intuitively, ∥x∥ is the length or magnitude of x.
Example. We will first look at finite-dimensional spaces. This is typically
R
n
with different norms.
– Consider R
n
, with the Euclidean norm
∥x∥
2
=
X
x
2
i
2
.
This is also known as the usual norm. It is easy to check that this is a
norm, apart from the triangle inequality. So we’ll just do this. We have
∥x + y∥
2
=
n
X
i=1
(x
i
+ y
i
)
2
= ∥x∥
2
+ ∥y∥
2
+ 2
X
x
i
y
i
≤ ∥x∥
2
+ ∥y∥
2
+ 2∥x∥y∥
= (∥x∥
2
+ ∥y∥
2
),
where we used the Cauchy-Schwarz inequality. So done.
– We can have the following norm on R
n
:
∥x∥
1
=
X
|x
i
|.
It is easy to check that this is a norm.
– We can also have the following norm on R
n
:
∥x∥
∞
= max{|x
i
| : 1 ≤ i ≤ n}.
It is also easy to check that this is a norm.
– In general, we can define the p norm (for p ≥ 1) by
∥x∥
p
=
X
|x
i
|
p
1/p
.
It is, however, not trivial to check the triangle inequality, and we will not
do this.
We can show that as
p → ∞
,
∥
x
∥
p
→ ∥
x
∥
∞
, which justifies our notation
above.
We also have some infinite dimensional examples. Often, we can just extend our
notions on
R
n
to infinite sequences with some care. We write
R
N
for the set of
all infinite real sequences (
x
k
). This is a vector space with termwise addition
and scalar multiplication.
– Define
ℓ
1
=
n
(x
k
) ∈ R
N
:
X
|x
k
| < ∞
o
.
This is a linear subspace of R
N
. We define the norm by
∥(x
k
)∥
1
= ∥(x
k
)∥
ℓ
1
=
X
|x
k
|.
– Similarly, we can define ℓ
2
by
ℓ
2
=
n
(x
k
) ∈ R
N
:
X
x
2
k
< ∞
o
.
The norm is defined by
∥(x
k
)∥
2
= ∥(x
k
)∥
ℓ
2
=
X
x
2
k
1/2
.
We can also write this as
∥(x
k
)∥
ℓ
2
= lim
n→∞
∥(x
1
, ··· , x
n
)∥
2
.
So the triangle inequality for the Euclidean norm implies the triangle
inequality for ℓ
2
.
– In general, for p ≥ 1, we can define
ℓ
p
=
n
(x
k
) ∈ R
N
:
X
|x
k
|
p
< ∞
o
with the norm
∥(x
k
)∥
p
= ∥(x
k
)∥
ℓ
p
=
X
|x
k
|
p
1/p
.
– Finally, we have ℓ
∞
, where
ℓ
∞
= {(x
k
) ∈ R
N
: sup |x
k
| < ∞},
with the norm
∥(x
k
)∥
∞
= ∥(x
k
)∥
ℓ
∞
= sup |x
k
|.
Finally, we can have examples where we look at function spaces, usually
C
([
a, b
]),
the set of continuous real functions on [a, b].
– We can define the L
1
norm by
∥f∥
L
1
= ∥f∥
1
=
Z
b
a
|f| dx.
– We can define L
2
similarly by
∥f∥
L
2
= ∥f∥
2
=
Z
b
a
f
2
dx
!
1
2
.
– In general, we can define L
p
for p ≥ 1 by
∥f∥
L
p
= ∥f∥
p
=
Z
b
a
f
p
dx
!
1
p
.
– Finally, we have L
∞
by
∥f∥
L
∞
= ∥f∥
∞
= sup |f|.
This is also called the uniform norm, or the supremum norm.
Later, when we define convergence for general normed space, we will show
that convergence under the uniform norm is equivalent to uniform convergence.
To show that
L
2
is actually a norm, we can use the Cauchy-Schwarz inequality
for integrals.
Lemma (Cauchy-Schwarz inequality (for integrals)). If
f, g ∈ C
([
a, b
]),
f, g ≥
0,
then
Z
b
a
fg dx ≤
Z
b
a
f
2
dx)
!
1/2
Z
b
a
g
2
dx
!
1/2
.
Proof.
If
R
b
a
f
2
d
x
= 0, then
f
= 0 (since
f
is continuous). So the inequality
holds trivially.
Otherwise, let A
2
=
R
b
a
f
2
dx = 0, B
2
=
R
b
a
g
2
dx. Consider the function
ϕ(t) =
Z
b
a
(g − tf)
2
dt ≥ 0.
for every t. We can expand this as
ϕ(t) = t
2
A
2
− 2t
Z
b
a
gf dx + B
2
.
The conditions for a quadratic in t to be non-negative is exactly
Z
b
a
gf dx
!
2
− A
2
B
2
≤ 0.
So done.
Note that the way we defined
L
p
is rather unsatisfactory. To define the
ℓ
p
spaces, we first have the norm defined as a sum, and then
ℓ
p
to be the set of
all sequences for which the sum converges. However, to define the
L
p
space, we
restrict ourselves to
C
([0
,
1]), and then define the norm. Can we just define,
say,
L
1
to be the set of all functions such that
R
1
0
|f|
d
x
exists? We could,
but then the norm would no longer be the norm, since if we have the function
f
(
x
) =
(
1 x = 0.5
0 x = 0.5
, then
f
is integrable with integral 0, but is not identically
zero. So we cannot expand our vector space to be too large. To define
L
p
properly, we need some more sophisticated notions such as Lebesgue integrability
and other fancy stuff, which will be done in the IID Probability and Measure
course.
We have just defined many norms on the same space
R
n
. These norms are
clearly not the same, in the sense that for many x,
∥
x
∥
1
and
∥
x
∥
2
have different
values. However, it turns out the norms are all “equivalent” in some sense. This
intuitively means the norms are “not too different” from each other, and give
rise to the same notions of, say, convergence and completeness.
A precise definition of equivalence is as follows:
Definition (Lipschitz equivalence of norms). Let
V
be a (real) vector space.
Two norms
∥ · ∥, ∥ · ∥
′
on
V
are Lipschitz equivalent if there are real constants
0 < a < b such that
a∥x∥ ≤ ∥x∥
′
≤ b∥x∥
for all x ∈ V .
It is easy to show this is indeed an equivalence relation on the set of all norms
on V .
We will show that if two norms are equivalent, the “topological” properties of
the space do not depend on which norm we choose. For example, the norms will
agree on which sequences are convergent and which functions are continuous.
It is possible to reformulate the notion of equivalence in a more geometric
way. To do so, we need some notation:
Definition (Open ball). Let (
V, ∥ · ∥
) be a normed space, a
∈ V
,
r >
0. The
open ball centered at a with radius r is
B
r
(a) = {x ∈ V : ∥x − a∥ < r}.
Then the requirement that
a∥
x
∥ ≤ ∥
x
∥
′
≤ b∥
x
∥
for all x
∈ V
is equivalent
to saying
B
1/b
(0) ⊆ B
′
1
(0) ⊆ B
1/a
(0),
where
B
′
is the ball with respect to
∥ · ∥
′
, while
B
is the ball with respect to
∥ · ∥. Actual proof of equivalence is on the second example sheet.
Example. Consider
R
2
. Then the norms
∥ · ∥
∞
and
∥ · ∥
2
are equivalent. This
is easy to see using the ball picture:
where the blue ones are the balls with respect to
∥ · ∥
∞
and the red one is the
ball with respect to ∥ · ∥
2
.
In general, we can consider R
n
, again with ∥ · ∥
2
and ∥ · ∥
∞
. We have
∥x∥
∞
≤ ∥x∥
2
≤
√
n∥x∥
∞
.
These are easy to check manually. However, later we will show that in fact,
any two norms on a finite-dimensional vector space are Lipschitz equivalent.
Hence it is more interesting to look at infinite dimensional cases.
Example. Let V = C([0, 1]) with the norms
∥f∥
1
=
Z
1
0
|f| dx, ∥f∥
∞
= sup
[0,1]
|f|.
We clearly have the bound
∥f∥
1
≤ ∥f∥
∞
.
However, there is no constant b such that
∥f∥
∞
≤ b∥f∥
1
for all f. This is easy to show by constructing a sequence of functions f
n
by
x
y
1
1
n
where the width is
2
n
and the height is 1. Then
∥f
n
∥
∞
= 1 but
∥f
n
∥
1
=
1
n
→
0.
Example. Similarly, consider the space
ℓ
2
=
(x
n
) :
P
x
2
n
< ∞
under the
regular ℓ
2
norm and the ℓ
∞
norm. We have
∥(x
k
)∥
∞
≤ ∥(x
k
)∥
ℓ
2
,
but there is no b such that
∥(x
k
)∥
ℓ
2
≤ b∥(x
k
)∥
∞
.
For example, we can consider the sequence
x
n
= (1
,
1
, ··· ,
1
,
0
,
0
, ···
), where the
first n terms are 1.
So far in all our examples, out of the two inequalities, one holds and one does
not. Is it possible for both inequalities to not hold? The answer is yes. This is
an exercise on the second example sheet as well.
This is all we are going to say about Lipschitz equivalence. We are now going
to define convergence, and study the consequences of Lipschitz equivalence to
convergence.
Definition (Bounded subset). Let (
V, ∥ · ∥
) be a normed space. A subset
E ⊆ V
is bounded if there is some R > 0 such that
E ⊆ B
R
(0).
Definition (Convergence of sequence). Let (
V, ∥ · ∥
) be a normed space. A
sequence (
x
k
) in
V
converges to x
∈ V
if
∥
x
k
−
x
∥ →
0 (as a sequence in
R
), i.e.
(∀ε > 0)(∃N )(∀k ≥ N) ∥x
k
− x∥ < ε.
These two definitions, obviously, depends on the chosen norm, not just the
vector space
V
. However, if two norms are equivalent, then they agree on what
is bounded and what converges.
Proposition. If
∥ · ∥
and
∥ · ∥
′
are Lipschitz equivalent norms on a vector
space V , then
(i)
A subset
E ⊆ V
is bounded with respect to
∥ · ∥
if and only if it is bounded
with respect to ∥ · ∥
′
.
(ii)
A sequence
x
k
converges to
x
with respect to
∥ · ∥
if and only if it converges
to x with respect to ∥ · ∥
′
.
Proof.
(i) This is direct from definition of equivalence.
(ii) Say we have a, b such that a∥y∥ ≤ ∥y∥
′
≤ b∥y∥ for all y. So
a∥x
k
− x∥ ≤ ∥x
k
− x∥
′
≤ b∥x
k
− x∥.
So ∥x
k
− x∥ → 0 if and only if ∥x
k
− x∥
′
→ 0. So done.
What if the norms are not equivalent? It is not surprising that there are
some sequences that converge with respect to one norm but not another. More
surprisingly, it is possible that a sequence converges to different limits under
different norms. This is, again, on the second example sheet.
We have some easy facts about convergence:
Proposition. Let (V, ∥ · ∥) be a normed space. Then
(i) If x
k
→ x and x
k
→ y, then x = y.
(ii) If x
k
→ x, then ax
k
→ ax.
(iii) If x
k
→ x, y
k
→ y, then x
k
+ y
k
→ x + y.
Proof.
(i) ∥x − y∥ ≤ ∥x − x
k
∥ + ∥x
k
− y∥ → 0. So ∥x − y∥ = 0. So x = y .
(ii) ∥ax
k
− ax∥ = |a|∥x
k
− x∥ → 0.
(iii) ∥(x
k
+ y
k
) − (x + y)∥ ≤ ∥x
k
− x∥ + ∥y
k
− y∥ → 0.
Proposition. Convergence in
R
n
(with respect to, say, the Euclidean norm) is
equivalent to coordinate-wise convergence, i.e. x
(k)
→
x if and only if
x
(k)
j
→ x
j
for all j.
Proof.
Fix
ε >
0. Suppose x
(k)
→
x. Then there is some
N
such that for any
k ≥ N such that
∥x
(k)
− x∥
2
2
=
n
X
j=1
(x
(k)
j
− x
j
)
2
< ε.
Hence |x
(k)
j
− x
j
| < ε for all k ≤ N.
On the other hand, for any fixed
j
, there is some
N
j
such that
k ≥ N
j
implies
|x
(k)
j
− x
j
| <
ε
√
n
. So if k ≥ max{N
j
: j = 1, ··· , n}, then
∥x
(k)
− x∥
2
=
n
X
j=1
(x
(k)
j
− x
j
)
2
1
2
< ε.
So done
Another space we would like to understand is the space of continuous functions.
It should be clear that uniform convergence is the same as convergence under the
uniform norm, hence the name. However, there is no norm such that convergence
under the norm is equivalent to pointwise convergence, i.e. pointwise convergence
is not normable. In fact, it is not even metrizable. However, we will not prove
this.
We’ll now generalize the Bolzano-Weierstrass theorem to R
n
.
Theorem (Bolzano-Weierstrass theorem in
R
n
). Any bounded sequence in
R
n
(with, say, the Euclidean norm) has a convergent subsequence.
Proof.
We induct on
n
. The
n
= 1 case is the usual Bolzano-Weierstrass on the
real line, which was proved in IA Analysis I.
Assume the theorem holds in
R
n−1
, and let x
(k)
= (
x
(k)
1
, ··· , x
(k)
n
) be a
bounded sequence in
R
n
. Then let y
(k)
= (
x
(k)
1
, ··· , x
(k)
n−1
). Since for any
k
, we
know that
∥y
(k)
∥
2
+ |x
(k)
n
|
2
= ∥x
(k)
∥
2
,
it follows that both (y
(k)
) and (
x
(k)
n
) are bounded. So by the induction hypothesis,
there is a subsequence (
k
j
) of (
k
) and some y
∈ R
n−1
such that y
(k
j
)
→
y. Also,
by Bolzano-Weierstrass in
R
, there is a further subsequence (
x
(k
j
ℓ
)
n
) of (
x
(k
j
)
n
)
that converges to, say, y
n
∈ R. Then we know that
x
(k
j
ℓ
)
→ (y, y
n
).
So done.
Note that this is generally not true for normed spaces. Finite-dimensionality
is important for both of these results.
Example. Consider (
ℓ
∞
, ∥ · ∥
∞
). We let
e
(k)
j
=
δ
jk
be the sequence with 1
in the
k
th component and 0 in other components. Then
e
(k)
j
→
0 for all fixed
j
, and hence
e
(k)
converges componentwise to the zero element 0 = (0
,
0
, ···
).
However,
e
(k)
does not converge to the zero element since
∥e
(k)
−
0
∥
∞
= 1 for
all
k
. Also, this is bounded but does not have a convergent subsequence for the
same reasons.
We know that all finite dimensional vector spaces are isomorphic to
R
n
as vector spaces for some
n
, and we will later show that all norms on finite
dimensional spaces are equivalent. This means every finite-dimensional normed
space satisfies the Bolzano-Weierstrass property. Is the converse true? If a
normed vector space satisfies the Bolzano-Weierstrass property, must it be finite
dimensional? The answer is yes, and the proof is in the example sheet.
Example. Let
C
([0
,
1]) have the
∥ · ∥
L
2
norm. Consider
f
n
(
x
) =
sin
2
nπx
. We
know that
∥f
n
∥
2
L
2
=
Z
1
0
|f
n
|
2
=
1
2
.
So it is bounded. However, it doesn’t have a convergent subsequence. If it did,
say f
n
j
→ f in L
2
, then we must have
∥f
n
j
− f
n
j+1
∥
2
→ 0.
However, by direct calculation, we know that
∥f
n
j
− f
n
j+1
∥
2
=
Z
1
0
(sin 2n
j
πx − sin 2n
j+1
πx)
2
= 1.
Note that the same argument shows also that the sequence (
sin
2
nπx
) has no
subsequence that converges pointwise on [0
,
1]. To see this, we need the result
that if (
f
j
) is a sequence in
C
([0
,
1]) that is uniformly bounded with
f
j
→ f
pointwise, then
f
j
converges to
f
under the
L
2
norm. However, we will not
be able to prove this (in a nice way) without Lebesgue integration from IID
Probability and Measure.