III Advanced Probability - Martingales in discrete time

2Martingales in discrete time

III Advanced Probability

2.3 Martingale convergence theorems

One particularly nice property of martingales is that they have nice conver-

gence properties. We shall begin by proving a pointwise version of martingale

convergence.

Theorem

(Almost sure martingale convergence theorem)

Suppose (

)

n≥0

is a super-martingale that is bounded in

, i.e.

sup

E|X

| < ∞

. Then there

exists an F

∞

-measurable X

∞

∈ L

such that

→ X

∞

a.s. as n → ∞.

To begin, we need a convenient characterization of when a series converges.

Definition

(Upcrossing)

Let (

) be a sequence and (

a, b

) an interval. An

upcrossing of (

a, b

) by (

) is a sequence

j, j

+ 1

, . . . , k

such that

≤ a

and

≥ b. We define

[a, b, (x

)] = number of disjoint upcrossings contained in {1, . . . , n}

U[a, b, (x

)] = lim

n→∞

[a, b, x].

We can then make the following elementary observation:

Lemma.

Let (

)

n≥0

be a sequence of numbers. Then

converges in

if and

only if

(i) lim inf |x

| < ∞.

(ii) For all a, b ∈ Q with a < b, we have U[a, b, (x

)] < ∞.

For our martingales, since they are bounded in

, Fatou’s lemma tells us

E lim inf |X

| < ∞

. So

lim inf |X

= 0 almost surely. Thus, it remains to show

that for any fixed

a < b ∈ Q

, we have

(

[

a, b,

(

)] =

∞

) = 0. This is a

consequence of Doob’s upcrossing lemma.

Lemma (Doob’s upcrossing lemma). If X

is a super-martingale, then

(b −a)E(U

[a, b(X

)]) ≤ E(X

− a)

−

Proof.

Assume that

is a positive super-martingale. We define stopping times

, T

as follows:

– T

= 0

– S

k+1

= inf{n : X

≤ a, n ≥ T

}

– T

k+1

= inf{n : X

≥ b, n ≥ S

k+1

Given an

, we want to count the number of upcrossings before

. There are

two cases we might want to distinguish:

Now consider the sum

k=1

∧n

− X

∧n

In the first case, this is equal to

k=1

− X

k=U

− X

≥ (b − a)U

In the second case, it is equal to

k=1

−X

+(X

−X

k=U

−X

≥ (b−a)U

+(X

−X

Thus, in general, we have

k=1

∧n

− X

∧n

≥ (b − a)U

+ (X

− X

∧n

By definition,

< T

≤ n

. So the expectation of the LHS is always non-negative

by super-martingale convergence, and thus

0 ≥ (b − a)EU

+ E(X

− X

∧n

Then observe that

− X

≥ −(X

− a)

−

The almost-sure martingale convergence theorem is very nice, but often it is

not good enough. For example, we might want convergence in

instead. The

following example shows this isn’t always possible:

Example. Suppose (ρ

)

n≥0

is a sequence of iid random variables and

P(ρ

= 0) =

= P(ρ

= 2).

Let

k=0

Then this is a martingale, and

= 1. On the other hand,

→

0 almost

surely. So kX

− X

∞

does not converge to 0.

For

p >

1, if we want convergence in

, it is not surprising that we at least

need the sequence to be

bounded. We will see that this is in fact sufficient.

For

= 1, however, we need a bit more than being bounded in

. We will need

uniform integrability.

To prove this, we need to establish some inequalities.

Lemma

(Maximal inequality)

Let (

) be a sub-martingale that is non-

negative, or a martingale. Define

∗

= sup

k≤n

|, X

∗

= lim

n→∞

∗

If λ ≥ 0, then

λP(X

∗

≥ λ) ≤ E[|X

∗

≥λ

In particular, we have

λP(X

∗

≥ λ) ≤ E[|X

|].

Markov’s inequality says almost the same thing, but has

[

∗

] instead of

E[|X

|]. So this is a stronger inequality.

Proof.

is a martingale, then

is a sub-martingale. So it suffices to

consider the case of a non-negative sub-martingale. We define the stopping time

T = inf{n : X

≥ λ}.

By optional stopping,

≥ EX

T ∧n

= EX

T ≤n

+ EX

T >n

≥ λP(T ≤ n) + EX

T >n

= λP(X

∗

≥ λ) + EX

T >n

Lemma (Doob’s L

inequality). For p > 1, we have

∗

≤

p − 1

for all n.

Proof. Let k > 0, and consider

∗

∧ kk

= E|X

∗

∧ k|

We use the fact that

p−1

ds.

So we have

∗

∧ kk

= E|X

∗

∧ k|

= E

∗

∧k

p−1

= E

p−1

∗

≥x

p−1

P(X

∗

≥ x) dx (Fubini)

≤

p−2

∗

≥x

dx (maximal inequality)

= EX

p−2

∗

≥x

dx (Fubini)

p − 1

∗

∧ k)

p−1

≤

p − 1

(E(X

∗

∧ k)

)

p−1

(H¨older)

p − 1

∗

∧ kk

p−1

Now take the limit k → ∞ and divide by kX

∗

p−1

Theorem

(

martingale convergence theorem)

Let (

)

n≥0

be a martingale,

and p > 1. Then the following are equivalent:

(i) (X

)

n≥0

is bounded in L

, i.e. sup

E|X

< ∞.

(ii)

(

)

n≥0

converges as

n → ∞

to a random variable

∞

∈ L

almost surely

and in L

(iii) There exists a random variable Z ∈ L

such that

= E(Z | F

)

Moreover, in (iii), we always have X

∞

= E(Z | F

∞

This gives a bijection between martingales bounded in

and

(

∞

sending (X

)

n≥0

7→ X

∞

Proof.

–

(i)

⇒

(ii): If (

)

n≥0

is bounded in

, then it is bounded in

. So by

the martingale convergence theorem, we know (

)

n≥0

converges almost

surely to X

∞

. By Fatou’s lemma, we have X

∞

∈ L

Now by monotone convergence, we have

∗

= lim

∗

≤

p − 1

sup

< ∞.

By the triangle inequality, we have

− X

∞

| ≤ 2X

∗

a.s.

So by dominated convergence, we know that X

→ X

∞

in L

– (ii) ⇒ (iii): Take Z = X

∞

. We want to prove that

= E(X

∞

| F

To do so, we show that

− E

(

∞

| F

)

= 0. For

n ≥ m

, we know

this is equal to

kE(X

| F

)−E(X

∞

| F

= kE(X

−X

∞

| F

≤ kX

−X

∞

→ 0

n → ∞

, where the last step uses Jensen’s. But it is also a constant. So

we are done.

–

(iii)

⇒

(i): Since expectation decreases

norms, we already know that

)

n≥0

is L

-bounded.

To show the “moreover” part, note that

n≥0

is a

-system that

generates F

∞

. So it is enough to prove that

∞

= E(E(Z | F

∞

But if A ∈ F

, then

∞

= lim

n→∞

= lim

n→∞

E(E(Z | F

)

= lim

n→∞

E(E(Z | F

∞

where the last step relies on the fact that 1

is F

-measurable.

We finally finish off the

= 1 case with the additional uniform integrability

condition.

Theorem

(Convergence in

)

Let (

)

n≥0

be a martingale. Then the follow-

ing are equivalent:

(i) (X

)

n≥0

is uniformly integrable.

(ii) (X

)

n≥0

converges almost surely and in L

(iii) There exists Z ∈ L

such that X

= E(Z | F

) almost surely.

Moreover, X

∞

= E(Z | F

∞

The proof is very similar to the L

case.

Proof.

–

(i)

⇒

(ii): Let (

)

n≥0

be uniformly integrable. Then (

)

n≥0

is bounded

. So the (

)

n≥0

converges to

∞

almost surely. Then by measure

theory, uniform integrability implies that in fact X

→ L

– (ii) ⇒ (iii): Same as the L

case.

–

(iii)

⇒

(i): For any

Z ∈ L

, the collection

(

Z | G

) ranging over all

σ-subalgebras G is uniformly integrable.

Thus, there is a bijection between uniformly integrable martingales and

∞

We now revisit optional stopping for uniformly integrable martingales. Recall

that in the statement of optional stopping, we needed our stopping times to be

bounded. It turns out if we require our martingales to be uniformly integrable,

then we can drop this requirement.

Theorem.

If (

)

n≥0

is a uniformly integrable martingale, and

S, T

are arbi-

trary stopping times, then E(X

| F

) = X

S∧T

. In particular EX

= X

Note that we are now allowing arbitrary stopping times, so

may be infinite

with non-zero probability. Hence we define

∞

n=0

T =n

+ X

∞

T =∞

Proof. By optional stopping, for every n, we know that

E(X

T ∧n

| F

) = X

S∧T ∧n

We want to be able to take the limit as

n → ∞

. To do so, we need to show

that things are uniformly integrable. First, we apply optional stopping to write

T ∧n

= E(X

| F

T ∧n

)

= E(E(X

∞

| F

) | F

T ∧n

)

= E(X

∞

| F

T ∧n

So we know (

)

n≥0

is uniformly integrable, and hence

n∧T

→ X

almost

surely and in L

To understand E(X

T ∧n

| F

), we note that

kE(X

n∧T

− X

| F

≤ kX

n∧T

− X

→ 0 as n → ∞.

So it follows that E(X

n∧T

| F

) → E(X

| F

) as n → ∞.