2Hypothesis testing

IB Statistics

2 Hypothesis testing

Often in statistics, we have some hypothesis to test. For example, we want to

test whether a drug can lower the chance of a heart attack. Often, we will have

two hypotheses to compare: the null hypothesis states that the drug is useless,

while the alternative hypothesis states that the drug is useful. Quantitatively,

suppose that the chance of heart attack without the drug is

θ

0

and the chance

with the drug is

θ

. Then the null hypothesis is

H

0

:

θ

=

θ

0

, while the alternative

hypothesis is H

1

: θ = θ

0

.

It is important to note that the null hypothesis and alternative hypothesis

are not on equal footing. By default, we assume the null hypothesis is true.

For us to reject the null hypothesis, we need a lot of evidence to prove that.

This is since we consider incorrectly rejecting the null hypothesis to be a much

more serious problem than accepting it when we should not. For example, it is

relatively okay to reject a drug when it is actually useful, but it is terrible to

distribute drugs to patients when the drugs are actually useless. Alternatively,

it is more serious to deem an innocent person guilty than to say a guilty person

is innocent.

In general, let

X

1

, ··· , X

n

be iid, each taking values in

X

, each with unknown

pdf/pmf

f

. We have two hypotheses,

H

0

and

H

1

, about

f

. On the basis of data

X = x, we make a choice between the two hypotheses.

Example.

–

A coin has

P

(

Heads

) =

θ

, and is thrown independently

n

times. We could

have H

0

: θ =

1

2

versus H

1

: θ =

3

4

.

–

Suppose

X

1

, ··· , X

n

are iid discrete random variables. We could have

H

0

:

the distribution is Poisson with unknown mean, and

H

1

: the distribution

is not Poisson.

–

General parametric cases: Let

X

1

, ··· , X

n

be iid with density

f

(

x | θ

).

f

is known while

θ

is unknown. Then our hypotheses are

H

0

:

θ ∈

Θ

0

and

H

1

: θ ∈ Θ

1

, with Θ

0

∩ Θ

1

= ∅.

–

We could have

H

0

:

f

=

f

0

and

H

1

:

f

=

f

1

, where

f

0

and

f

1

are densities

that are completely specified but do not come form the same parametric

family.

Definition (Simple and composite hypotheses). A simple hypothesis

H

specifies

f completely (e.g. H

0

: θ =

1

2

). Otherwise, H is a composite hypothesis.