2Hypothesis testing

IB Statistics

2 Hypothesis testing
Often in statistics, we have some hypothesis to test. For example, we want to
test whether a drug can lower the chance of a heart attack. Often, we will have
two hypotheses to compare: the null hypothesis states that the drug is useless,
while the alternative hypothesis states that the drug is useful. Quantitatively,
suppose that the chance of heart attack without the drug is
θ
0
and the chance
with the drug is
θ
. Then the null hypothesis is
H
0
:
θ
=
θ
0
, while the alternative
hypothesis is H
1
: θ = θ
0
.
It is important to note that the null hypothesis and alternative hypothesis
are not on equal footing. By default, we assume the null hypothesis is true.
For us to reject the null hypothesis, we need a lot of evidence to prove that.
This is since we consider incorrectly rejecting the null hypothesis to be a much
more serious problem than accepting it when we should not. For example, it is
relatively okay to reject a drug when it is actually useful, but it is terrible to
distribute drugs to patients when the drugs are actually useless. Alternatively,
it is more serious to deem an innocent person guilty than to say a guilty person
is innocent.
In general, let
X
1
, ··· , X
n
be iid, each taking values in
X
, each with unknown
pdf/pmf
f
. We have two hypotheses,
H
0
and
H
1
f
. On the basis of data
X = x, we make a choice between the two hypotheses.
Example.
A coin has
P
(
) =
θ
, and is thrown independently
n
times. We could
have H
0
: θ =
1
2
versus H
1
: θ =
3
4
.
Suppose
X
1
, ··· , X
n
are iid discrete random variables. We could have
H
0
:
the distribution is Poisson with unknown mean, and
H
1
: the distribution
is not Poisson.
General parametric cases: Let
X
1
, ··· , X
n
be iid with density
f
(
x | θ
).
f
is known while
θ
is unknown. Then our hypotheses are
H
0
:
θ
Θ
0
and
H
1
: θ Θ
1
, with Θ
0
Θ
1
= .
We could have
H
0
:
f
=
f
0
and
H
1
:
f
=
f
1
, where
f
0
and
f
1
are densities
that are completely specified but do not come form the same parametric
family.
Definition (Simple and composite hypotheses). A simple hypothesis
H
specifies
f completely (e.g. H
0
: θ =
1
2
). Otherwise, H is a composite hypothesis.