3Linear models
IB Statistics
3.5 Inference for β
We know that
ˆ
β ∼ N
p
(β, σ
2
(X
T
X)
−1
). So
ˆ
β
j
∼ N (β
j
, σ
2
(X
T
X)
−1
jj
).
The standard error of
ˆ
β
j
is defined to be
SE(
ˆ
β
j
) =
q
˜σ
2
(X
T
X)
−1
jj
,
where
˜σ
2
=
RSS/
(
n − p
). Unlike the actual variance
σ
2
(
X
T
X
)
−1
jj
, the standard
error is calculable from our data.
Then
ˆ
β
j
− β
j
SE(
ˆ
β
j
)
=
ˆ
β
j
− β
j
q
˜σ
2
(X
T
X)
−1
jj
=
(
ˆ
β
j
− β
j
)/
q
σ
2
(X
T
X)
−1
jj
p
RSS/((n − p)σ
2
)
By writing it in this somewhat weird form, we now recognize both the numer-
ator and denominator. The numerator is a standard normal
N
(0
,
1), and the
denominator is an independent
q
χ
2
n−p
/(n − p)
, as we have previously shown.
But a standard normal divided by χ
2
is, by definition, the t distribution. So
ˆ
β
j
− β
j
SE(
ˆ
β
j
)
∼ t
n−p
.
So a 100(1
− α
)% confidence interval for
β
j
has end points
ˆ
β
j
± SE
(
ˆ
β
j
)
t
n−p
(
α
2
).
In particular, if we want to test H
0
: β
j
= 0, we use the fact that under H
0
,
ˆ
β
j
SE(
ˆ
β
j
)
∼ t
n−p
.