3Linear models

IB Statistics

3.5 Inference for β

We know that

ˆ

β ∼ N

p

(β, σ

2

(X

T

X)

−1

). So

ˆ

β

j

∼ N (β

j

, σ

2

(X

T

X)

−1

jj

).

The standard error of

ˆ

β

j

is defined to be

SE(

ˆ

β

j

) =

q

˜σ

2

(X

T

X)

−1

jj

,

where

˜σ

2

=

RSS/

(

n − p

). Unlike the actual variance

σ

2

(

X

T

X

)

−1

jj

, the standard

error is calculable from our data.

Then

ˆ

β

j

− β

j

SE(

ˆ

β

j

)

=

ˆ

β

j

− β

j

q

˜σ

2

(X

T

X)

−1

jj

=

(

ˆ

β

j

− β

j

)/

q

σ

2

(X

T

X)

−1

jj

p

RSS/((n − p)σ

2

)

By writing it in this somewhat weird form, we now recognize both the numer-

ator and denominator. The numerator is a standard normal

N

(0

,

1), and the

denominator is an independent

q

χ

2

n−p

/(n − p)

, as we have previously shown.

But a standard normal divided by χ

2

is, by definition, the t distribution. So

ˆ

β

j

− β

j

SE(

ˆ

β

j

)

∼ t

n−p

.

So a 100(1

− α

)% confidence interval for

β

j

has end points

ˆ

β

j

± SE

(

ˆ

β

j

)

t

n−p

(

α

2

).

In particular, if we want to test H

0

: β

j

= 0, we use the fact that under H

0

,

ˆ

β

j

SE(

ˆ

β

j

)

∼ t

n−p

.