5Eigenvalues and eigenvectors

IA Vectors and Matrices

5.8 Eigenvalues and eigenvectors of a Hermitian matrix

5.8.1 Eigenvalues and eigenvectors

Theorem. The eigenvalues of a Hermitian matrix H are real.

Proof. Suppose that H has eigenvalue λ with eigenvector v 6= 0. Then

Hv = λv.

We pre-multiply by v

†

, a 1 × n row vector, to obtain

v

†

Hv = λv

†

v (∗)

We take the Hermitian conjugate of both sides. The left hand side is

(v

†

Hv)

†

= v

†

H

†

v = v

†

Hv

since H is Hermitian. The right hand side is

(λv

†

v)

†

= λ

∗

v

†

v

So we have

v

†

Hv = λ

∗

v

†

v.

From (

∗

), we know that

λv

†

v

=

λ

∗

v

†

v

. Since

v 6

= 0, we know that

v

†

v

=

v · v 6= 0. So λ = λ

∗

and λ is real.

Theorem.

The eigenvectors of a Hermitian matrix

H

corresponding to distinct

eigenvalues are orthogonal.

Proof. Let

Hv

i

= λ

i

v

i

(i)

Hv

j

= λ

j

v

j

. (ii)

Pre-multiply (i) by v

†

j

to obtain

v

†

j

Hv

i

= λ

i

v

†

j

v

i

. (iii)

Pre-multiply (ii) by v

†

i

and take the Hermitian conjugate to obtain

v

†

j

Hv

i

= λ

j

v

†

j

v

i

. (iv)

Equating (iii) and (iv) yields

λ

i

v

†

j

v

i

= λ

j

v

†

j

v

i

.

Since

λ

i

6

=

λ

j

, we must have

v

†

j

v

i

= 0. So their inner product is zero and are

orthogonal.

So we know that if a Hermitian matrix has

n

distinct eigenvalues, then

the eigenvectors form an orthonormal basis. However, if there are degenerate

eigenvalues, it is more difficult, and requires the Gram-Schmidt process.

5.8.2 Gram-Schmidt orthogonalization (non-examinable)

Suppose we have a set

B

=

{w

1

, w

2

, ··· , w

r

}

of linearly independent vectors.

We want to find an orthogonal set

˜

B = {v

1

, v

2

, ··· , v

r

}.

Define the projection of

w

onto

v

by

P

v

(

w

) =

hv|wi

hv|vi

v

. Now construct

˜

B

iteratively:

(i) v

1

= w

1

(ii) v

2

= w

2

− P

v

1

(w)

Then we get that hv

1

| v

2

i = hv

1

| w

2

i −

hv

1

|w

2

i

hv

1

|v

1

i

hv

1

| v

1

i = 0

(iii) v

3

= w

3

− P

v

1

(w

3

) − P

v

2

(w

3

)

(iv)

.

.

.

(v) v

r

= w

r

−

r−1

X

j=1

P

v

j

(w

r

)

At each step, we subtract out the components of

v

i

that belong to the space

of

{v

1

, ··· , v

k−1

}

. This ensures that all the vectors are orthogonal. Finally, we

normalize each basis vector individually to obtain an orthonormal basis.

5.8.3 Unitary transformation

Suppose

U

is the transformation between one orthonormal basis and a new

orthonormal basis {u

1

, u

2

, ··· , u

n

}, i.e. hu

i

| u

j

i = δ

ij

. Then

U =

(u

1

)

1

(u

2

)

1

··· (u

n

)

1

(u

1

)

2

(u

2

)

2

··· (u

n

)

2

.

.

.

.

.

.

.

.

.

.

.

.

(u

1

)

n

(u

2

)

n

··· (u

n

)

n

Then

(U

†

U)

ij

= (U

†

)

ik

U

kj

= U

∗

ki

U

kj

= (u

i

)

∗

k

(u

j

)

k

= hu

i

| u

j

i

= δ

ij

So U is a unitary matrix.

5.8.4 Diagonalization of n × n Hermitian matrices

Theorem.

An

n × n

Hermitian matrix has precisely

n

orthogonal eigenvectors.

Proof.

(Non-examinable) Let

λ

1

, λ

2

, ··· , λ

r

be the distinct eigenvalues of

H

(

r ≤

n

), with a set of corresponding orthonormal eigenvectors

B

=

{v

1

, v

2

, ··· , v

r

}

.

Extend to a basis of the whole of C

n

B

0

= {v

1

, v

2

, ··· , v

r

, w

1

, w

2

, ··· , w

n−r

}

Now use Gram-Schmidt to create an orthonormal basis

˜

B = {v

1

, v

2

, ··· , v

r

, u

1

, u

2

, ··· , u

n−r

}.

Now write

P =

↑ ↑ ↑ ↑ ↑

v

1

v

2

··· v

r

u

1

··· u

n−r

↓ ↓ ↓ ↓ ↓

We have shown above that this is a unitary matrix, i.e.

P

−1

=

P

†

. So if we

change basis, we have

P

−1

HP = P

†

HP

=

λ

1

0 ··· 0 0 0 ··· 0

0 λ

2

··· 0 0 0 ··· 0

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

0

0 0 ··· λ

r

0 0 ··· 0

0 0 ··· 0 c

11

c

12

··· c

1,n−r

0 0 ··· 0 c

21

c

22

··· c

2,n−r

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

0 0 ··· 0 c

n−r,1

c

n−r,2

··· c

n−r,n−r

Here

C

is an (

n − r

)

×

(

n − r

) Hermitian matrix. The eigenvalues of

C

are also

eigenvalues of

H

because

det

(

H − λI

) =

det

(

P

†

HP − λI

) = (

λ

1

− λ

)

···

(

λ

r

−

λ) det(C − λI). So the eigenvalues of C are the eigenvalues of H.

We can keep repeating the process on

C

until we finish all rows. For example,

if the eigenvalues of

C

are all distinct, there are

n − r

orthonormal eigenvectors

w

j

(for j = r + 1, ··· , n) of C. Let

Q =

1

1

.

.

.

1

↑ ↑ ↑

w

r+1

w

r+2

··· w

n

↓ ↓ ↓

with other entries 0. (where we have a

r × r

identity matrix block on the top

left corner and a (n − r) ×(n −r) with columns formed by w

j

)

Since the columns of

Q

are orthonormal,

Q

is unitary. So

Q

†

P

†

HP Q

=

diag

(

λ

1

, λ

2

, ··· , λ

r

, λ

r+1

, ··· , λ

n

), where the first

r λ

s are distinct and the re-

maining ones are copies of previous ones.

The n linearly-independent eigenvectors are the columns of PQ.

So it now follows that

H

is diagonalizable via transformation

U

(=

P Q

).

U

is a unitary matrix because P and Q are. We have

D = U

†

HU

H = UDU

†

Note that a real symmetric matrix

S

is a special case of Hermitian matrices. So

we have

D = Q

T

SQ

S = QDQ

T

Example.

Find the orthogonal matrix which diagonalizes the following real

symmetric matrix: S =

1 β

β 1

with β 6= 0 ∈ R.

We find the eigenvalues by solving the characteristic equation:

det

(

S−λI

) = 0,

and obtain λ = 1 ± β.

The corresponding eigenvectors satisfy (

S − λI

)

x

= 0, which gives

x

=

1

√

2

1

±1

We change the basis from the standard basis to

1

√

2

1

1

,

1

√

2

1

−1

(which

is just a rotation by π/4).

The transformation matrix is

Q

=

1/

√

2 1/

√

2

1/

√

2 −1/

√

2

. Then we know that

S = QDQ

T

with D = diag(1, −1)

5.8.5 Normal matrices

We have seen that the eigenvalues and eigenvectors of Hermitian matrices satisfy

some nice properties. More generally, we can define the following:

Definition

(Normal matrix)

.

A normal matrix as a matrix that commutes with

its own Hermitian conjugate, i.e.

NN

†

= N

†

N

Hermitian, real symmetric, skew-Hermitian, real anti-symmetric, orthogonal,

unitary matrices are all special cases of normal matrices.

It can be shown that:

Proposition.

(i) If λ is an eigenvalue of N, then λ

∗

is an eigenvalue of N

†

.

(ii) The eigenvectors of distinct eigenvalues are orthogonal.

(iii)

A normal matrix can always be diagonalized with an orthonormal basis of

eigenvectors.