5Eigenvalues and eigenvectors

IA Vectors and Matrices

5.8 Eigenvalues and eigenvectors of a Hermitian matrix
5.8.1 Eigenvalues and eigenvectors
Theorem. The eigenvalues of a Hermitian matrix H are real.
Proof. Suppose that H has eigenvalue λ with eigenvector v 6= 0. Then
Hv = λv.
We pre-multiply by v
, a 1 × n row vector, to obtain
v
Hv = λv
v ()
We take the Hermitian conjugate of both sides. The left hand side is
(v
Hv)
= v
H
v = v
Hv
since H is Hermitian. The right hand side is
(λv
v)
= λ
v
v
So we have
v
Hv = λ
v
v.
From (
), we know that
λv
v
=
λ
v
v
. Since
v 6
= 0, we know that
v
v
=
v · v 6= 0. So λ = λ
and λ is real.
Theorem.
The eigenvectors of a Hermitian matrix
H
corresponding to distinct
eigenvalues are orthogonal.
Proof. Let
Hv
i
= λ
i
v
i
(i)
Hv
j
= λ
j
v
j
. (ii)
Pre-multiply (i) by v
j
to obtain
v
j
Hv
i
= λ
i
v
j
v
i
. (iii)
Pre-multiply (ii) by v
i
and take the Hermitian conjugate to obtain
v
j
Hv
i
= λ
j
v
j
v
i
. (iv)
Equating (iii) and (iv) yields
λ
i
v
j
v
i
= λ
j
v
j
v
i
.
Since
λ
i
6
=
λ
j
, we must have
v
j
v
i
= 0. So their inner product is zero and are
orthogonal.
So we know that if a Hermitian matrix has
n
distinct eigenvalues, then
the eigenvectors form an orthonormal basis. However, if there are degenerate
eigenvalues, it is more difficult, and requires the Gram-Schmidt process.
5.8.2 Gram-Schmidt orthogonalization (non-examinable)
Suppose we have a set
B
=
{w
1
, w
2
, ··· , w
r
}
of linearly independent vectors.
We want to find an orthogonal set
˜
B = {v
1
, v
2
, ··· , v
r
}.
Define the projection of
w
onto
v
by
P
v
(
w
) =
hv|wi
hv|vi
v
. Now construct
˜
B
iteratively:
(i) v
1
= w
1
(ii) v
2
= w
2
P
v
1
(w)
Then we get that hv
1
| v
2
i = hv
1
| w
2
i
hv
1
|w
2
i
hv
1
|v
1
i
hv
1
| v
1
i = 0
(iii) v
3
= w
3
P
v
1
(w
3
) P
v
2
(w
3
)
(iv)
.
.
.
(v) v
r
= w
r
r1
X
j=1
P
v
j
(w
r
)
At each step, we subtract out the components of
v
i
that belong to the space
of
{v
1
, ··· , v
k1
}
. This ensures that all the vectors are orthogonal. Finally, we
normalize each basis vector individually to obtain an orthonormal basis.
5.8.3 Unitary transformation
Suppose
U
is the transformation between one orthonormal basis and a new
orthonormal basis {u
1
, u
2
, ··· , u
n
}, i.e. hu
i
| u
j
i = δ
ij
. Then
U =
(u
1
)
1
(u
2
)
1
··· (u
n
)
1
(u
1
)
2
(u
2
)
2
··· (u
n
)
2
.
.
.
.
.
.
.
.
.
.
.
.
(u
1
)
n
(u
2
)
n
··· (u
n
)
n
Then
(U
U)
ij
= (U
)
ik
U
kj
= U
ki
U
kj
= (u
i
)
k
(u
j
)
k
= hu
i
| u
j
i
= δ
ij
So U is a unitary matrix.
5.8.4 Diagonalization of n × n Hermitian matrices
Theorem.
An
n × n
Hermitian matrix has precisely
n
orthogonal eigenvectors.
Proof.
(Non-examinable) Let
λ
1
, λ
2
, ··· , λ
r
be the distinct eigenvalues of
H
(
r
n
), with a set of corresponding orthonormal eigenvectors
B
=
{v
1
, v
2
, ··· , v
r
}
.
Extend to a basis of the whole of C
n
B
0
= {v
1
, v
2
, ··· , v
r
, w
1
, w
2
, ··· , w
nr
}
Now use Gram-Schmidt to create an orthonormal basis
˜
B = {v
1
, v
2
, ··· , v
r
, u
1
, u
2
, ··· , u
nr
}.
Now write
P =
v
1
v
2
··· v
r
u
1
··· u
nr
We have shown above that this is a unitary matrix, i.e.
P
1
=
P
. So if we
change basis, we have
P
1
HP = P
HP
=
λ
1
0 ··· 0 0 0 ··· 0
0 λ
2
··· 0 0 0 ··· 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0
0 0 ··· λ
r
0 0 ··· 0
0 0 ··· 0 c
11
c
12
··· c
1,nr
0 0 ··· 0 c
21
c
22
··· c
2,nr
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 ··· 0 c
nr,1
c
nr,2
··· c
nr,nr
Here
C
is an (
n r
)
×
(
n r
) Hermitian matrix. The eigenvalues of
C
are also
eigenvalues of
H
because
det
(
H λI
) =
det
(
P
HP λI
) = (
λ
1
λ
)
···
(
λ
r
λ) det(C λI). So the eigenvalues of C are the eigenvalues of H.
We can keep repeating the process on
C
until we finish all rows. For example,
if the eigenvalues of
C
are all distinct, there are
n r
orthonormal eigenvectors
w
j
(for j = r + 1, ··· , n) of C. Let
Q =
1
1
.
.
.
1
w
r+1
w
r+2
··· w
n
with other entries 0. (where we have a
r × r
identity matrix block on the top
left corner and a (n r) ×(n r) with columns formed by w
j
)
Since the columns of
Q
are orthonormal,
Q
is unitary. So
Q
P
HP Q
=
diag
(
λ
1
, λ
2
, ··· , λ
r
, λ
r+1
, ··· , λ
n
), where the first
r λ
s are distinct and the re-
maining ones are copies of previous ones.
The n linearly-independent eigenvectors are the columns of PQ.
So it now follows that
H
is diagonalizable via transformation
U
(=
P Q
).
U
is a unitary matrix because P and Q are. We have
D = U
HU
H = UDU
Note that a real symmetric matrix
S
is a special case of Hermitian matrices. So
we have
D = Q
T
SQ
S = QDQ
T
Example.
Find the orthogonal matrix which diagonalizes the following real
symmetric matrix: S =
1 β
β 1
with β 6= 0 R.
We find the eigenvalues by solving the characteristic equation:
det
(
SλI
) = 0,
and obtain λ = 1 ± β.
The corresponding eigenvectors satisfy (
S λI
)
x
= 0, which gives
x
=
1
2
1
±1
We change the basis from the standard basis to
1
2
1
1
,
1
2
1
1
(which
is just a rotation by π/4).
The transformation matrix is
Q
=
1/
2 1/
2
1/
2 1/
2
. Then we know that
S = QDQ
T
with D = diag(1, 1)
5.8.5 Normal matrices
We have seen that the eigenvalues and eigenvectors of Hermitian matrices satisfy
some nice properties. More generally, we can define the following:
Definition
(Normal matrix)
.
A normal matrix as a matrix that commutes with
its own Hermitian conjugate, i.e.
NN
= N
N
Hermitian, real symmetric, skew-Hermitian, real anti-symmetric, orthogonal,
unitary matrices are all special cases of normal matrices.
It can be shown that:
Proposition.
(i) If λ is an eigenvalue of N, then λ
is an eigenvalue of N
.
(ii) The eigenvectors of distinct eigenvalues are orthogonal.
(iii)
A normal matrix can always be diagonalized with an orthonormal basis of
eigenvectors.