5Eigenvalues and eigenvectors
IA Vectors and Matrices
5.8 Eigenvalues and eigenvectors of a Hermitian matrix
5.8.1 Eigenvalues and eigenvectors
Theorem. The eigenvalues of a Hermitian matrix H are real.
Proof. Suppose that H has eigenvalue λ with eigenvector v 6= 0. Then
Hv = λv.
We pre-multiply by v
†
, a 1 × n row vector, to obtain
v
†
Hv = λv
†
v (∗)
We take the Hermitian conjugate of both sides. The left hand side is
(v
†
Hv)
†
= v
†
H
†
v = v
†
Hv
since H is Hermitian. The right hand side is
(λv
†
v)
†
= λ
∗
v
†
v
So we have
v
†
Hv = λ
∗
v
†
v.
From (
∗
), we know that
λv
†
v
=
λ
∗
v
†
v
. Since
v 6
= 0, we know that
v
†
v
=
v · v 6= 0. So λ = λ
∗
and λ is real.
Theorem.
The eigenvectors of a Hermitian matrix
H
corresponding to distinct
eigenvalues are orthogonal.
Proof. Let
Hv
i
= λ
i
v
i
(i)
Hv
j
= λ
j
v
j
. (ii)
Pre-multiply (i) by v
†
j
to obtain
v
†
j
Hv
i
= λ
i
v
†
j
v
i
. (iii)
Pre-multiply (ii) by v
†
i
and take the Hermitian conjugate to obtain
v
†
j
Hv
i
= λ
j
v
†
j
v
i
. (iv)
Equating (iii) and (iv) yields
λ
i
v
†
j
v
i
= λ
j
v
†
j
v
i
.
Since
λ
i
6
=
λ
j
, we must have
v
†
j
v
i
= 0. So their inner product is zero and are
orthogonal.
So we know that if a Hermitian matrix has
n
distinct eigenvalues, then
the eigenvectors form an orthonormal basis. However, if there are degenerate
eigenvalues, it is more difficult, and requires the Gram-Schmidt process.
5.8.2 Gram-Schmidt orthogonalization (non-examinable)
Suppose we have a set
B
=
{w
1
, w
2
, ··· , w
r
}
of linearly independent vectors.
We want to find an orthogonal set
˜
B = {v
1
, v
2
, ··· , v
r
}.
Define the projection of
w
onto
v
by
P
v
(
w
) =
hv|wi
hv|vi
v
. Now construct
˜
B
iteratively:
(i) v
1
= w
1
(ii) v
2
= w
2
− P
v
1
(w)
Then we get that hv
1
| v
2
i = hv
1
| w
2
i −
hv
1
|w
2
i
hv
1
|v
1
i
hv
1
| v
1
i = 0
(iii) v
3
= w
3
− P
v
1
(w
3
) − P
v
2
(w
3
)
(iv)
.
.
.
(v) v
r
= w
r
−
r−1
X
j=1
P
v
j
(w
r
)
At each step, we subtract out the components of
v
i
that belong to the space
of
{v
1
, ··· , v
k−1
}
. This ensures that all the vectors are orthogonal. Finally, we
normalize each basis vector individually to obtain an orthonormal basis.
5.8.3 Unitary transformation
Suppose
U
is the transformation between one orthonormal basis and a new
orthonormal basis {u
1
, u
2
, ··· , u
n
}, i.e. hu
i
| u
j
i = δ
ij
. Then
U =
(u
1
)
1
(u
2
)
1
··· (u
n
)
1
(u
1
)
2
(u
2
)
2
··· (u
n
)
2
.
.
.
.
.
.
.
.
.
.
.
.
(u
1
)
n
(u
2
)
n
··· (u
n
)
n
Then
(U
†
U)
ij
= (U
†
)
ik
U
kj
= U
∗
ki
U
kj
= (u
i
)
∗
k
(u
j
)
k
= hu
i
| u
j
i
= δ
ij
So U is a unitary matrix.
5.8.4 Diagonalization of n × n Hermitian matrices
Theorem.
An
n × n
Hermitian matrix has precisely
n
orthogonal eigenvectors.
Proof.
(Non-examinable) Let
λ
1
, λ
2
, ··· , λ
r
be the distinct eigenvalues of
H
(
r ≤
n
), with a set of corresponding orthonormal eigenvectors
B
=
{v
1
, v
2
, ··· , v
r
}
.
Extend to a basis of the whole of C
n
B
0
= {v
1
, v
2
, ··· , v
r
, w
1
, w
2
, ··· , w
n−r
}
Now use Gram-Schmidt to create an orthonormal basis
˜
B = {v
1
, v
2
, ··· , v
r
, u
1
, u
2
, ··· , u
n−r
}.
Now write
P =
↑ ↑ ↑ ↑ ↑
v
1
v
2
··· v
r
u
1
··· u
n−r
↓ ↓ ↓ ↓ ↓
We have shown above that this is a unitary matrix, i.e.
P
−1
=
P
†
. So if we
change basis, we have
P
−1
HP = P
†
HP
=
λ
1
0 ··· 0 0 0 ··· 0
0 λ
2
··· 0 0 0 ··· 0
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0
0 0 ··· λ
r
0 0 ··· 0
0 0 ··· 0 c
11
c
12
··· c
1,n−r
0 0 ··· 0 c
21
c
22
··· c
2,n−r
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0 0 ··· 0 c
n−r,1
c
n−r,2
··· c
n−r,n−r
Here
C
is an (
n − r
)
×
(
n − r
) Hermitian matrix. The eigenvalues of
C
are also
eigenvalues of
H
because
det
(
H − λI
) =
det
(
P
†
HP − λI
) = (
λ
1
− λ
)
···
(
λ
r
−
λ) det(C − λI). So the eigenvalues of C are the eigenvalues of H.
We can keep repeating the process on
C
until we finish all rows. For example,
if the eigenvalues of
C
are all distinct, there are
n − r
orthonormal eigenvectors
w
j
(for j = r + 1, ··· , n) of C. Let
Q =
1
1
.
.
.
1
↑ ↑ ↑
w
r+1
w
r+2
··· w
n
↓ ↓ ↓
with other entries 0. (where we have a
r × r
identity matrix block on the top
left corner and a (n − r) ×(n −r) with columns formed by w
j
)
Since the columns of
Q
are orthonormal,
Q
is unitary. So
Q
†
P
†
HP Q
=
diag
(
λ
1
, λ
2
, ··· , λ
r
, λ
r+1
, ··· , λ
n
), where the first
r λ
s are distinct and the re-
maining ones are copies of previous ones.
The n linearly-independent eigenvectors are the columns of PQ.
So it now follows that
H
is diagonalizable via transformation
U
(=
P Q
).
U
is a unitary matrix because P and Q are. We have
D = U
†
HU
H = UDU
†
Note that a real symmetric matrix
S
is a special case of Hermitian matrices. So
we have
D = Q
T
SQ
S = QDQ
T
Example.
Find the orthogonal matrix which diagonalizes the following real
symmetric matrix: S =
1 β
β 1
with β 6= 0 ∈ R.
We find the eigenvalues by solving the characteristic equation:
det
(
S−λI
) = 0,
and obtain λ = 1 ± β.
The corresponding eigenvectors satisfy (
S − λI
)
x
= 0, which gives
x
=
1
√
2
1
±1
We change the basis from the standard basis to
1
√
2
1
1
,
1
√
2
1
−1
(which
is just a rotation by π/4).
The transformation matrix is
Q
=
1/
√
2 1/
√
2
1/
√
2 −1/
√
2
. Then we know that
S = QDQ
T
with D = diag(1, −1)
5.8.5 Normal matrices
We have seen that the eigenvalues and eigenvectors of Hermitian matrices satisfy
some nice properties. More generally, we can define the following:
Definition
(Normal matrix)
.
A normal matrix as a matrix that commutes with
its own Hermitian conjugate, i.e.
NN
†
= N
†
N
Hermitian, real symmetric, skew-Hermitian, real anti-symmetric, orthogonal,
unitary matrices are all special cases of normal matrices.
It can be shown that:
Proposition.
(i) If λ is an eigenvalue of N, then λ
∗
is an eigenvalue of N
†
.
(ii) The eigenvectors of distinct eigenvalues are orthogonal.
(iii)
A normal matrix can always be diagonalized with an orthonormal basis of
eigenvectors.