3Sturm-Liouville Theory
IB Methods
3.2 Least squares approximation
So far, we have expanded functions in terms of infinite series. However, in real
life, when we ask a computer to do this for us, it is incapable of storing and
calculating an infinite series. So we need to truncate it at some point.
Suppose we approximate some function
f
: Ω
→ C
by a finite set of eigen-
functions y
n
(x). Suppose we write the approximation g as
g(x) =
n
X
k=1
c
k
y
k
(x).
The objective here is to figure out what values of the coefficients
c
k
are “the
best”, i.e. can make g represent f as closely as possible.
Firstly, we have to make it explicit what we mean by “as closely as possible”.
Here we take this to mean that we want to minimize the
w
-norm (
f −g, f −g
)
w
.
By linearity of the norm, we know that
(f − g, f − g)
w
= (f, f)
w
+ (g, g)
w
− (f, g)
w
− (g, f)
w
.
To minimize this norm, we want
0 =
∂
∂c
j
(f − g, f − g)
w
=
∂
∂c
j
[(f, f)
w
+ (g, g)
w
− (f, g)
w
− (g, f)
w
].
We know that the (
f, f
)
w
term vanishes since it does not depend on
c
k
. Expanding
our definitions of g, we can get
0 =
∂
∂c
j
∞
X
i=1
|c
k
|
n
−
n
X
k=1
ˆ
f
∗
k
c
k
−
n
X
k=1
c
∗
k
ˆ
f
k
!
= c
∗
j
−
ˆ
f
∗
j
.
Note that here we are treating
c
∗
j
and
c
j
as distinct quantities. So when we vary
c
j
,
c
∗
j
is unchanged. To formally justify this treatment, we can vary the real and
imaginary parts separately.
Hence, the extremum is achieved at
c
∗
j
=
ˆ
f
∗
j
. Similarly, varying with respect
to c
∗
j
, we get that c
j
=
ˆ
f
j
.
To check that this is indeed an minimum, we can look at the second-derivatives
∂
2
∂c
i
∂c
j
(f − g, f − g)
w
=
∂
2
∂c
∗
i
c
∗
j
(f − g, f − g)
w
= 0,
while
∂
2
∂c
∗
i
∂c
j
(f − g, f − g)
w
= δ
ij
≥ 0.
Hence this is indeed a minimum.
Thus we know that (f − g, f − g)
w
is minimized over all g(x) when
c
k
=
ˆ
f
k
= (y
k
, f)
w
.
These are exactly the coefficients in our infinite expansion. Hence if we truncate
our series at an arbitrary point, it is still the best approximation we can get.