3Sturm-Liouville Theory

IB Methods 3.2 Least squares approximation
So far, we have expanded functions in terms of infinite series. However, in real
life, when we ask a computer to do this for us, it is incapable of storing and
calculating an infinite series. So we need to truncate it at some point.
Suppose we approximate some function
f
: Ω
C
by a finite set of eigen-
functions y
n
(x). Suppose we write the approximation g as
g(x) =
n
X
k=1
c
k
y
k
(x).
The objective here is to figure out what values of the coefficients
c
k
are “the
best”, i.e. can make g represent f as closely as possible.
Firstly, we have to make it explicit what we mean by “as closely as possible”.
Here we take this to mean that we want to minimize the
w
-norm (
f g, f g
)
w
.
By linearity of the norm, we know that
(f g, f g)
w
= (f, f)
w
+ (g, g)
w
(f, g)
w
(g, f)
w
.
To minimize this norm, we want
0 =
c
j
(f g, f g)
w
=
c
j
[(f, f)
w
+ (g, g)
w
(f, g)
w
(g, f)
w
].
We know that the (
f, f
)
w
term vanishes since it does not depend on
c
k
. Expanding
our definitions of g, we can get
0 =
c
j
X
i=1
|c
k
|
n
n
X
k=1
ˆ
f
k
c
k
n
X
k=1
c
k
ˆ
f
k
!
= c
j
ˆ
f
j
.
Note that here we are treating
c
j
and
c
j
as distinct quantities. So when we vary
c
j
,
c
j
is unchanged. To formally justify this treatment, we can vary the real and
imaginary parts separately.
Hence, the extremum is achieved at
c
j
=
ˆ
f
j
. Similarly, varying with respect
to c
j
, we get that c
j
=
ˆ
f
j
.
To check that this is indeed an minimum, we can look at the second-derivatives
2
c
i
c
j
(f g, f g)
w
=
2
c
i
c
j
(f g, f g)
w
= 0,
while
2
c
i
c
j
(f g, f g)
w
= δ
ij
0.
Hence this is indeed a minimum.
Thus we know that (f g, f g)
w
is minimized over all g(x) when
c
k
=
ˆ
f
k
= (y
k
, f)
w
.
These are exactly the coefficients in our infinite expansion. Hence if we truncate
our series at an arbitrary point, it is still the best approximation we can get.