III Modern Statistical Methods

0Introduction

0 Introduction

In recent years, there has been a rather significant change in what sorts of data

we have to handle and what questions we ask about them, witnessed by the

popularity of the buzzwords “big data” and “machine learning”. In classical

statistics, we usually have a small set of parameters, and a very large data set.

We then use the large data set to estimate the parameters.

However, nowadays we often see scenarios where we have a very large number

of parameters, and the data set is relatively small. If we tried to apply our

classical linear regression, then we would just be able to tune the parameters so

that we have a perfect fit, and still have great freedom to change the parameters

without affecting the fitting.

One example is that we might want to test which genomes are responsible for

a particular disease. In this case, there is a huge number of genomes to consider,

and there is good reason to believe that most of them are irrelevant, i.e. the

parameters should be set to zero. Thus, we want to develop methods that find

the “best” fitting model that takes this into account.

Another problem we might encounter is that we just have a large data set,

and doing linear regression seems a bit silly. If we have so much data, we might

as well try to fit more complicated curves, such as polynomial functions and

friends. Perhaps more ambitiously, we might try to find the best continuously

differentiable function that fits the curve, or, as analysts will immediately suggest

as an alternative, weakly differentiable functions.

There are many things we can talk about, and we can’t talk about all of

them. In this course, we are going to cover 4 different topics of different size:

– Kernel machines

– The Lasso and its extensions

– Graphical modeling and causal inference

– Multiple testing and high-dimensional inference

The four topics are rather disjoint, and draw on different mathematical skills.