Probstat/notes/limit theorems

This is part of probstat.

In this section, we shall briefly describe one of the reasons why normal distributions appears very often in statistics. This also justifies our assumptions in many applications that the populations we are working on are normally distributed.

Limit theorems, important results in probability theory, consider situations when the size of the sample is very large. There are usually two types of limit results, each with many specific versions under many assumptions. They are referred to as:

• Laws of large numbers --- discuss the fact that as we increase the size of the sample, the sample mean converges to the population mean.
• Central limit theorems -- discuss the fact that the distributions of the sample mean converges to a normal distribution.

Settings: Here we describe the settings of both results. We have random variables $X_{1},X_{2},\ldots ,X_{n}$ which are independent identically distributed with mean $\mu$ and variance $\sigma ^{2}$. (This is true when we take a sample $X_{1},X_{2},\ldots ,X_{n}$ of size $n$ from a population whose mean is $\mu$ and variance $\sigma ^{2}$.)

Law of large numbers

See also the wikipedia article.

The week law of large numbers

We have random variables $X_{1},X_{2},\ldots ,X_{n}$ which are independent identically distributed with mean $\mu$ and variance $\sigma ^{2}$. For any $\epsilon >0$,

$P\left\{|{\frac {X_{1}+X_{2}+\cdots +X_{n}}{n}}-\mu |>\epsilon \right\}\rightarrow 0$

as $n\rightarrow \infty$.

Central limit theorems

See also the wikipedia article.