Probstat/notes/sample means and sample variances

This is part of probstat

Consider a certain distribution. The mean $\mu$ of the distribution is the expected value of a random variable $X$ sample from the distribution. I.e.,

Error

Too many requests (f061ab2)

If you report this error to the Wikimedia System Administrators, please include the details below.

Request served via cp5021 cp5021, Varnish XID 309609340
Upstream caches: cp5021 int
Error: 429, Too many requests (f061ab2) at Mon, 15 Jun 2026 19:44:23 GMT


Sensitive client information

IP address: 158.108.32.49

$\mu =E[X]$.

Also recall that the variance of the distribution is

$\sigma ^{2}=Var(X)=E[(X-\mu )^{2}]=E[X^{2}]-E[X]^{2}.$

And finally, the standard deviation is

Error

Too many requests (f061ab2)

If you report this error to the Wikimedia System Administrators, please include the details below.

Request served via cp5021 cp5021, Varnish XID 308005002
Upstream caches: cp5021 int
Error: 429, Too many requests (f061ab2) at Mon, 15 Jun 2026 19:44:24 GMT


Sensitive client information

IP address: 158.108.32.49

$\sigma ={\sqrt {Var(X)}}$.

Sample Statistics

Suppose that you take $n$ samples $X_{1},X_{2},\ldots ,X_{n}$ independently from this distribution. (Note that $X_{1},X_{2},\ldots ,X_{n}$ are random variables.)

Sample means

The statistic

Error

Too many requests (f061ab2)

If you report this error to the Wikimedia System Administrators, please include the details below.

Request served via cp5021 cp5021, Varnish XID 331880867
Upstream caches: cp5021 int
Error: 429, Too many requests (f061ab2) at Mon, 15 Jun 2026 19:44:25 GMT


Sensitive client information

IP address: 158.108.32.49

${\bar {X}}={\frac {X_{1}+X_{2}+\cdots +X_{n}}{n}}={\frac {1}{n}}\sum _{i=1}^{n}X_{i}$

is called a sample mean. Since

Error

Too many requests (f061ab2)

If you report this error to the Wikimedia System Administrators, please include the details below.

Request served via cp5021 cp5021, Varnish XID 317401513
Upstream caches: cp5021 int
Error: 429, Too many requests (f061ab2) at Mon, 15 Jun 2026 19:44:25 GMT


Sensitive client information

IP address: 158.108.32.49

$X_{1},X_{2},\dots ,X_{n}$ are random variables, the mean ${\bar {X}}$ is also a random variable.

We hope that ${\bar {X}}$ approximates $\mu$ well. We can compute:

Error

Too many requests (f061ab2)

If you report this error to the Wikimedia System Administrators, please include the details below.

Request served via cp5021 cp5021, Varnish XID 307839387
Upstream caches: cp5021 int
Error: 429, Too many requests (f061ab2) at Mon, 15 Jun 2026 19:44:26 GMT


Sensitive client information

IP address: 158.108.32.49

$E[{\bar {X}}]=E\left[{\frac {1}{n}}\sum _{i=1}^{n}X_{i}\right]={\frac {1}{n}}E\left[\sum _{i=1}^{n}X_{i}\right]={\frac {1}{n}}\sum _{i=1}^{n}E[X_{i}]={\frac {1}{n}}n\mu =\mu$

and since $X_{1},X_{2},\ldots ,X_{n}$ are independent, we have that

Sample variances and sample standard deviations

We can also use the sample to estimate $\sigma ^{2}$.

The statistic

is called a sample variance. The sample standard deviation is .

Note that the denominator is instead of $n$.

We can show that .

We note that since $X_{i}$ and are independent, we have that

.

Let's deal with the middle term here.

Let's work on the third term which ends up being the same as the middle term.

$E\left[\left(\sum _{j=1}^{n}X_{j}\right)^{2}\right]=E\left[\sum _{j=1}^{n}\sum _{k=1}^{n}X_{j}X_{k}\right]=\sum _{j=1}^{n}\sum _{k=1}^{n}E[X_{j}X_{k}]=nE[X^{2}]+n(n-1)\mu ^{2}.$

Let's put everything together:

${\begin{array}{rcl}\mathrm {E} [S^{2}]&=&{\frac {1}{n-1}}\left(\sum _{i=1}^{n}E[X_{i}^{2}]-(2/n)\cdot \sum _{i=1}^{n}\sum _{j=1}^{n}E\left[X_{i}\cdot X_{j}\right]+(1/n)\cdot E\left[\left(\sum _{j=1}^{n}X_{j}\right)^{2}\right]\right)\\&=&{\frac {1}{n-1}}\left(nE[X^{2}]-(2/n)(nE[X^{2}]+n(n-1)\mu ^{2})+(1/n)(nE[X^{2}]+n(n-1)\mu ^{2})\right)\\&=&{\frac {1}{n-1}}\left(nE[X^{2}]-2E[X^{2}]-2(n-1)\mu ^{2}+E[X^{2}]+(n-1)\mu ^{2}\right)\\&=&{\frac {1}{n-1}}\left((n-1)E[X^{2}]-(n-1)\mu ^{2}\right)\\&=&E[X^{2}]-\mu ^{2}=\sigma ^{2}\\\end{array}}$

Summary

Sample means:

${\bar {X}}={\frac {X_{1}+X_{2}+\cdots +X_{n}}{n}}={\frac {1}{n}}\sum _{i=1}^{n}X_{i}$

Sample variance:

$S^{2}={\frac {\sum _{i=1}^{n}(X_{i}-{\bar {X}})^{2}}{n-1}}$

Properties of sample means and sample variances

• 

  Error

  Too many requests (f061ab2)

  If you report this error to the Wikimedia System Administrators, please include the details below.

  Request served via cp5021 cp5021, Varnish XID 336627530
Upstream caches: cp5021 int
Error: 429, Too many requests (f061ab2) at Mon, 15 Jun 2026 19:44:27 GMT


  Sensitive client information

  IP address: 158.108.32.49

  $E[{\bar {X}}]=\mu$
• $Var[{\bar {X}}]=\sigma ^{2}/n$
• $E[S^{2}]=\sigma ^{2}$

Distribution of sample means

While we know basic properties of sample means ${\bar {X}}$, if we want to perform other statistical calculation (i.e., computing confidence intervals or testing hypotheses), it is very useful to know the exact distribution of ${\bar {X}}$.

For a general population, it will be hard to deal the the distribution of ${\bar {X}}$ exactly. However, if the population is normal, we are in a very good shape.

Recall the definition of ${\bar {X}}$:

${\bar {X}}=(X_{1}+X_{2}+\cdots +X_{n})/n.$

Therefore, ${\bar {X}}$ is a sum of independent normally distributed random variables. A nice property of normal random variables is that the sum of normally distributed random variables remains a normal random variable. Since a normal random variable is uniquely determined by its mean and variance, we have the following observation.

Distribution of sample means of normal populations.

If the population is normally distributed with mean $\mu$ and variance $\sigma ^{2}$, the distribution of ${\bar {X}}$ is normal with mean $\mu$ and variance $\sigma ^{2}/n$.

Examples

Ex1. Suppose that the population has mean $\mu =15$ and variance $\sigma ^{2}=15$. If you select a sample of size 20, what is the probability that the sample mean ${\bar {X}}$ is greater than 17?

Solution:

The sample mean ${\bar {X}}$ is normal with mean $\mu =15$ and variance $\sigma ^{2}/n=15/20=0.75$. Therefore,

${\frac {{\bar {X}}-15}{\sqrt {0.75}}}$

is unit normal.

Note that

$P\{{\bar {X}}>17\}=P\{{\frac {{\bar {X}}-15}{\sqrt {0.75}}}>{\frac {17-15}{\sqrt {0.75}}}\}\approx P\{{\frac {{\bar {X}}-15}{\sqrt {0.75}}}>2.309\}$

We can look at the standard normal table and find out that $P\{0\leq Z\leq 2.31\}=0.48956$, for a unit normal random variable Z. Thus, the probability

$P\{{\bar {X}}>17\}\approx 0.5-0.48956=0.0104,$

which is roughly 1%.

Ex2.

To be added...

Why do we use normal distributions?

Normal random variables appear very often in our treatment of statistics. This is not just a coincidence. See limit theorems.