Probstat/notes/sample means and sample variances

This is part of probstat

Consider a certain distribution. The mean $\mu$ of the distribution is the expected value of a random variable $X$ sample from the distribution. I.e.,

$\mu =E[X]$.

Also recall that the variance of the distribution is

$\sigma ^{2}=Var(X)=E[(X-\mu )^{2}]=E[X^{2}]-E[X]^{2}.$

And finally, the standard deviation is $\sigma ={\sqrt {Var(X)}}$.

Sample Statistics

Suppose that you take $n$ samples independently from this distribution. (Note that $X_{1},X_{2},\ldots ,X_{n}$ are random variables.)

Sample means

The statistic

is called a sample mean. Since $X_{1},X_{2},\dots ,X_{n}$ are random variables, the mean is also a random variable.

We hope that approximates $\mu$ well. We can compute:

and since are independent, we have that

Error

Too many requests (f061ab2)

If you report this error to the Wikimedia System Administrators, please include the details below.

Request served via cp5021 cp5021, Varnish XID 33169929
Upstream caches: cp5021 int
Error: 429, Too many requests (f061ab2) at Sat, 21 Feb 2026 13:00:11 GMT


Sensitive client information

IP address: 158.108.32.49

$Var({\bar {X}})=Var\left({\frac {X_{1}+X_{2}+\cdots +X_{n}}{n}}\right)={\frac {1}{n^{2}}}Var(X_{1}+X_{2}+\cdots +X_{n})={\frac {1}{n^{2}}}\cdot n\cdot Var(X)={\frac {\sigma ^{2}}{n}}.$

Sample variances and sample standard deviations

We can also use the sample to estimate .

The statistic

is called a sample variance. The sample standard deviation is $S={\sqrt {S^{2}}}={\sqrt {\frac {\sum _{i=1}^{n}(X_{i}-{\bar {X}})^{2}}{n-1}}}$.

Note that the denominator is

Error

Too many requests (f061ab2)

If you report this error to the Wikimedia System Administrators, please include the details below.

Request served via cp5021 cp5021, Varnish XID 38178613
Upstream caches: cp5021 int
Error: 429, Too many requests (f061ab2) at Sat, 21 Feb 2026 13:00:02 GMT


Sensitive client information

IP address: 158.108.32.49

$n-1$ instead of $n$.

We can show that .

${\begin{array}{rcl}\mathrm {E} [S^{2}]&=&\mathrm {E} \left[{\frac {\sum _{i=1}^{n}(X_{i}-{\bar {X}})^{2}}{n-1}}\right]\\&=&\mathrm {E} \left[{\frac {\sum _{i=1}^{n}(X_{i}^{2}-2X_{i}{\bar {X}}+{\bar {X}}^{2}}{n-1}}\right]\\&=&{\frac {1}{n-1}}\left(\sum _{i=1}^{n}E[X_{i}^{2}]-2\cdot \sum _{i=1}^{n}E[X_{i}{\bar {X}}]+\sum _{i=1}^{n}E[{\bar {X}}^{2}]\right)\\&=&{\frac {1}{n-1}}\left(\sum _{i=1}^{n}E[X_{i}^{2}]-2\cdot \sum _{i=1}^{n}E\left[X_{i}\left((1/n)\sum _{j=1}^{n}X_{j}\right)\right]+\sum _{i=1}^{n}E\left[\left((1/n)\sum _{j=1}^{n}X_{j}\right)^{2}\right]\right)\\&=&{\frac {1}{n-1}}\left(\sum _{i=1}^{n}E[X_{i}^{2}]-(2/n)\cdot \sum _{i=1}^{n}\sum _{j=1}^{n}E\left[X_{i}\cdot X_{j}\right]+n\cdot E\left[\left((1/n)\sum _{j=1}^{n}X_{j}\right)^{2}\right]\right)\\&=&{\frac {1}{n-1}}\left(\sum _{i=1}^{n}E[X_{i}^{2}]-(2/n)\cdot \sum _{i=1}^{n}\sum _{j=1}^{n}E\left[X_{i}\cdot X_{j}\right]+(1/n)\cdot E\left[\left(\sum _{j=1}^{n}X_{j}\right)^{2}\right]\right)\\\end{array}}$

We note that since and are independent, we have that

$E[X_{i}X_{j}]=E[X_{i}]E[X_{j}]=\mu \cdot \mu =\mu ^{2}$.

Let's deal with the middle term here.

${\begin{array}{rcl}\sum _{i=1}^{n}\sum _{j=1}^{n}E\left[X_{i}\cdot X_{j}\right]&=&\sum _{i=1}^{n}E[X_{i}X_{i}]+\sum _{i=1}^{n}\sum _{j\neq i}E[X_{i}X_{j}]\\&=&\sum _{i=1}^{n}E[X_{i}^{2}]+\sum _{i=1}^{n}\sum _{j\neq i}E[X_{i}]E[X_{j}]\\&=&nE[X^{2}]+n(n-1)E[X]E[X]\\&=&nE[X^{2}]+n(n-1)\mu ^{2}\\\end{array}}$

Let's work on the third term which ends up being the same as the middle term.

Let's put everything together:

Summary

Sample means:

Sample variance:

$S^{2}={\frac {\sum _{i=1}^{n}(X_{i}-{\bar {X}})^{2}}{n-1}}$

Properties of sample means and sample variances

• $E[{\bar {X}}]=\mu$

Distribution of sample means

While we know basic properties of sample means , if we want to perform other statistical calculation (i.e., computing confidence intervals or testing hypotheses), it is very useful to know the exact distribution of .

For a general population, it will be hard to deal the the distribution of ${\bar {X}}$ exactly. However, if the population is normal, we are in a very good shape.

Recall the definition of :

Therefore, is a sum of independent normally distributed random variables. A nice property of normal random variables is that the sum of normally distributed random variables remains a normal random variable. Since a normal random variable is uniquely determined by its mean and variance, we have the following observation.

Distribution of sample means of normal populations.

If the population is normally distributed with mean and variance , the distribution of ${\bar {X}}$ is normal with mean $\mu$ and variance $\sigma ^{2}/n$.

Examples

Ex1. Suppose that the population has mean and variance

Error

Too many requests (f061ab2)

If you report this error to the Wikimedia System Administrators, please include the details below.

Request served via cp5021 cp5021, Varnish XID 39062236
Upstream caches: cp5021 int
Error: 429, Too many requests (f061ab2) at Sat, 21 Feb 2026 13:00:25 GMT


Sensitive client information

IP address: 158.108.32.49

$\sigma ^{2}=15$. If you select a sample of size 20, what is the probability that the sample mean ${\bar {X}}$ is greater than 17?

Solution:

The sample mean is normal with mean

Error

Too many requests (f061ab2)

If you report this error to the Wikimedia System Administrators, please include the details below.

Request served via cp5021 cp5021, Varnish XID 26094629
Upstream caches: cp5021 int
Error: 429, Too many requests (f061ab2) at Sat, 21 Feb 2026 13:00:04 GMT


Sensitive client information

IP address: 158.108.32.49

$\mu =15$ and variance . Therefore,

is unit normal.

Note that

Error

Too many requests (f061ab2)

If you report this error to the Wikimedia System Administrators, please include the details below.

Request served via cp5021 cp5021, Varnish XID 1068464793
Upstream caches: cp5021 int
Error: 429, Too many requests (f061ab2) at Sat, 21 Feb 2026 13:00:04 GMT


Sensitive client information

IP address: 158.108.32.49

$P\{{\bar {X}}>17\}=P\{{\frac {{\bar {X}}-15}{\sqrt {0.75}}}>{\frac {17-15}{\sqrt {0.75}}}\}\approx P\{{\frac {{\bar {X}}-15}{\sqrt {0.75}}}>2.309\}$

We can look at the standard normal table and find out that $P\{0\leq Z\leq 2.31\}=0.48956$, for a unit normal random variable Z. Thus, the probability

Error

Too many requests (f061ab2)

If you report this error to the Wikimedia System Administrators, please include the details below.

Request served via cp5021 cp5021, Varnish XID 36213731
Upstream caches: cp5021 int
Error: 429, Too many requests (f061ab2) at Sat, 21 Feb 2026 13:00:05 GMT


Sensitive client information

IP address: 158.108.32.49

$P\{{\bar {X}}>17\}\approx 0.5-0.48956=0.0104,$

which is roughly 1%.

Ex2.

To be added...

Why do we use normal distributions?

Normal random variables appear very often in our treatment of statistics. This is not just a coincidence. See limit theorems.