Probstat/notes/t-distributions

This is part of probstat.

In many applications including computing confidence intervals and also hypothesis testing, we need to know the distribution of the sample mean. Let ${\bar {X}}$ be a sample mean of a normal population computed from a sample of size $n$. If the variance $\sigma ^{2}$ of the population is known, we have that

${\frac {{\bar {X}}-\mu }{\sigma /{\sqrt {n}}}}\sim Normal(0,1).$

However, in most cases, we do not know $\sigma ^{2}$ and we only have the sample variance

$S^{2}={\frac {\sum _{i=1}^{n}(X_{i}-{\bar {X}})^{2}}{n-1}}$.

Therefore, we would like to use the sample standard deviation $S$ instead of the real standard deviation $\sigma$. We hope that

${\frac {{\bar {X}}-\mu }{S/{\sqrt {n}}}}$

will be close to normal. This is indeed true. Although ${\sqrt {n}}({\bar {X}}-\mu )/S$ is not normal, it is t-distributed with $n-1$ degrees of freedom. (See definition below.) Therefore, we can use the table for the t-distribution to compute probabilities for various events related to this statistic. Note that it is slightly harder to use than the standard normal table, because there are two parameters, i.e., the degrees of freedom and the level of confidence that you want.

We will show example usages for the t-distributions and then discuss the mechanics behind this usage of the t-distribution.

EX1: We take a sample of size 10 and calculate the sample mean ${\bar {X}}=100$ and the sample variance $S^{2}=5$. Find the confidence interval with 80% confidence level.

Solution: Since the sample size is 10, the degrees of freedom to use is 9. We look at the t-table and find out that

$P\{-1.383<T<1.383\}=0.8$,

where $T$ is a t-distributed random variable with 9 degrees of freedom.

Since $T={\sqrt {n}}({\bar {X}}-\mu )/S$, we have that

${\begin{array}{rcl}P\{-1.383<{\sqrt {n}}({\bar {X}}-\mu )/S<1.383\}&=&P\{-1.383/{\sqrt {n}}<({\bar {X}}-\mu )/S<1.383/{\sqrt {n}}\}\\&=&P\{-1.383S/{\sqrt {n}}<{\bar {X}}-\mu <1.383S/{\sqrt {n}}\}\\&=&0.8.\end{array}}$

If we put in the actual values of ${\bar {X}}$ and $S$ we have that the confidence interval with 80% confidence level is

$(100-1.383{\sqrt {5}}/{\sqrt {10}},100+1.383{\sqrt {5}}/{\sqrt {10}})=(99.022,100.978).$

Comparison with the case where $\sigma ^{2}$ is known. As a comparison, suppose that the real variance $\sigma ^{2}$ is actually 5. Using the standard normal table, our confidence interval will be smaller. That is, we have that

$P\{-1.28<{\sqrt {n}}({\bar {X}}-\mu )/\sigma <1.28\}=P\{-1.28\sigma /{\sqrt {n}}<{\bar {X}}-\mu <1.28\sigma /{\sqrt {n}}\}=0.8.$

Plugging in the values of ${\bar {X}}$ and $\sigma ^{2}$ we have that the confidence interval with 80% confidence level is

$(100-1.28{\sqrt {5}}/{\sqrt {10}},100+1.28{\sqrt {5}}/{\sqrt {10}})=(99.094,100.905),$

which is slightly smaller the previous interval.

Student's t-Distribution

Note that our goal

${\frac {{\bar {X}}-\mu }{S/{\sqrt {n}}}}={\frac {\left({\frac {{\bar {X}}-\mu }{\sigma /{\sqrt {n}}}}\right)}{S/\sigma }}.$

Let's recall two important facts about sample means and sample variances. First, $(n-1)S^{2}/\sigma ^{2}\sim \chi _{n-1}^{2}$ and further more, ${\bar {X}}$ and $S^{2}$ are independent. These imply that the above quantity is a quotient of a unit normal random variable and a square root of a chi-squared random variable. This is exactly what a t-distributed random variable is.

Definition: If $Z$ is a unit normal random variable and $X$ is a chi-squared with $n$ degrees of freedom, such that $X$ and $Z$ are independent, we call a random variable

$T_{n}={\frac {Z}{\sqrt {X/n}}}$

a t-distributed random variable with $n$ degrees of freedom.

To see that $({\bar {X}}-\mu )/(S/{\sqrt {n}})$ is t-distributed with $n-1$ degrees of freedom, we calculate

${\frac {{\bar {X}}-\mu }{S/{\sqrt {n}}}}={\frac {\left({\frac {{\bar {X}}-\mu }{\sigma /{\sqrt {n}}}}\right)}{S/\sigma }}={\frac {\left({\frac {{\bar {X}}-\mu }{\sigma /{\sqrt {n}}}}\right)}{\sqrt {\frac {S^{2}}{\sigma ^{2}}}}}={\frac {\left({\frac {{\bar {X}}-\mu }{\sigma /{\sqrt {n}}}}\right)}{\sqrt {{\frac {(n-1)S^{2}}{\sigma ^{2}}}/(n-1)}}},$

and note that the form of the quantity matches the definition of the t-distributed random variable where $Z={\frac {{\bar {X}}-\mu }{\sigma /{\sqrt {n}}}}$ and $X={\frac {(n-1)S^{2}}{\sigma ^{2}}}$.

The t-distribution is a flatten-out version of the normal distribution (see pdf below). It gets closer and closer to the normal distribution as the degrees of freedom increase. See wikipedia's article for more graphs and information.

(Image from [1])

Links

• wikipedia article