Probstat/notes/hypothesis testing

This is part of probstat.

Since it is very hard to obtain complete information of the population, we usually end up with a collection of much smaller sample data. A question arises: how can we be confident if the conclusion we make from the collected sample is correct or it is only by chance?

This section tries to answer this question.

A motivating example

You friend gives you a coin and claims that this coin is special. (It is unclear what special is about this coin.)

You want to prove it so you toss the coin for 20 times.

If you get 10 heads, do you believe your friend that the coin is special?

If you get 12 heads, do you believe your friend that the coin is special?

How about 15 heads? How about 18 heads? How about 20 heads?

Let's consider each case.

10: What is the probability that a normal coin turns up at least 10 heads from 20 coin tosses? 58% So that this does not show anything special about this coin.

12: What is the probability that a normal coin turns up at least 12 heads from 20 coin tosses? 25% So this coin might be a bit special?

15: What is the probability that a normal coin turns up at least 15 heads from 20 coin tosses? 2% This coin is special or I am very lucky.

18: What is the probability that a normal coin turns up at least 18 heads from 20 coin tosses? 0.02% This coin is special of I am extremely lucky.

20: What is the probability that a normal coin turns up at least 20 heads from 20 coin tosses? 1 in a million. I definitely should believe that this coin is special.