Probstat/notes/basic

This is part of probstat. These notes are meant to be used in complement to the video lectures. They only contain summary of the materials discussed in the video. Don't use them to avoid watching the clips please.

Random experiments

When we would like to talk about probability, we shall start with a random experiment. After we perform this experiment we get an outcome. The set of all possible outcomes is called a sample space, usually denoted by S.

We are generally interested in outcomes with certain properties, usually referred to as an event. Formally, an event is a subset of the sample space S.

Probability axioms

We would like to assign probabilities to events. Let S denote the sample space. Formally a function P is a probability function if it satisfies the follow 3 axioms.

Axiom 1: For any event E, $0\leq P(E)\leq 1$.

Axiom 2: $P(S)=1$.

Axiom 3: For any countable sequence of mutually exclusive events $E_{1},E_{2},\ldots$, we have that

$P(\bigcup _{i}E_{i})=\sum _{i}P(E_{i})$.

Useful properties

There are various properties that can be proved from the axiom. We list a few here.

1: $P(\emptyset )=0$.

2: $P(A^{c})=1-P(A)$.

3: For events A and B, $P(A\cup B)\geq P(A)$.

4: For events A and B, $P(A\cap B)\leq P(A)$.

5: For events A and B, $P(A\cup B)=P(A)+P(B)-P(AB)$.

5a: For events A and B, $P(A\cup B)\leq P(A)+P(B)$. This is usually referred to as the union bound.

Conditional probabilities

Suppose that we know that event B has occurred. We denote by $P(A|B)$ the probability that event A occurs given that event B has occurred. Intuitively under this condition, we know that if A is going to occur, it must occur with B, i.e., event AB must occur.

When $P(B)\neq 0$, we define the probability of event A given that event B has occurred as

$P(A|B)={\frac {P(AB)}{P(B)}}$.

Independence

Events A and B are independent when the knowledge that A has occurs does not change the probability that B occurs.

Formally, we say that A and B are independent if and only if

$P(AB)=P(A)P(B)$

This implies that if $P(A)\neq 0$, $P(B)=P(B|A)$.

If you have more than 2 events, independence becomes slightly more complicated. Consider events $A_{1},A_{2},\ldots ,A_{n}$. We say that these events are mutually independent (or just independent) if and only if, for any subset $I\subseteq \{1,2,\ldots ,n\}$ of indicies, we have that

$P(\bigcap _{i\in I}A_{i})=\prod _{i\in I}P(A_{i})$.

A weaker form is independence is called pair-wise independence. In this case, events $A_{1},A_{2},\ldots ,A_{n}$ are said to be pair-wise independent iff for any different i and j, $A_{i}$ and $A_{j}$ are independent (i.e., $P(A_{i}A_{j})=P(A_{i})P(A_{j})$).

Bayes' formula

For any events A and B, if we know $P(A)$, $P(B|A)$ and $P(B|A^{c})$, we can find the "inverse" probability

$P(A|B)={\frac {P(AB)}{P(B)}}={\frac {P(B|A)P(A)}{P(B|A)P(A)+P(B|A^{c})P(A^{c})}}={\frac {P(B|A)P(A)}{P(B|A)P(A)+P(B|A^{c})(1-P(A))}}$.