Probability Revision

Sample space

For any probabilistic experiment or process, the set $\Omega$ of all its possible outcomes is called its sample space.

In general, sample spaces need not be finite, and they need not even be countable. In this course, we focus on finite and countable sample spaces. This simplifies the axiomatic treatment needed to do probability theory.

In any repeated situation, such as flipping a coin $n$ times, the sample space is given by:

$\Omega = \\{H,T\\}^n$

So the size of the sample space is the number of options in each trial to the power of the number of trials.

Probability distributions

A probability distribution over a finite or countable set $\Omega$, is a function:

$P: \Omega \to [0,1]$

Such that $\textstyle\sum_{s \in \Omega} P(s) = 1$.

It is the set of probabilities of each of the outcomes in the sample space.

Events

For a countable sample space $\Omega$, an event, E, is simply a subset $E \subseteq \Omega$ of the set of possible outcomes. Given a probability distribution $P: \Omega \to [0,1]$, we define the probability of the event as $P(E) = \textstyle\sum_{s \in E} P(s)$.

Conditional probability

Let $P: \Omega \to [0,1]$ be a probability distribution, and let $E,F \subseteq \Omega$ be two events, such that $P(F) \gt 0$. The conditional probability of $E$ given $F$, denoted $P(E|F)$, is defined by:

$P(E|F) = \dfrac{P(E \cap F)}{P(F)}$

Independence

Events $A$ and $B$ are called independent if $P(A \cap B) = P(A)P(B)$.

This means that if $A$ and $B$ are independent and $P(B) \gt 0$ then:

$P(A|B) = \dfrac{P(A \cap B)}{P(B)} = P(A)$

Bernoulli trials

A Bernoulli trial is a probabilistic experiment that has two possible outcomes: success or failure.

Binomial distribution

Take PwA distributions sheet into the exam with you.

Random variables

A random variable is a function $X: \Omega \to R$, that assigns a real value to each outcome in a sample space $\Omega$.

We can define a probability distribution for each possible value of a random variable. This is often denoted as:

$P(X = k)$

The ‘range’ of a random variable is the set of all the possible values it can have.

Bayes’ theorem

Extending everything we have covered in this topic, we have seen different ways to approach conditions in probability. This culminates in a formula that is often applied to these situations depending on the information you are given.

$\Bbb{P}(E|F) = \dfrac{\Bbb{P}(F|E)\Bbb{P}(E)}{\Bbb{P}(F)}$

Expected Values

The expected value, orexpectation, of a random variable is defined by:

$E(X) = \textstyle\sum_{s \in \Omega} P(s) X(s)$

Where $P$ is the underlying probability distribution and $X$ is the value assigned to the random variable.

For example, let $X$ be the output value of rolling a six-sided die. Then the expected value of $X$ is given by:

$E(X) = \displaystyle\sum_{i=1}^6 \dfrac{1}{6} i = \dfrac{21}{6} = \dfrac{7}{2}$

However, this method is not always feasible. There are examples where the sum would have thousands of terms and remember that you don’t get a calculator in this exam. So to fix this we can also define:

$E(X) = \textstyle\sum_{r \in \text{range}(X)} P(X = r) r$

The expected # of successes in $n$ (independent) Bernoulli trials, with probability $p$ of success in each, is $np$. This is basically a binomial distribution.

The expected # of trails needed to obtain success, with probability $p$ of success in each, is $\frac{1}{p}$. This is basically a geometric distribution.

Linearity of expectation

Apparently, this is very important.

For any random variables $X,X_1,...,X_n$ on $\Omega$:

$E(X_1 + ... + X_n) = E(X_1) + ... + E(X_n)$

Furthermore, for any $a,b \in \Reals$:

$E(aX + b) = aE(X) + b$

Independent RVs

Two random variables, $X$ and $Y$, are called independent if for all $r_1,r_2 \in \Reals$:

$P(X = r_1 \text{ and } Y = r_2) = P(X = r_1) P(Y = r_2)$

If $X$ and $Y$ are independent random variables on the same space $\Omega$. Then:

$E(XY) = E(X)E(Y)$$

Variance