### Probability and Random Variables Define $\Omega$ as the set of all possible outcomes. For example, fair coin thrown twice where $h=$ heads and $t=$ tails $\Omega = \\{ hh, ht, th, tt \\}$ Definition: Discrete Random Variable $X \equiv$ A random variable that can take only a finite number of values. Definition: $P(X=n) = \frac{\text{number of ways} X=n}{\text{total number of outcomes}}$ --- ### Probability and Random Variables Property: Sum of all probabilities over all values that $X$ can take equals 1: If $X$ takes the values $x_{1}, x_{2},\dots,x_{N}$, then $\sum_{i=1}^{N} P(X = x_{i}) = 1$ Two discrete random variables $X$ and $Y$ taking on possible values $x_{1}, x_{2},\dots$, and $y_{1}, y_{2},\dots$ are *independent* if for all $i$ and $j$ $P(X=x_{i}, Y=y_{j}) = P(X=x_{i})P(Y=y_{j})$ --- ### Bernoulli Random Variable A Bernoulli random variable $X$ takes on only two values: 0 and 1, with probabilities $1-p$ and $p$ respectively. The pobability distribution for $X$ is defined as follows: $P(X = 1) = p(1) = p$ $P(X = 0) = p(0) = 1 - p$ $P(X = x) = p(x) = 0$ if $x \ne 0$ or $x \ne 1$ --- ### Binomial Distribution Assume that $n$ independent experiments or trials are performed, where $n$ is a fixed number and each experiment results in a "success" with probability $p$ and "failure" with probability $1-p$. The total number of successes, $X$, is a binomial random variable with parameters $n$ and $p$. The binomial distribution is $P(X = k) = p(k) = \binom{n}{k}p^{k}(1-p)^{n-k}$ where $\binom{n}{k} = \frac{n!}{k!(n-k)!}$ --- ### Poisson Distribution The Poisson distribution can be derived from the binomial distribution as the number of trials, $n$, approaches infinity (i.e., $n \rightarrow \infty$), and the probability of success on each trial, $p$, approaches zero (i.e., $p \rightarrow 0$), in such a way that $np = \lambda$ $P(X = k) = \frac{\lambda^{k}}{k!}e^{-\lambda}$ --- ### Normal Distribution The normal distribution can be derived from the binomial distribution as the number of trials, $n$, approaches infinity (i.e., $n \rightarrow \infty$), and the probability of success on each trial, $p$, is not too close to 0 or 1 $P(X = x) = \frac{1}{\sigma \sqrt{2 \pi}} e^{-(x - \mu)^2/2\sigma^2}$ where $\mu = np$ and $\sigma^2 = np(1-p)$ --- ### Expectation If X is a discrete random variable with probability distribution $P(X = x) = p(x)$, the expected value of $X$, denoted by $E(X)$, is $E(X) = \sum_{i} x_{i}p(x_{i})$ $E(X)$ is the mean of $X$ and is often denoted by $\mu$ The expectation is a linear operation $E(a + \sum_{i}^{n} b_{i}X_{i}) = a + \sum_{i}^{n}b_{i}E(X_{i})$ --- ### Variance If $X$ is a random variable with expected value $E(X)$, the variance of $X$ is $Var(X) = E\\{[X - E(X)]^2\\}$ If X is a discrete random variable with probability distribution $p(x)$ and expected value $\mu = E(X)$ $Var(X) = \sum_{i} (x_{i} - \mu)^2 p(x_{i})$ The variance is often denoted by $\sigma^2$ and the standard deviation by $\sigma$ --- ### Variance and Covariance $Var(a + bX) = b^2 Var(X)$ $Var(X) = E(X^2) - [E(X)]^2$ $Cov(X, Y) = E[(X - \mu_{X})(Y - \mu_{Y})]$ $U = a + \sum_{i=1}^{n} b_{i} X_{i}$ and $V = c + \sum_{j=1}^{m} d_{j} Y_{j}$ $Cov(U,V) = \sum_{i=1}^{n} \sum_{j=1}^{m} b_{i} d_{j} Cov(X_i, Y_j)$ --- ### Variance and Covariance $Var(a + \sum_{i=1}^{n} b_{i} X_{i}) = \sum_{i=1}^{n} \sum_{j=1}^{n} b_{i} b_{j} Cov(X_i, X_j)$ If $X_{i}$ are independent, then $Cov(X_{i}, X_{j}) = 0$ for $i \ne j$. In that case $Var(\sum_{i=1}^{n} X_{i}) = \sum_{i=1}^{n} Var(X_{i})$ The correlation coefficient, $\rho$, is defined as follows $\rho = \frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}}$ --- ### Sample Mean & Variance Denote $n$ as the sample size and assume $X_{1}, X_{2}, \dots, X_{n}$ are random variables (not fixed values!), the sample mean and variance are $\overline{X} = \frac{1}{n} \sum_{i=1}^{n} X_{i}$; $s^2 = \frac{1}{n-1} \sum_{i=1}^{n} (X_{i} - \overline{X})^2$ Given $E(X_{i}) = \mu$ and $Var(X_{i}) = \sigma^2$ $E(\overline{X}) = \mu$ and $Var(\overline{X}) = \frac{\sigma^2}{n}$ --- ### Moment-Generating Function The moment-generating function (mgf) of a random variable $X$ is $M(t) = E(e^{tX})$ $M(t) = \displaystyle \sum_{x} e^{tx} p(x)$; $M(t) = \int_{-\infty}^{\infty} e^{tx} f(x) dx$ $$M^{(r)}(t) = \frac{d^{n}}{dt^{n}} \int_{-\infty}^{\infty} e^{tx} f(x) dx = \int_{-\infty}^{\infty} x^r e^{tx} f(x) dx$$ $M^{(r)}(0) = E(X^r)$ --- ### Moment-Generating Function If $X$ and $Y$ are independent random variables with mgf's $M_{X}$ and $M_{Y}$ and $Z= X + Y$ then $M_{Z}(t) = M_{X}(t)M_{Y}(t)$ If $X$ has mgf $M_{X}(t)$ and $Y = a + bX$, then $M_{Y}(t) = e^{at}M_{X}(bt)$ If $X$ follows a normal distribution with mean $\mu$ and standard deviation $\sigma$ $M_{X}(t) = e^{\mu t + \sigma^2 t^2/2}$ --- ### Cummulative Distribution Function The cummualative distribution function (cdf) of a continous random variable $X$ is $P(X \le x) = F(x) = \int_{-\infty}^{x} f(u) du$ The cdf of standard normal with $\mu = 0$ and $\sigma = 1$ is $\Phi(x) = \int_{-\infty}^{x} \frac{1}{\sqrt{2 \pi}} e^{-u^2/2} du$ --- ### Central Limit Theorem Let $X_{1}, X_{2},\dots$ be independent random variables with mean 0 and variance $\sigma^2$, common distribution function $G$ and moment-generating function $M$. Let $S_{n} = \sum_{i=1}^n X_{i}$, then $$\lim_{n \to \infty} P(\frac{S_{n}}{\sigma \sqrt{n}} \le x) = \Phi(x), -\infty < x < \infty$$ Where $\Phi(x)$ is the cumulative distribution function of the standard normal with mean 0 and variance 1. --- ### Use of Distributions in Functional Genomics 1. Null models for hypothesis testing 2. Noise or background models for site detection and differential enrichment analysis 3. Error models in regression/machine learning For example, error model for a random sample $X_{i} = \mu + \epsilon_{1} + \epsilon_{2} + \epsilon_{3} \cdots = \mu + \epsilon$ where $\epsilon \sim N(0, \sigma^2)$ expected from CLT