Exponential Random Variable

Definition (Exponential)

A continuous RV $X$ is called exponential with parameter $\lambda$ if its pdf is $f(x)=\left\{ \begin{array} \ \lambda e^{-\lambda x}&x\ge0\\ 0&\mbox{otherwise} \end{array} \right.$ and cdf $F(x)=\left\{ \begin{array} 01-e^{-\lambda x}&x>0\\ 0&x\le0 \end{array} \right.$ For RV $X\sim \mbox{Exp}(\lambda)$ $\begin{align*} E[X]&=\frac{1}{\lambda}\\ \mbox{Var}(X)&=\frac{1}{\lambda^{2}} \end{align*}$

Intuition

The exponential RV is used to model the waiting time between states, if the average waiting time is constant . The parameter $\lambda$ models the rate at which the states transition. See also What is the intuition behind the exponential distribution? - Mathematics Stack Exchange

Theorem

For $n=1,\cdots,$ let $X_n$ be the elapsed time between the occurrences of the $n-1$ th and $n$ th events of a Poisson Process with rate $\lambda$ . Then $X_n$ is the exponential RV with parameter $\lambda$

Theorem (Memoryless property)

Let $T$ be a RV taking positive values. Then $T$ has an exponential distribution if and only if it has the following memoryless property $P(T>s+t|T>s)=P(T>t), \ \mbox{for all }s,t\ge0$

Remark

Conditional on $\{T>s\}$ , then $T-s$ has the same distribution of $T$ .

Theorem (Sum of Independent Exponential RVs)

Let $X_{1},X_{2},\ldots$ be a sequence of independent exponentially distributed random variables. Assume $X_{n}\sim\mbox{Exp}(\lambda_{n})$ for $\lambda_{n}>0$ and each integer $n\ge0$ , then,

If $\sum\limits_{n=1}^{\infty} \frac{1}{\lambda_{n}}<\infty$ , then $P\left(\sum\limits_{n=1}^{\infty}X_{n}<\infty\right)=1$
If $\sum\limits_{n=1}^{\infty} \frac{1}{\lambda_{n}}=\infty$ , then $P\left(\sum\limits_{n=1}^{\infty}X_{n}=\infty\right)=1$

Theorem (Infimum of Independent Exponential RVs)

Let $I$ be countable set. Let $\{T_{k}: k\in I\}$ be a sequence of independent exponentially distributed RVs such that $T_{k}\sim\mbox{Exp}(q_{k})$ and $0<q=\sum\limits_{k\in I}q_{k}<\infty$ . Denote $T$ as the infimum of the infinite sequence: $T=\inf\{T_{n}:n\in\mathbb{N} \}$ Then this infimum (or $T$ ) is attained at a unique random value $K$ of $k$ w.p.1 i.e. $P(K<\infty)=1$ Moreover $T$ and $K$ are independent with $T\sim\mbox{Exp}(q)$ and $P(K=k)=\frac{q_{k}}{q}$

Definition from class (which kinda sucks)

Now define $K$ as follows: if there exists unique $k\in I$ such that $T_{k}=T$ , then $K=k$ ; otherwise $K=\infty$ i.e. $K=\begin{cases}k&\exists\mbox{ unique }k\mbox{ s.t. }T_{k}=T \\ \infty&otherwise\end{cases}$ Then

$P(K<\infty)=1$ and $P(K=k)=\frac{q_{k}}{q}$
$T\sim\mbox{Exp}(q)$
$T$ and $K$ are independent

Linked from

Differential Entropy

Exponential Random Variable

Poisson Process

Holding Time