Invariant Distribution

Definition (Invariant distribution)

Let $S$ denote the state space, and $P=\{P_{ij}:i,j\in S\}$ be a transition matrix. A distribution $\lambda=\{\lambda_{i}:i\in S\}$ is said to be invariant with respect to $P$ if for any $i\in S$ , $\lambda_{i}=\sum\limits_{k\in S}\lambda_{k}P_{ki}$ That is, $\lambda=\lambda P$

Lemma (1)

If $\lambda$ is an invariant measure and $0<\sum\limits_{k\in S}\lambda_{k}<\infty$ , then $\tilde\lambda_{i}= \frac{\lambda_{i}}{\sum\limits_{j\in S}\lambda_{j}}$ is an invariant distribution.

Lemma (2)

If $\lambda$ is an invariant distribution and $X_{0}\sim\lambda$ , then

$X_{m}\sim\lambda$ for any $m\ge1$
$\{X_{m+n}:n\ge0\}$ is a $Markov(\lambda,P)$

Kac’s Lemma

Methods to find invariant distribution

Applying the Definition: Apply the definition of an invariant distribution i.e. $\lambda=\lambda P$ This always works but it could be complicated (especially if $|S|=\infty$ ). If $|S|<\infty$ then take $P^{T}(\lambda^T)=\lambda^{T}$ where $\lambda^{T}$ is the eigenvector corresponding to eigenvalue $\lambda=1$ .
Fix $k\in S$ , then find $\frac{\gamma^{k}_{i}}{E_{k}[T_{k}^{(i)}]} \ \mbox{for }i\in S$ If we can compute $\gamma_{i}^{k}$ then it works, could be difficult though
Detailed Balance

Linked from

Metropolis Hastings Algorithm

Detailed Balance

Ergodic Theorem

Invariant Distribution for CTMC

Invariant Distribution

Joint Markov Chain

Convergence to Equilibrium

Dobrushin's Ergodic Coefficient

Existence of Invariant measure

Invariant Distribution ↔ Positive Recurrence