Countable state space
For a Markov chain , if there exists an element i i i E i [ τ i ] < ∞ E_{i}[τ_{i}] < ∞ E i [ τ i ] < ∞ Invariant probability measure π k = E [ ∑ j = 0 τ i − 1 1 { x j = k } E i [ τ i ] | x 0 = i ] , k ∈ X \pi_{k}=\mathbb{E} \left[ \frac{\sum_{j=0}^{\tau_{i}-1}\mathbb{1}_{\{ x_{j}=k \}}}{\mathbb{E}_{i}[\tau_{i}]}\middle|x_{0}=i \right], \, k\in \mathbb{X} π k = E [ E i [ τ i ] ∑ j = 0 τ i − 1 1 { x j = k } x 0 = i ] , k ∈ X
Uncountable state space
For a Irreducible Markov chain for which E α [ τ α ] < ∞ \mathbb{E}_{\alpha}[\tau_{\alpha}]<\infty E α [ τ α ] < ∞ Atom α \alpha α Invariant probability measure π ( A ) = E [ ∑ j = 0 τ α − 1 1 { x j ∈ A } E α [ τ α ] | x 0 ∈ α ] , A ∈ B ( X ) \pi(A)=\mathbb{E} \left[ \frac{\sum_{j=0}^{\tau_{\alpha}-1}\mathbb{1}_{\{ x_{j}\in A \}}}{\mathbb{E}_{\alpha}[\tau_{\alpha}]}\middle|x_{0}\in\alpha \right], \, A\in \mathcal{B}(\mathbb{X}) π ( A ) = E [ E α [ τ α ] ∑ j = 0 τ α − 1 1 { x j ∈ A } x 0 ∈ α ] , A ∈ B ( X )