FIND ME ON

GitHub

LinkedIn

Ergodic Theorem

🌱

Definition
StochasticProcesses

For a state kXk\in \mathbb{X}, recall the first passage time Tk(1)=inf{n>0:Xn=k}T_{k}^{(1)}=\inf\{n>0:X_{n}=k\}For a fixed state kXk\in \mathbb{X}, define γik\gamma_{i}^{k} to be the number of visits to the state ii between two consecutive visits to kk, γik=Ek[n=0Tk(1)11{Xn=i}], \mboxforiS\gamma_{i}^{k}=E_{k}\left[\sum\limits_{n=0}^{T_{k}^{(1)}-1}\mathbb{1}_{\{X_{n}=i\}}\right], \ \mbox{for }i\in S

Let PP be irreducible and recurrent. Then 1. γkk=1\gamma_{k}^{k}=1 2. γk=γkP\gamma^{k}=\gamma^{k}P 3. 0<γik<0<\gamma_{i}^{k}<\infty for each iSi\in S

Invariant Measure

Kac’s Lemma

Let {Xn,n0}\{X_{n},n\ge0\} be \mboxMarkov(λ,P)\mbox{Markov}(\lambda,P). Assume PP is irreducible. Then for any iSi\in S P(limnVi(n)n=1mi)=1P\left(\lim_{n\to\infty}\frac{V_{i}(n)}{n}= \frac{1}{m_{i}}\right)=1where Vi(n)V_{i}(n) is the number of visits to state ii before the time nn, and mim_{i} is the expected return time to state ii, i.e., Vi(n)=k=0n11{Xk=i}V_{i}(n)=\sum\limits_{k=0}^{n-1}\mathbb{1}_{\{X_{k}=i\}}and mi=Ei[Ti(1)]m_{i}=E_{i}[T_{i}^{(1)}]

Let {Xn,n0}\{X_{n},n\ge0\} be \mboxMarkov(λ,P)\mbox{Markov}(\lambda,P). Assume PP is irreducible and has an invariant distribution π\pi. Then for any bounded function f:SRf:S\to\mathbb{R} P(limn1nk=0n1f(Xk)=f)=1P\left(\lim_{n\to\infty} \frac{1}{n}\sum\limits_{k=0}^{n-1}f(X_{k})=\overline f \right)=1where f=iSπif(i)\overline f=\sum\limits_{i\in S}\pi_{i}f(i) i.e. “time average” converges to “space average”

Linked from