For a statek∈X, recall the first passage timeTk(1)=inf{n>0:Xn=k}For a fixed state k∈X, define γik to be the number of visits to the state i between two consecutive visits to k, γik=Ekn=0∑Tk(1)−11{Xn=i},\mboxfori∈S
Let P be irreducible and recurrent. Then 1. γkk=1 2. γk=γkP 3. 0<γik<∞ for each i∈S
Let {Xn,n≥0} be \mboxMarkov(λ,P). Assume P is irreducible. Then for any i∈SP(n→∞limnVi(n)=mi1)=1where Vi(n) is the number of visits to statei before the time n, and mi is the expected return time to state i, i.e., Vi(n)=k=0∑n−11{Xk=i}and mi=Ei[Ti(1)]
Let {Xn,n≥0} be \mboxMarkov(λ,P). Assume P is irreducible and has an invariant distributionπ. Then for any bounded functionf:S→RP(n→∞limn1k=0∑n−1f(Xk)=f)=1where f=i∈S∑πif(i) i.e. “time average” converges to “space average”