Definition (Passage time)
Let {Xn},n≥0 be MC with the state space S. Let i∈S be a state. Denote Ti(r) to be the rth passage time to i. Specifically, let Ti(0)=0, and for r≥0, Ti(r+1)=inf{n>Ti(r):Xn=i}
Lemma
Let i∈S and r≥1. Then
- For r≥1, P(Ti(r+1)<∞∣Ti(r)<∞)=Pi(Ti(1)<∞)=hior the probability that we pass state i again given we did it r times is equivalent to probability that starting from i we return to i which is equivalent to the Hitting probability.
- Consequently, r≥0, Pi(Ti(r)<∞)=(Pi(Ti(1)<∞))r=(hi)r