Theorem (Chapman-Kolmogorov Equation)

Let $(X_n)_{n\ge0}\sim Markov(\lambda,P)$. Then for each $N\ge1$ and $j\in S$, $$P(X_N=j|X_0=i)=(P^N)_{ij}$$ Consequently, $$P(X_N=j)=(\lambda P^N)_j$$