FIND ME ON

GitHub

LinkedIn

Markov Property

🌱

Definition
StochasticProcesses

Let XnX_{n} be a MC s.t. (Xn)n0\mboxMarkov(λ,P)(X_n)_{n\ge0}\sim \mbox{Markov}(\lambda,P), and m0m\ge0 be a fixed integer. Then conditional on {Xm=i}\{X_m=i\} 1. (Xm+n)n0(X_{m+n})_{n\ge0} is independent of X0,,Xm1X_0,\cdots,X_{m-1} 2. (Xm+n)n0(X_{m+n})_{n\ge0} is \mboxMarkov(δi,P)\mbox{Markov}(\delta_i,P)

where δi\delta_{i} is the Dirac Distribution at state ii.

Let λ\lambda be a distribution on SS, and PP be a stochastic matrix. Assume for any N0N\ge0, and any states i0,,iNSi_0,\cdots,i_N\in S P(X0=i0,,XN=iN)=λi0pi0i1piN1iNP(X_0=i_0,\cdots,X_N=i_N)=\lambda_{i_0}p_{i_0i_1}\cdots p_{i_{N-1}i_N} Then Xn,n0Markov(λ,P)X_n,n\ge0\sim Markov(\lambda,P), or Xn,n0X_{n},n\ge0 is a MC.

Let (Xn)n0\mboxMarkov(λ,P)(X_{n})_{n\ge0}\sim\mbox{Markov}(\lambda,P) be a time homogeneous MC, and TT be a stopping time. Then for any integer m0m\ge0 and state iSi\in S, conditional on {T=m,XT=i}\{T=m,X_{T}=i\}, 1. (XT+n)n0(X_{T+n})_{n\ge0} is conditionally independent of X0,,XT,TX_{0},\cdots,X_{T},T 2. (XT+n)n0(X_{T+n})_{n\ge0} is \mboxMarkov(δi,P)\mbox{Markov}(\delta_{i},P)

where δi\delta_{i} is the Dirac Distribution at state ii.

Let (Xn)n0\mboxMarkov(λ,P)(X_{n})_{n\ge0}\sim\mbox{Markov}(\lambda,P) be a time homogeneous MC, and TT be a stopping time such that P(T<)=1P(T<\infty)=1. Then for any state iSi\in S, conditional on XT=iX_{T}=i, 1. (XT+n)n0(X_{T+n})_{n\ge0} is conditionally independent of X0,,XT,TX_{0},\cdots,X_{T},T 2. (XT+n)n0(X_{T+n})_{n\ge0} is \mboxMarkov(δi,P)\mbox{Markov}(\delta_{i},P)

where δi\delta_{i} is the Dirac Distribution at state ii.

1. Given AA, AA and {Xn}n0\{X_{n}\}_{n\ge0} may have a complicated dependence 2. {T=m}\{T=m\} only depends on {X0,,Xm}\{X_{0},\cdots,X_{m}\}

Intuition

This generalizes the Markov Property to also handle random times (i.e. the stopping time TT).

If {Xt:t0}\mboxMarkov(λ,Q)\{X_{t}:t\ge0\}\sim\mbox{Markov}(\lambda,Q), then fix a time s0s\ge0, conditional on {Xs=i}\{X_{s}=i\}, we have that - {Xt+s:t0}\mboxMarkov(δi,Q)\{X_{t+s}:t\ge0\}\sim\mbox{Markov}(\delta_{i},Q) - {Xt+s:t0}\{X_{t+s}:t\ge0\} is conditionally independent of {Xr:t<s}\{X_{r}:t< s\}

Linked from