FIND ME ON

GitHub

LinkedIn

Foster-Lyapunov Theorems

🌱

Definition
StochasticControl

Let SS be a petite set, bRb\in\mathbb{R}, ϵ>0\epsilon>0,and V:XR+V:\mathbb{X}\to \mathbb{R}^{+}, Let {Xk}kN\{ X_{k} \}_{k\in\mathbb{N}} be a Irreducible MC. If the following is satisfied xX\forall x\in\mathbb{X} E[V(xk+1)xk=x]V(x)ϵ+b1{xS}E[V(x_{k+1})|x_{k}=x]\le V(x)-\epsilon+b\mathbb{1}_{\{ x\in S \}} then {Xk}kN\{ X_{k} \}_{k\in\mathbb{N}} is Positive Harris Recurrent (equivalently \exists an Invariant probability measure π\pi).

Let SS be a petite set, bR+b\in\mathbb{R}^{+}, and V:XR+V:\mathbb{X}\to \mathbb{R}^{+}, f:X[0,)f:\mathbb{X}\to[0,\infty) for some ϵ>0\epsilon>0. Let {xk}kN\{ x_{k} \}_{k\in\mathbb{N}} be a Markov chain. If the following holds E[V(xk+1)xk=x]V(x)ϵ+b,    xXE[V(x_{k+1})|x_{k}=x]\le V(x)-\epsilon+b, \ \ \ \ x\in\mathbb{X}then for any Invariant probability measure π\pi we have finite expectation f(x)π(dx)b\int\limits f(x) \, \pi(dx)\le b

Let SS be a compact set, b<b<\infty, and V:XR+V:\mathbb{X}\to \mathbb{R}^{+} s.t. αR+, {x:V(x)α}\forall\alpha\in\mathbb{R}^{+}, \ \{ x:V(x)\le \alpha \} is compact or equivalently limxV(x)=\lim_{ \|x\| \to \infty } V(x)=\inftyLet {xk}kN\{ x_{k} \}_{k\in\mathbb{N}} be a Markov chain. Furthermore, let τS=min{t>0:xtS}\tau_{S}=\min\{ t>0: x_{t}\in S \}, τBN={t>0:xtBN}\tau_{B_{N}}=\{ t>0:x_{t}\in B_{N} \} where BN={z:V(z)N}B_{N}=\{ z:V(z)\ge N \}, if we have that Px(min(τS,τBN)<)=1P_{x}(\min(\tau_{S},\tau_{B_{N}})<\infty)=1then xX,  E[V(xk+1)xk=x]V(x)+b1{xS}    P(τS<)=1\forall x\in\mathbb{X}, \ \ E[V(x_{k+1})|x_{k}=x]\le V(x)+b\mathbb{1}_{\{ x\in S \}}\implies P(\tau_{S}<\infty)=1