FIND ME ON

GitHub

LinkedIn

Discounted Cost

🌱

Theorem
StochasticControl

Let A\mathbb{A} be a set and {fn}\{ f_{n} \} be a sequence of maps s.t. fn:AR,nNf_{n}:\mathbb{A}\to \mathbb{R}, \forall n\in\mathbb{N}. Then lim supninfxAfn(x)infxAlim supnfn(x)\limsup_{ n \to \infty }\inf_{x\in\mathbb{A}}f_{n}(x)\le \inf_{x\in\mathbb{A}}\limsup_{ n \to \infty }f_{n}(x)

Let Vn(x,u)V(x,u)V_{n}(x,u)\uparrow V(x,u) pointwise. Suppose that VnV_{n} and VV are continuous in uu for every xx, and uU(x)=Uu\in\mathbb{U}(x)=\mathbb{U} is compact. Then, limnminuU(x)Vn(x,u)=minuU(x)V(x,u)\lim_{ n \to \infty } \min_{u\in\mathbb{U}(x)}V_{n}(x,u)=\min_{u\in\mathbb{U}(x)}V(x,u)

We define the discounted cost optimality equation (DCOE) as (T(v))(x):=minuU{c(x,u)+βE[JβN1(x1)x0,u0]}(\mathbb{T}(v))(x):=\min_{u\in\mathbb{U}}\left\{ c(x,u)+\beta\, \mathbb{E}\left[ J_{\beta}^{N-1}(x_{1})|x_{0},u_{0} \right] \right\}

Define T:vT(v)\mathbb{T}:v\mapsto \mathbb{T}(v) as our DCOE 1. If vv is a measurable R+\mathbb{R}_{+}-valued function under measurable selection conditions such that vT(v)v\ge \mathbb{T}(v) then, v(x)Jβ(x)v(x)\ge J_{\beta}(x) 2. Let vT(v)v\le T(v) and limnβnExγ[v(xn)]=0,xX,γΓA\lim_{ n \to \infty }\beta^{n}E_{x}^{\gamma}[v(x_{n})]=0,\forall x\in\mathbb{X},\gamma\in\Gamma_{A}Then v(x)Jβ(x)v(x)\le J_{\beta}(x)

If v(x)=limTJβT(x)v(x)=\lim_{ T \to \infty } J_{\beta}^{T}(x)is so that v=T(v)v=\mathbb{T}(v) where, T(v)(x)=c(x,f(x))+βE[v(x1)x0=x,u0=f(x0)]\mathbb{T}(v)(x)=c(x,f(x))+\beta E[v(x_{1})|x_{0}=x,u_{0}=f(x_{0})] is such that with γ={f,f,}\gamma=\{ f, f, \dots\}, limnβnExγ[v(xn)]=0\lim_{ n \to \infty } \beta^{n}E_{x}^{\gamma}[v(x_{n})]=0then, γ\gamma is optimal.