FIND ME ON

GitHub

LinkedIn

Ionescu Tulcea Theorem

🌱

Theorem
ProbabilityStochasticProcesses

Let X0,X1,X_{0},X_{1},\dots be a sequence of Borel spaces and, for n=0,1,n=0,1,\dots define Yn:=X0××XnY_{n}:=X_{0}\times\dots \times X_{n} and Y:=n=0XnY:=\prod_{n=0}^{\infty}X_{n}. Let ν\nu be an arbitrary probability measure on X0X_{0} and for every n=0,1,,n=0,1,\dots, let Pn(dxn+1yn)P_{n}(dx_{n+1}\mid y_{n}) be a Stochastic Kernel on Xn+1X_{n+1} given YnY_{n}. Then there exists a unique probability measure PνP_{\nu} on YY s.t., for every measurable rectangle B0××BnB_{0}\times\dots \times B_{n} in YnY_{n}

Pν(B0××Bn)=B0ν(dx0)B1P0(dx1x0)B2P1(dx2x0,x1) ⁣BnPn1(dxnx0,,xn1) \begin{align*} P_{\nu}(B_{0}\times\dots \times B_{n})=&\int\limits _{B_{0}} \, \nu(dx_{0}) \int\limits _{B_{1}} \, P_{0}(dx_{1}\mid x_{0}) \int\limits _{B_{2}} \, P_{1}(dx_{2}\mid x_{0},x_{1})\\ & \dots \int\limits _{B_{n}} \, P_{n-1}(dx_{n}\mid x_{0},\dots,x_{n-1}) \end{align*}

^statement

Moreover, for any nonnegative measurable function uu on YY, the function xu(y)Px(dy)x\mapsto \int\limits u(y) \, P_{x}(dy) is measurable on X0X_{0}, where PxP_{x} stands for PνP_{\nu} when ν\nu is the probability Concentrated at xX0x \in X_{0}.

We often (informally) will write the measure PνP_{\nu} as Pν(dx0,dx1,dx2,)=ν(dx0)P0(dx1x0)P1(dx2x0,x1)P_{\nu}(dx_{0},dx_{1},dx_{2}\dots,)=\nu(dx_{0})P_{0}(dx_{1}\mid x_{0})P_{1}(dx_{2}\mid x_{0},x_{1})and let π={πt}t0\pi=\{ \pi_{t} \}_{t\ge 0} be some arbitrary control Policy, Then the strategic measure PνπP_{\nu}^{\pi} can be written as Pνπ(dx0,du0,dx1,du1,)=ν(dx0)π0(du0x0)Q(dx1x0,u0)π1(du1x0,u0,x1)Q(dx2x1,u1)\begin{align*} P_{\nu}^{\pi}(dx_{0},du_{0},dx_{1},du_{1},\dots)=&\nu(dx_{0})\pi_{0}(du_{0}\mid x_{0})Q(dx_{1}\mid x_{0},u_{0})\cdot\\ &\pi_{1}(du_{1}\mid x_{0},u_{0},x_{1})Q(dx_{2}\mid x_{1},u_{1})\dots \end{align*}

Source

Hernández-Lerma, Onésimo, and Jean B. Lasserre. Discrete-time Markov control processes: basic optimality criteria. Vol. 30. Springer Science & Business Media, 2012.

Linked from