Let {(xkā,ukā)} be a collection that satisfies this model: xk+1ā=f(xkā,ukā,wkā) where xtāāX represents the state variable, utāāU represents the action variable, wtāāW is an i.i.d noise process, and f a measurable function. We assume that X,U,W are Borel subsets of Polish spaces; these subsets are also called standard Borel. If {(xkā,ukā)} also satisfies P(xk+1āāBā£x[0,k]ā=a[0,t]ā,u[0,k]ā=b[0,t]ā)=P(xk+1āāBā£xkā=akā,ukā=bkā)āBāB(R),kāZ+ā Then we call {(xkā,ukā)} a controlled Markov chain.
conccccccxx # More information Consider this model again: xk+1ā=f(xkā,ukā,wkā)
^statespace
where xtāāX, utāāU, wtāāW, f a measurable function, X,U,W are standard Borel. We assume all random variables live in some probability space (Ī©,F,P). The collection, {(xkā,ukā)}, satisfying also satisfies
P(xk+1āāBā£x[0,k]ā=a[0,t]ā,u[0,k]ā=b[0,t]ā)ā=P(xk+1āāBā£xkā=akā,ukā=bkā)=:T(Bā£atā,btā)ā ^property
āBāB(R),kāZ+ā, where T(ā
ā£x,u) is a Stochastic Kernel s.t. T:XĆUāX so that: >- For every BāB(R), T(Bā£ā
,ā
) is a measurable function on XĆU and; >- For every fixed (a,b)āXĆU, T(ā
ā£x,u) is a probability measure on (X,B(X)).
That is, all Stochastic Processes that satisfy , admit a Stochastic Realization in the form of almost surely.