FIND ME ON

GitHub

LinkedIn

Stationary Team Policy

🌱

Definition
StochasticControl

Definition

Given a static stochastic team problem {J;Γi,iN}\{ J;\Gamma^{i},i\in\mathcal{N} \}, a policy NN-tuple γΓ\underline{\gamma}\in\mathbf{\Gamma} is stationary if: 1. J(γ)J(\underline{\gamma}) is finite. 2. The NN partial derivatives in the following equations are well-defined. 3. γ\underline{\gamma} satisfies these equations: ui(Eξyi[L(ξ;γi(yi),ui)])ui=γi(yi)=0 a.s. iN(🌝)\tag{🌝}\nabla_{u_{i}}\left(E_{\xi|y^{i}}[L(\xi;\underline{\gamma}^{-i}(\mathbf{y}^{-i}),u^{i})]\right)|_{u^{i}=\gamma^{i}(y^{i})}=0\text{ a.s. }\quad i\in\mathcal{N}whereγi(yi)={γm(ym):mNi}\underline{\gamma}^{-i}(\mathbf{y}^{-i})=\{ \gamma^{m}(y^{m}):m\in\mathcal{N} \setminus i \} ## Remark A pbp solution γΓ\underline{\gamma}^{*}\in\mathbf{\Gamma} for a static team problem (J,Γ)(J,\mathbf{\Gamma}) would be given by minβΓiJ(γi,,β)=J(γ)iN\min_{\beta\in\Gamma^{i}}J(\underline{\gamma}^{-i,*},\beta)=J(\underline{\gamma}^{*})\quad i\in\mathcal{N}which can be equivalently written as minuSiEξyi[L(ξ;γi,(yi),u)]=Eξyi[L(ξ;γ(y))],iN(⭐)\tag{⭐}\min_{u\in S^{i}}E_{\xi|y^{i}}\left[ L(\xi;\underline{\gamma}^{-i,*}(\mathbf{y}^{-i}),u) \right]=E_{\xi|y^{i}}\left[ L(\xi;\underline{\gamma}^{*}(\mathbf{y})) \right],\quad i\in\mathcal{N} We can then see that 🌝 is equivalent to ⭐ if L(ξ;u)L(\xi;\mathbf{u}) is convex in uiu^{i} and SiS^{i} are convex sets (as the derivative equalling zero of a convex function means we’ve achieved the minimum).

We can then relax the above conditions to: - L(ξ;u)L(\xi;\mathbf{u}) is continuously differentiable in each agent’s action variable for every ξΞ\xi\in\Xi, and; - Si,iNS^{i},i\in\mathcal{N}, are open subsets of finite-dimensional vector spaces. to then say that 🌝 is a necessary condition for ⭐.

This then leads us to Theorem 2.4.4 where we apply this idea along with Lemma 2.4.1.

Linked from