Consider the following linear control system $$x_{t+1}=Ax_{t}+Bu_{t}+w_{t}$$where $x,w\in\mathbb{R}^{n},u\in\mathbb{R}^{m}$. Suppose that $\{ w_{t} \}_{t\ge 0}$ is iid with $E[w_{t}]=0,E[w_{t}w_{t}^{T}]=W$, $\forall t\ge 0$. The goal is to find the minimum cost, $$\inf_{\gamma\in\Gamma_{A}}J(x,\gamma)=\inf_{\gamma\in\Gamma_{A}}E_{x}^{\gamma}\left[\left( \sum_{k=0}^{N-1}x_{k}^{T}Qx_{k}+u_{k}^{T}Ru_{k} \right)+ \underbrace{ x_{N}^{T}Q_{N}x_{N} }_{ \text{terminal cost} } \right]$$where $Q=Q^{T}\ge 0$ positive semidefinite and $R=R^{T}>0$ positive definite.