Theorem

Consider the system $$x_{t+1}=Ax_{t}+Bu_{t},\quad y_{t}=Cx_{t}$$ 1. If $(A,B)$ is controllable there exists a solution to the Riccati Equation $$P=Q+A^{T}PA-A^{T}P+B(R+B^{T}PB)^{-1}B^{T}PA$$ 2. If $(A,B)$ is controllable and, with $Q=C^{T}C$, $(A,C)$ is observable; as $t\to\infty$ the sequence of Riccati recursions $$P_{t}=Q+A^{T}P_{t+1}A-A^{T}P_{t+1}+B(R+B^{T}P_{t+1}B)^{-1}B^{T}P_{t+1}A$$converges to some limit $P$ that satisfies $$P=Q+A^{T}PA-A^{T}P+B(R+B^{T}PB)^{-1}B^{T}PA$$That is, convergence takes place for any initial condition $\bar{P}$. Furthermore, such a $P$ is unique, and is positive definite. Finally under the optimal stationary control policy $$u_{t}=-(B^{T}PB+R)^{-1}B^{T}PAx_{t}$$the solution to $x_{t+1}=Ax_{t}+Bu_{t}$ is stable; i.e. $x_{t}\to0$ 3. Under the conditions of 2, the stationary policy above minimizes $$\limsup_{ N \to \infty } \frac{1}{N}E_{x}^{\gamma}\left[ \sum_{t=0}^{N-1}x_{t}^{T}Qx_{t}+u_{t}^{T}Ru_{t} \right]$$for the following system $$x_{t+1}=Ax_{t}+Bu_{t}+w_{t}$$for every $x\in\mathbb{R}^{n}$. Furthermore, the optimal cost is $E[w^{T}Pw]=\text{Trace}(PW)$.