ℹ

Orthogonality Principle

Recall

In our closed-loop linear predictor we defined the error as en=Xn−∑i=1maiXn−ie_{n}=X_{n}-\sum_{i=1}^{m}a_{i}X_{n-i}We then have that E[enXn−j]=E[(Xn−∑i=1maiXn−i)Xn−j]=E[XnXn−j]−∑k=1maiE[Xn−kXn−j]\begin{align*} E[e_{n}X_{n-j}]&=E\left[ \left( X_{n}-\sum_{i=1}^{m}a_{i}X_{n-i} \right)X_{n-j} \right]\\ &=E[X_{n}X_{n-j}]-\sum_{k=1}^{m}a_{i}E[X_{n-k}X_{n-j}] \end{align*}We see that from (*), a1,…,ama_{1},\dots,a_{m} is optimal if and only if E[enXn−j]=0E[e_{n}X_{n-j}]=0, j=1,…,mj=1,\dots,m.

Theorem (Orthogonality Principle)

The linear predictor X^n=∑k=1makXn−k\hat{X}_{n}=\sum_{k=1}^{m}a_{k}X_{n-k} is optimal in the MSE sense if and only if the prediction error is orthogonal to all Xn−jX_{n-j} i.e.X^n optimal   ⟺  (Xn−X^n)⊥Xn−j   j=1,…,m\hat{X}_{n}\text{ optimal }\iff(X_{n}-\hat{X}_{n})\perp X_{n-j} \ \ \ j=1,\dots,m