Definition

Let $\mathbf{R_{X}}$ be the autocorrelation matrix for our input $\mathbf{X}$. We first order the eigenvalues of $\mathbf{R_{X}}$ s.t. $$\lambda_{1}\ge \dots\ge \lambda_{k}\ge 0$$and let $\mathbf{u}_{1},\dots,\mathbf{u}_{k}$ be the corresponding orthogonal eigenvectors. We then normalize $\mathbf{u}_{i}$ to unit length $$\lVert \mathbf{u}_{i} \rVert =1, \quad i=1,\dots,k$$Now we define $\mathbf{U}=\begin{bmatrix}\mathbf{u}_{1}&\mathbf{u}_{2}&\dots&\mathbf{u}_{k}\end{bmatrix}$ and let $$\mathbf{T}=\mathbf{U}^{T}=\begin{bmatrix}\mathbf{u}_{1}^{T}\\\mathbf{u}_{2}^{T}\\\vdots\\\mathbf{u}_{k}^{T}\end{bmatrix}$$ We call $T$ the Karhunen-Loeve transform (KLT) matrix for $\mathbf{X}$.

Remarks

• $\mathbf{T}$ is an orthogonal matrix
• $\mathbf{T}$ is not unique in general