Definition (Covariance)
Let X and Y be RVs, their Covariance is defined as Cov(X,Y)=E[(X−E[X])(Y−E[Y])]=E[XY]−E[X]E[Y]
Proposition (Properties of Covariance)
- Linearity: For α1,α2∈R Cov(X,α1Y1+α2Y2)=α1Cov(X,Y1)+α2Cov(X,Y2)
- Independence: If X and Y are independent then Cov(X,Y)=0
Lemma (Variance Identity)
Var(αX+βY)=α2Cov(X,X)+β2Cov(Y,Y)+2αβCov(X,Y)