If {Xi}i=1∞ are Independent in L2(Ω,F,P) with E[Xi]=μ,∀i≥1 and Var(Xi)≤V<∞,∀i≥1, then n1i=1∑nXin→∞μ\mboxinprobabilityor P(n1i=1∑nXi−μ≥ϵ)→1,∀ϵ>0
\begin{proof} Sufficient to prove for μ=0 (we can define Xi=Xi′−μ) For μ=0, P(n1i=1∑nXi≥ϵ)≤ϵ21En1i=1∑nXi2=(ϵn)21i=1∑nE[Xi2]≤ϵ2n2Vn=ϵ2nV=Oϵ(n1)→0Chebyshevindependence of variance where the first inequality is due to Chebyshev Inequality, the second is by the fact that Variance and the third is by our upper bound on the variance. \end{proof}
The Strong one
Let Xn,n∈N be a sequence of iidrv on (Ω,F,P), with ∣Xi∣≤C,∀i≥1. Let μ=E[∣Xi∣],∀i≥1. Then, P(n→∞limn1i=1∑nXi=μ)=1
\begin{proof} Again we reduce to the case where μ=0. Let σ2=E[Xi2]<∞. We first note that E[nX1+⋯+Xn2]=n21(σ2+⋯+σ2)=nσ2Let Zn=nX1+⋯+Xn2, we then note that m≥1∑E[Zm2]=m≥1∑m2σ2<∞.So by Beppo Levi Theorem we have that: - ∑m≥1Zm2<∞a.s. and hence - Zm2→0a.s. But, now suppose n→∞ and find m=m(n) with m2≤n<(m+1)2. Have that n−m2≤(m+1)2−m2=2m+1.Then, nX1+⋯+Xn=nX1+⋯+Xm2+O(2m+1)≤m2X1+⋯+Xm2+O(m1)→0a.s..
\end{proof} # Intuition You can think of this as a sequence of i.i.d. RVs with finite expectation converging to the mean almost surely (or w.p.1.).