If {Xi}i=1∞ are Independent random variables in L2(Ω,F,P) with E[Xi]=μ,∀i≥1 and Var(Xi)≤V<∞,∀i≥1, then n1i=1∑nXin→∞μ\mboxinprobabilityor P(n1i=1∑nXi−μ≥ϵ)→1,∀ϵ>0
\begin{proof} Sufficient to prove for μ=0 (we can define Xi=Xi′−μ) For μ=0, P(n1i=1∑nXi≥ϵ)≤ϵ21En1i=1∑nXi2=(ϵn)21i=1∑nE[Xi2]≤ϵ2n2Vn=ϵ2nV=Oϵ(n1)→0Chebyshevindependence of variance where the first inequality is due to Chebyshev Inequality, the second is by the fact that Independence of Variance and the third is by our upper bound on the Variance. \end{proof}
Theorem (Strong Law of Large Numbers)
Let Xn,n∈N be a sequence of iidrv on (Ω,F,P), with ∣Xi∣≤C,∀i≥1. Let μ=E[∣Xi∣],∀i≥1. Then, P(n→∞limn1i=1∑nXi=μ)=1
\begin{proof} Again we reduce to the case where μ=0. Let σ2=E[Xi2]<∞. We first note that E[nX1+⋯+Xn2]=n21(σ2+⋯+σ2)=nσ2Let Zn=nX1+⋯+Xn2, we then note that m≥1∑E[Zm2]=m≥1∑m2σ2<∞.So by Beppo-Levi (Probability) we have that:
Zm2→0a.s. But, now suppose n→∞ and find m=m(n) with m2≤n<(m+1)2. Have that n−m2≤(m+1)2−m2=2m+1.Then, nX1+⋯+Xn=nX1+⋯+Xm2+O(2m+1)≤m2X1+⋯+Xm2+O(m1)→0a.s..
\end{proof}
Intuition
You can think of this as a sequence of i.i.d. RVs with finite expectation converging to the mean almost surely (or w.p.1.).