Independent

Definition (Independent events)

Pairwise Independence (For 2 events) Let (Ω,F,P)(\Omega,\mathcal{F},P) be a probability space. Let G1,G2F\mathcal{G}_1,\mathcal{G}_2\subset\mathcal{F}. G1,G2\mathcal{G}_{1},\mathcal{G}_{2} are independent if A1G1,A2G2: P(A1A2)=P(A1)P(A2)\forall A_{1}\in\mathcal{G}_{1},\forall A_{2}\in\mathcal{G}_{2}: \ P(A_1\cap A_2)=P(A_1)P(A_2)Joint Independence (For n events) Let A1,...,AnFA_1,...,A_n\in\mathcal{F}. A1,...,AnA_1,...,A_n are independent if for any 2kn2\le k\le n and 1i1<<ikn1\le i_1 \lt\ldots\lt i_k\le n, P(Ai1Aik)=P(Ai1)P(Aik)P(A_{i_1}\cap\ldots\cap A_{i_k})=P(A_{i_1})\ldots P(A_{i_k}) Independence for countable events Let A1,A2,A_{1},A_{2},\dots be a countable sequence of events. We say they’re independent if P(Ai1Ain)=P(Ai1)P(Ain),{i1,,in}N\mathbb{P}(A_{i_{1}}\cap\dots \cap A_{i_{n}})=\mathbb{P}(A_{i_{1}})\dots \mathbb{P}(A_{i_{n}}),\quad\forall \{ i_{1},\dots,i_{n} \}\subset \mathbb{N}

Definition (Independent random variables)

Pairwise Independence (For 2 rvs) For X,YX,Y rvs, X ⁣ ⁣ ⁣YX\perp\!\!\!\perp Y means P(XS1,YS1)=P(XS1)P(YS2),S1,S2B(R)\mathbb{P}(X\in S_{1},Y\in S_{1})=\mathbb{P}(X\in S_{1})\mathbb{P}(Y\in S_{2}),\quad\forall S_{1},S_{2}\in \mathcal{B}(\mathbb{R}) Joint Independence (For nn rvs) Let X1,,XnX_1,\cdots,X_n be RVs. They are independent if for any A1,,AnB(R)A_1,\cdots,A_n\in \mathcal{B}(\mathbb{R}) P(X1A1,,XnAn)=P(X1A1)P(XnAn)P(X_1\in A_1,\cdots,X_n\in A_n)=P(X_1\in A_1)\cdots P(X_n\in A_n)

Theorem

rvs X,YX,Y are independent if and only if E[f(X)g(Y)]=E[f(X)]E[g(Y)]\mathbb{E}[f(X)g(Y)]=\mathbb{E}[f(X)]\mathbb{E}[g(Y)]for all bounded Borel Measurable Functions f,g:RRf,g:\mathbb{R}\to \mathbb{R}.

\begin{proof} Sufficiency P(XA,YB)=E[1A(X)1B(Y)]=E[1A(X)]E[1B(Y)]=P(XA)P(YB)\mathbb{P}(X\in A,Y\in B)=\mathbb{E}[\mathbb{1}_{A}(X)\mathbb{1}_{B}(Y)]=\mathbb{E}[\mathbb{1}_{A}(X)]\mathbb{E}[\mathbb{1}_{B}(Y)]=\mathbb{P}(X\in A)\mathbb{P}(Y\in B) Necessity If X ⁣ ⁣ ⁣Y    E[1A(X)1B(Y)]=E[1A(X)]E[1B(Y)]X\perp\!\!\!\perp Y\implies \mathbb{E}[\mathbb{1}_{A}(X)\mathbb{1}_{B}(Y)]=\mathbb{E}[\mathbb{1}_{A}(X)]\mathbb{E}[\mathbb{1}_{B}(Y)]. Then, taking linear combinations for simple f,gf,g then E[f(X)g(Y)]=E[f(X)]E[g(Y)]\mathbb{E}[f(X)g(Y)]=\mathbb{E}[f(X)]\mathbb{E}[g(Y)]Then, if f,gf,g not simple suppose f,g0f,g\ge 0 and find simple functions that approximate these and get the same result.

If f,gf,g are signed then split them into positive and negative parts and we get the same thing again.

\end{proof}

Proposition

If (Xi)i=1n(X_{i})_{i=1}^{n} are and fi(Xi)L1,1inf_{i}(X_{i})\in \mathscr{L}^{1},1\le i\le n then i=1nfi(Xi)L1\prod_{i=1}^{n}f_{i}(X_{i})\in \mathscr{L}^{1} and E[i=1nfi(Xi)]=i=1nE[fi(Xi)]\mathbb{E}\left[ \prod_{i=1}^{n}f_{i}(X_{i}) \right]=\prod_{i=1}^{n}\mathbb{E}[f_{i}(X_{i})]

Note

Pairwise independence does NOT imply joint independence.

Theorem

Any collection of random variables X1,,XnX_1,\cdots,X_n are independent if and only if pX1,,Xn(x1,,xn)=p1(x1)pn(xn)p_{X_1,\cdots,X_n}(x_1,\cdots,x_n)=p_1(x_1)\cdots p_n(x_n)

Linked from