FIND ME ON

GitHub

LinkedIn

Independent

🌱

Definition
ProbabilityStochasticProcessesInfoTheory

Pairwise Independence (For 2 events) Let (Ī©,F,P)(\Omega,\mathcal{F},P) be a probability space. Let G1,G2āŠ‚F\mathcal{G}_1,\mathcal{G}_2\subset\mathcal{F}. G1,G2\mathcal{G}_{1},\mathcal{G}_{2} are independent if āˆ€A1∈G1,āˆ€A2∈G2:Ā P(A1∩A2)=P(A1)P(A2)\forall A_{1}\in\mathcal{G}_{1},\forall A_{2}\in\mathcal{G}_{2}: \ P(A_1\cap A_2)=P(A_1)P(A_2)Joint Independence (For n events) Let A1,...,An∈FA_1,...,A_n\in\mathcal{F}. A1,...,AnA_1,...,A_n are independent if for any 2≤k≤n2\le k\le n and 1≤i1<…<ik≤n1\le i_1 \lt\ldots\lt i_k\le n, P(Ai1āˆ©ā€¦āˆ©Aik)=P(Ai1)…P(Aik)P(A_{i_1}\cap\ldots\cap A_{i_k})=P(A_{i_1})\ldots P(A_{i_k}) Independence for countable events Let A1,A2,…A_{1},A_{2},\dots be a countable sequence of events. We say they’re independent if P(Ai1āˆ©ā‹Æāˆ©Ain)=P(Ai1)…P(Ain),āˆ€{i1,…,in}āŠ‚N\mathbb{P}(A_{i_{1}}\cap\dots \cap A_{i_{n}})=\mathbb{P}(A_{i_{1}})\dots \mathbb{P}(A_{i_{n}}),\quad\forall \{ i_{1},\dots,i_{n} \}\subset \mathbb{N}

Pairwise Independence (For 2 rvs) For X,YX,Y rvs, XāŠ„ā€‰ā£ā€‰ā£ā€‰ā£āŠ„YX\perp\!\!\!\perp Y means P(X∈S1,Y∈S1)=P(X∈S1)P(Y∈S2),āˆ€S1,S2∈B(R)\mathbb{P}(X\in S_{1},Y\in S_{1})=\mathbb{P}(X\in S_{1})\mathbb{P}(Y\in S_{2}),\quad\forall S_{1},S_{2}\in \mathcal{B}(\mathbb{R}) Joint Independence (For nn rvs) Let X1,⋯ ,XnX_1,\cdots,X_n be RVs. They are independent if for any A1,⋯ ,An∈B(R)A_1,\cdots,A_n\in \mathcal{B}(\mathbb{R}) P(X1∈A1,⋯ ,Xn∈An)=P(X1∈A1)⋯P(Xn∈An)P(X_1\in A_1,\cdots,X_n\in A_n)=P(X_1\in A_1)\cdots P(X_n\in A_n)

rvs X,YX,Y are independent if and only if E[f(X)g(Y)]=E[f(X)]E[g(Y)]\mathbb{E}[f(X)g(Y)]=\mathbb{E}[f(X)]\mathbb{E}[g(Y)]for all bounded Borel functions f,g:R→Rf,g:\mathbb{R}\to \mathbb{R}.

\begin{proof} Sufficiency P(X∈A,Y∈B)=E[1A(X)1B(Y)]=E[1A(X)]E[1B(Y)]=P(X∈A)P(Y∈B)\mathbb{P}(X\in A,Y\in B)=\mathbb{E}[\mathbb{1}_{A}(X)\mathbb{1}_{B}(Y)]=\mathbb{E}[\mathbb{1}_{A}(X)]\mathbb{E}[\mathbb{1}_{B}(Y)]=\mathbb{P}(X\in A)\mathbb{P}(Y\in B) Necessity If XāŠ„ā€‰ā£ā€‰ā£ā€‰ā£āŠ„Yā€…ā€ŠāŸ¹ā€…ā€ŠE[1A(X)1B(Y)]=E[1A(X)]E[1B(Y)]X\perp\!\!\!\perp Y\implies \mathbb{E}[\mathbb{1}_{A}(X)\mathbb{1}_{B}(Y)]=\mathbb{E}[\mathbb{1}_{A}(X)]\mathbb{E}[\mathbb{1}_{B}(Y)]. Then, taking linear combinations for simple f,gf,g then E[f(X)g(Y)]=E[f(X)]E[g(Y)]\mathbb{E}[f(X)g(Y)]=\mathbb{E}[f(X)]\mathbb{E}[g(Y)]Then, if f,gf,g not simple suppose f,g≄0f,g\ge 0 and find simple functions that approximate these and get the same result.

If f,gf,g are signed then split them into positive and negative parts and we get the same thing again.

\end{proof}

If (Xi)i=1n(X_{i})_{i=1}^{n} are and fi(Xi)∈L1,1≤i≤nf_{i}(X_{i})\in \mathscr{L}^{1},1\le i\le n then āˆi=1nfi(Xi)∈L1\prod_{i=1}^{n}f_{i}(X_{i})\in \mathscr{L}^{1} and E[āˆi=1nfi(Xi)]=āˆi=1nE[fi(Xi)]\mathbb{E}\left[ \prod_{i=1}^{n}f_{i}(X_{i}) \right]=\prod_{i=1}^{n}\mathbb{E}[f_{i}(X_{i})]

Note

Pairwise independence does NOT imply joint independence.

Any collection of random variables X1,⋯ ,XnX_1,\cdots,X_n are independent if and only if pX1,⋯ ,Xn(x1,⋯ ,xn)=p1(x1)⋯pn(xn)p_{X_1,\cdots,X_n}(x_1,\cdots,x_n)=p_1(x_1)\cdots p_n(x_n)

Linked from