FIND ME ON

GitHub

LinkedIn

Joint AEP

🌱

Theorem
InfoTheory

Let {(Xi,Yi)}i=1\{(X_{i},Y_{i})\}^\infty_{i=1} be a |DMS with common pmf pXYp_{XY} on X×Y\mathcal{X}\times\mathcal{Y}. Given δ>0\delta>0 and integer n1n\ge1, the jointly typical set Aδ(n)A_\delta^{(n)} (or jointly δ\delta -typical set) with respect to the source is Aδ(n)={(xn,yn)Xn×Yn:1nlog2(pXn(xn))H(X)δ,1nlog2(pYn(yn))H(Y)δ,1nlog2(pXnYn(xn,yn))H(X,Y)δ}\begin{align*} A_{\delta}^{(n)}=\left\{(x^{n},y^{n})\in \mathcal{X}^{n}\times\mathcal{Y}^{n}:\left|-\frac{1}{n}\log_{2}(p_{X^{n}}(x^{n}))-H(X)\right|\le\delta,\right.\\ \left|-\frac{1}{n}\log_{2}(p_{Y^{n}}(y^{n}))-H(Y)\right|\le\delta,\\ \left.\left|-\frac{1}{n}\log_{2}(p_{X^{n}Y^{n}}(x^{n},y^{n}))-H(X,Y)\right|\le\delta \right\}\\ \end{align*}

For a DMS {(Xi,Yi)}i=1\{(X_{i},Y_{i})\}^\infty_{i=1} with pmf pXYp_{XY} on X×Y\mathcal{X}\times\mathcal{Y}, then the joint typical set Aδ(n)A_{\delta}^{(n)} defined with respect to source pXYp_{XY} satisfies: 1. PXnYn(Aδ(n))=P((Xn,Yn)Aδ(n))>1δP_{X^{n}Y^{n}}(A_\delta^{(n)})=P((X^{n},Y^{n})\in A_{\delta}^{(n)})>1-\delta for nn sufficiently large 2. Aδ(n)2n(H(X,Y)+δ)|A_\delta^{(n)}|\le2^{n(H(X,Y)+\delta)} 3. Aδ(n)(1δ)2n(H(X,Y)δ)|A_\delta^{(n)}|\ge(1-\delta)2^{n(H(X,Y)-\delta)} for nn sufficiently large