FIND ME ON

GitHub

LinkedIn

Consequence of the AEP

🌱

Theorem
InfoTheory

For the memoryless continuous source {Xi}i=1\{X_{i}\}^\infty_{i=1} with pdf fXf_{X} defined on SXRS_{X}\subset\mathbb{R} and differential entropy h(X)h(X), the following hold: 1. limnPXn(Aϵ(n))=1, \mboxi.e.PXn(Aϵ(n))>1ϵ \mboxforn\mboxsufficientlylarge\lim_{n\to\infty}P_{X^{n}}\left(A_{\epsilon}^{(n)}\right)=1,\ \mbox{ i.e. }P_{X^{n}}\left(A_{\epsilon}^{(n)}\right)>1-\epsilon \ \mbox{ for }n\mbox{ sufficiently large} 2. vol(Aϵ(n))2n(h(X))+ϵ nvol(A_{\epsilon}^{(n)})\le 2^{n(h(X))+\epsilon} \ \forall n 3. vol(Aϵ(n)>(1ϵ)2n(h(X)ϵ)) \mboxforn\mboxsufficientlylargevol(A_{\epsilon}^{(n)}>(1-\epsilon)2^{n(h(X)-\epsilon)}) \ \mbox{for }n\mbox{ sufficiently large}