For a DMS $\{X_i\}_{i=1}^\infty$ with pmf $p_X$ and alphabet $\mathcal{X}$, $$-\frac{1}{n}\log_2(p_{X^n}(X^n))\xrightarrow{n\to\infty}H(X)\mbox{ in probability}$$for the average information of $X_i,n\ge1$ converges in probability to the entropy of $X$. For a stationary ergodic source $\{X_i\}_{i=1}^\infty$ we have $$-\frac{1}{n}\log_2(p_{X^n}(X^n))\xrightarrow{n\to\infty}H(\mathcal{X})\mbox{ in probability}$$

For a DMS $\{X_i\}_{i=1}^\infty$ with pmf $p_X$ on $\mathcal{X}$ and entropy $H(X)$, the typical set $A_\epsilon^{(n)}$ satisfies: 1. $\lim_{n\to\infty}P(A_\epsilon^{(n)})=1$ 2. $|A_\epsilon^{(n)}|\le2^{n(H(X)+\epsilon)}$ 3. $|A_\epsilon^{(n)}|\ge(1-\epsilon)2^{n(H(X)-\epsilon)}$ for $n$ sufficiently large