If source $\{X_{i}\}^{\infty}_{i=1}$ is a DMS (i.e. iid), then $$H(\mathcal{X})=\lim_{n\to\infty} \frac{1}{n}H(X_{1},\cdots,X_{n})=\lim_{n\to\infty} \frac{1}{n}(nH(X_{1}))=H(X_{1})$$

- Since $\tilde H(\mathcal{X})=H(\mathcal{X})$ and $H(X_{n}|X^{n-1})\downarrow\tilde H(\mathcal{X})$ by lemma 1, then we have $$H(\mathcal{X})\le H(X_{n}|X^{n-1}) \ \forall n\ge 1$$ - Can also be shown that $\frac{1}{n}H(X^{n})$ is decreasing in $n$ $$H(\mathcal{X})\le \frac{1}{n}H(X^{n}) \ \forall n\ge1$$