Lemma

Consider a real-valued RV $X$ with support $S_{X}=[a,b)$ and pdf $f_{X}$ such that: 1. $f_{X}\log_{2}f_{X}$ is Riemann-integrable (i.e. integrable in the sense we’re originally familiar with) and, 2. $$-\int_{a}^{b}f_{X}(t)\log_{2}f_{X}(t)dt=\infty$$or the entropy over the support is infinite.

Let $[X]_{n}$ be the uniform quantization of $X$ with quantization step size $$\Delta=\frac{b-a}{2^{n}}$$i.e. with $n$-bit accuracy.

Then for $n$ sufficiently large, $$\tag{bits}H([X]_{n})\approx-\int_{a}^{b}f_{X}(t)\log_{2}f_{X}(t)dt \ + \ n$$i.e. $$\lim_{n\to\infty}\left[H([X]_{n})-n\right]=-\int_{a}^{b}f_{X}(t)\log_{2}f_{X}(t)dt$$ ## Note This result also holds for any real-valued RV with: 1. Generally unbounded support and, 2. Well-defined pdf