A source can be compressed only when it has redundancy which is the amount of “useless information” that can be eliminated via fixed length compression codes. >[!def] Redundancy >For a stationary ergodic source, its total redundancy $\rho_{T}$ is defined as: $$\rho_T=\log_2|\mathcal{X}|-H(\mathcal{X})$$where $\log_2|\mathcal{X}|$ is the compression rate of UD code and $H(\mathcal{X})$ is the rate for asymptotically lossless code.

where: >[!def] Intra-symbol source redundancy >$\rho_D$ is the redundancy due to source’s non-uniform marginal pmf $$\rho_D=\log_2|\mathcal{X}|-H(X_1)$$this looks at the redundancy for each RV isolated from others (i.e. some letters in the alphabet are just more common than others independent of their ordering in words or sentences)