Average Code Rate

Huffman Lemma

Kraft Inequality

Redundancy

Shannon Coding Theorem

Uniquely Decodable

Variable Length Code

Prefix Code

Lossless Joint Source-Channel Coding Theorem

Source-Channel Fixed-Length Code

Distortion

Gaussian Source Maximizes Rate Distortion Function

Hamming Distortion

Rate Distortion Function

Shannon Limit

Differential Entropy

Asymptotic Equipartition Property

Discrete Memoryless Source

Entropy Rate

Stationary & Ergodic Source