Average Code Rate

Redundancy

Uniquely Decodable

Variable Length Code

Huffman Lemma

Kraft Inequality for UD VLCs

Shannon Coding Theorem

Prefix Code

Source-Channel Fixed-Length Code

Joint AEP

Lossless Joint Source-Channel Coding Theorem

Distortion

Rate Distortion Function

Reproduction

Shannon Limit

Gaussian Source Maximizes Rate Distortion Function

Hamming Rate Distortion Function

Differential Entropy

Discrete Memoryless Source

Entropy Rate

Ergodic

Stationary & Ergodic Source