Average Code Rate

Code Rate

Codebook

Codeword

Dirichlet Mixture Distribution

Divergence Bound on KT

Fixed-Length (Block) Code

Huffman Lemma

Kac's Lemma (Stationary Ergodic)

Kraft Inequality for UD VLCs

Krichevsky and Trofimov Coding Distribution

Lempel-Ziv Coding

Max Redundancy on KT

Nonsingular Code

Number of Types

Penalty Lemma

Prefix Code

Prefix Code Redundancy

Probability of Decoding Error

Probability Simplex

Redundancy

Set of Types with Denominator n

Shannon Coding Theorem

Shannon-Fano-Elias Code

Size of Type Class

Type

Type Class

Uniquely Decodable

Universal Code

Universal Code PMF Q

Variable Length Code

Zero Penalty iff Universal Code