Asymptotic Equipartition Property

Cesaro-Mean Theorem

Convexity or Concavity of Information Measure

Cross-Entropy

Data Processing Inequality

Discrete Memoryless Source

Divergence

Entropy

Entropy Rate

Ergodic

Fundamental Inequality

Jensen-Shannon Divergence

Joint Entropy

Kraft Inequality

Log-Sum Inequality

Mutual Information

Renyi Divergence

Renyi Entropy

Self-Information

Source

Source Entropy

State

Stationary

Stationary & Ergodic Source

Stationary Distribution

Typical Set