Cross-Entropy

Discrete Memoryless Source

Divergence

Entropy

Entropy Rate

Ergodic

Jensen-Shannon Divergence

Joint Entropy

Mutual Information

Renyi Divergence

Renyi Entropy

Self-Information

Source

Source Entropy

State

Stationary

Stationary & Ergodic Source

Stationary Distribution

Typical Set