Achievability of the Rate Distortion Function

Achievable

Achievable (Continuous)

AEP For Continuous IID RVs

Arithmetic-Geometric Mean Inequality

Asymptotic Equipartition Property

Average Code Rate

Average Cost Constraint

Average Power Constraint

Average Probability of Error (Channel Code)

Average Probability of Error (Source-Channel Code)

Basic Cell

Binary Erasure Channel

Binary Symmetric Channel

Binary Symmetric Erasure Channel

Bit Allocation Problem

Block Codes for the Gaussian Channel

Bucklew and Wise

Capacity of Correlated Parallel Gaussian Channels

Capacity of Uncorrelated Parallel Gaussian Channels

Capacity with Decoder-Side Information (DSI)

Capacity with Full-Side Information (FSI)

Centroid Condition

Centroid Condition for MSE

Cesaro-Mean Theorem

Channel Coding Operation

Closed-loop Prediction Gain

Closed-loop Predictive Quantization

Closed-loop Predictor Coefficients

Code Rate

Codebook

Codeword

Coding Gain

Companding Quantization

Concavity of Channel Capacity in P

Consequence of the AEP

Continuous Memoryless Source

Convex Polytope

Convexity or Concavity of Information Measure

Cross-Entropy

Data Processing Inequality

Difference Quantization

Differential Cross-Entropy

Differential Divergence

Differential Entropy

Dirichlet Mixture Distribution

Discrete Communication Channel

Discrete Memoryless Channel

Discrete Memoryless Source

Discrete-Time Continuous Memoryless Channels

Discrete-Time Memoryless Gaussian Channel (AWGN Channel)

Distortion

Distortion Measure

Distortion Rate Function

Divergence

Divergence Bound on KT

Encoder Decoder Structure of Quantization

Entropy

Entropy Rate

Equivalent Properties in Discrete & Continuous IT

Ergodic

Estimation Error and differential entropy

Existence of Companding Realization

Fading Channel

FIR Filter

Fixed-Length (Block) Code

Fixed-Length Codes for Discrete Channels

Fundamental Inequality

Gain of Transform Coding

Gaussian Noise Minimizes Capacity of Additive-Noise Channel

Gaussian Rate Distortion Function

Gaussian Source Maximizes Rate Distortion Function

Hadamard's Inequality

Hamming Distortion

Hamming Rate Distortion Function

High Resolution Optimality of KLT

High-Resolution Conditions

High-Resolution Conditions LVQ

High-Resolution Optimal Vector Quantizer

Huffman Lemma

Information Capacity

Information Capacity Calculation

Information Capacity with Input Cost

Jensen-Shannon Divergence

Joint AEP

Joint Differential Entropy of Multivariate Gaussian

Joint Entropy

Kac's Lemma (Stationary Ergodic)

Karhunen-Loeve Transform

Karush-Kuhn-Tucker Conditions (KKT)

KL Transform Decorrelates X

Kraft Inequality

Kraft Inequality for UD VLCs

Krichevsky and Trofimov Coding Distribution

Lagrange Multiplier

Lagrange Multipliers Technique

Lattice

Lattice Vector Quantizer

Lempel-Ziv Coding

Linear Prediction

Lloyd-Max Algorithm

Lloyd-Max Quantizer

Log-Sum Inequality

Lossless Joint Source-Channel Coding Theorem

Lossy Source-Channel Coding Theorem

Max Redundancy on KT

Mean Squared Error Distortion

Multivariate Gaussian

Mutual Information

Nearest Neighbour Condition

Nearest Neighbour Vector Quantizer (NNVQ)

Nonsingular Code

Number of Types

Operational Channel Capacity

Operational Channel Capacity (Continuous)

Optimal Bit Allocation

Optimal Compressor

Optimal Distortion

Optimal Power Allotment

Optimal Quantizer

Optimal Vector Quantizer

Orthogonality Principle

Penalty Lemma

Performance Analysis of Quantizers

Prediction Error Upper Bound

Predictive Quantization vs Scalar Quantization

Prefix Code

Prefix Code Redundancy

Probability of Decoding Error

Probability Simplex

Quantizer Cell

R(D) is Convex in D

Rate

Rate Distortion Function

Rate Distortion Theorem

Redundancy

Regular Quantizer

Regular Vector Quantizer

Renyi Divergence

Renyi Entropy

Reproduction

Scalar Quantizer

Self-Information

Set of Types with Denominator n

Shannon Coding Theorem

Shannon Limit

Shannon-Fano-Elias Code

Shannon's Channel Coding Theorem for AWGN

Shannon's Channel Coding Theorem for the DMC

Signal to Noise Ratio

Simplification of the Norm

Size of Type Class

Source

Source Entropy

Source-Channel Fixed-Length Code

State

Stationary

Stationary & Ergodic Source

Stationary Distribution

Symmetric Channel

System SNR

Tangent Plane for LM

Transform Coding Distortion

Transform Coding with Scalar Quantization

Type

Type Class

Typical Set

Typical Set

Uniform Quantization of Real-Valued Source

Uniform Quantizer

Uniquely Decodable

Universal Code

Universal Code PMF Q

Upper Bound on Channel Capacity

Variable Length Code

Vector Quantizer

Volume

Voronoi Regions

Wide Sense Stationary Process

WSS Predictor Coefficients

Zero Penalty iff Universal Code