(A) Ring Generated by R

(Cumulative) Distribution Function

(Ɛ) Simple Predicable Processes

(n-μ)-small

(P) Predictable σ-algebra

(R) Set of Predictable Rectangles

(Λ) Set of Predictable Locally Integrable Processes

Acceptance Rate

Accessible

Achievability of the Rate Distortion Function

Achievable

Achievable (Continuous)

Adapted

Admissible

AEP For Continuous IID RVs

Almost Sure Convergence

Almost surely

Aperiodic

Arithmetic-Geometric Mean Inequality

Asian Call Option

Asymptotic Equipartition Property

Atom

Average Code Rate

Average Cost Constraint

Average Power Constraint

Average Probability of Error (Channel Code)

Average Probability of Error (Source-Channel Code)

Axioms of Probability

Basic Cell

Bayes' Rule

Bayesian Statistics

Bernoulli Trial

Bichteler-Dellacherie Theorem

Binary Erasure Channel

Binary Symmetric Channel

Binary Symmetric Erasure Channel

Binomial Random Variable

Bit Allocation Problem

Block Codes for the Gaussian Channel

Borel-Cantelli Lemma

Brownian Motion

Bucklew and Wise

Càdlàg

Capacity of Correlated Parallel Gaussian Channels

Capacity of Uncorrelated Parallel Gaussian Channels

Capacity with Decoder-Side Information (DSI)

Capacity with Full-Side Information (FSI)

Cauchy-Schwartz Inequality

Central Limit Theorem

Centroid Condition

Centroid Condition for MSE

Cesaro-Mean Theorem

Channel Coding Operation

Chapman Kolmogorov Equation

Chebyshev Inequality

Closed-loop Prediction Gain

Closed-loop Predictive Quantization

Closed-loop Predictor Coefficients

Code Rate

Codebook

Codeword

Coding Gain

Coin Tossing Probability Space

Communication

Companding Quantization

Concavity of Channel Capacity in P

Conditional Expectation

Conditional Independence

Conditional Probability Density Function

Conditional Probability Mass Function

Conditional Probability of Events

Conditional Variance

Consequence of the AEP

Construction of Invariant measure

Continuity of Probability

Continuous Martingale with Finite Variation is Constant

Continuous Memoryless Source

Continuous Time Markov Chain

Convergence in Distribution

Convergence in Expectation

Convergence to Equilibrium

Convex Polytope

Convexity or Concavity of Information Measure

Countable Sub-additivity of Events

Covariance

Covariance Matrix

Cross Variation

Cross-Entropy

Data Processing Inequality

De Finetti's Theorem

Decreasing Events

Density of Ɛ

Derivative

Detailed Balance

Difference Quantization

Differential Cross-Entropy

Differential Divergence

Differential Entropy

Dirac Distribution

Dirichlet Mixture Distribution

Discrete Communication Channel

Discrete Memoryless Channel

Discrete Memoryless Source

Discrete-Time Continuous Memoryless Channels

Discrete-Time Memoryless Gaussian Channel (AWGN Channel)

Disintegration

Distortion

Distortion Measure

Distortion Rate Function

Distribution

Divergence

Divergence Bound on KT

Dobrushin's Ergodic Coefficient

Doléans Measure

Doob's Forward Convergence Theorem

Doob's Maximal Inequalities

Doob's Optional Sampling Theorem

Doob's Upcrossing Inequality

Encoder Decoder Structure of Quantization

Entropy

Entropy Rate

Equivalent Bounds

Equivalent Properties in Discrete & Continuous IT

Ergodic

Ergodic Theorem

Estimation Error and differential entropy

European Call Option

European Put Option

Event Space

Exchangeable

Excursion Time

Existence of Càdlàg Version

Existence of Companding Realization

Existence of Invariant measure

Existence of Sequences of Independent rvs

Existence of Uniform Measure

Expectation

Expectation of a Function of a Random Variable

Exponential Random Variable

Extension Theorem

Fading Channel

Feller condition for invariant measure

Feller Property

Filtration

FIR Filter

Fixed-Length (Block) Code

Fixed-Length Codes for Discrete Channels

Forward Contract

Foster-Lyapunov Theorems

Fundamental Inequality

Gain of Transform Coding

Gamma Random Variable

Gaussian Noise Minimizes Capacity of Additive-Noise Channel

Gaussian Process

Gaussian Random Variable

Gaussian Rate Distortion Function

Gaussian Source Maximizes Rate Distortion Function

Geometric Random Variable

Hadamard's Inequality

Hamming Distortion

Hamming Rate Distortion Function

Harris Recurrent

High Resolution Optimality of KLT

High-Resolution Conditions

High-Resolution Conditions LVQ

High-Resolution Optimal Vector Quantizer

Hitting Time

Holding Time

Huffman Lemma

iid

In Probability Convergence

Increasing Events

Independence of Bounded RV

Independent

Indistinguishable

Indistinguishable

Information Capacity

Information Capacity Calculation

Information Capacity with Input Cost

Invariant Distribution

Invariant Distribution ↔ Positive Recurrence

Invariant Distribution for CTMC

Invariant Measure

Invariant probability measure

Inversion Method

Ionescu Tulcea Theorem

Irreducible

Itô Isometry

Itô Isometry v2

Itô Isometry v3

Itô Representation Theorem

Itô Stochastic Integral

Itô Stochastic Integral is a Local Martingale

Itô Stochastic Integral on Ɛ is a R.C. Martingale

Itô Stochastic Integral on Λ is a R.C. Martingale

Itô's Formula

Jensen-Shannon Divergence

Jensen's Inequality

Joint AEP

Joint Differential Entropy of Multivariate Gaussian

Joint Distribution Function

Joint Entropy

Joint Markov Chain

Joint Probability Density Function

Joint Probability Mass Function

Jump Chain

Jump Time

Kac's Lemma

Kac's Lemma (Stationary Ergodic)

Karhunen-Loeve Transform

Karush-Kuhn-Tucker Conditions (KKT)

KL Transform Decorrelates X

Kolmogorov 0-1 Law

Kolmogorov Extension Theorem

Kraft Inequality

Kraft Inequality for UD VLCs

Krichevsky and Trofimov Coding Distribution

Lagrange Multiplier

Lagrange Multipliers Technique

Lattice

Lattice Vector Quantizer

Law of Large Numbers

Law of Total Probability

Left Continuous Adapted = Predictable

Lempel-Ziv Coding

Lévy's Convergence Theorems

Likelihood

Limits of Events

Linear Prediction

Lloyd-Max Algorithm

Lloyd-Max Quantizer

Local Martingale

Log-likelihood

Log-Sum Inequality

Lossless Joint Source-Channel Coding Theorem

Lossy Source-Channel Coding Theorem

Lp Martingale

Marginal Probability Density Function

Marginal Probability Mass Function

Market Model

Markov chain

Markov Property

Markov's Inequality

Martingale

Martingale Convergence Theorem

Martingale Equivalence for Stopping Times

Max Redundancy on KT

Maximal Likelihood Estimator

MCMC

Mean Squared Error Distortion

Memoryless Property 1

Metropolis Hastings Algorithm

Modification

Moment

Monte Carlo Method

Moving from small to petite sets

Multidimensional Itô Formula

Multivariate Gaussian

Mutual Information

Mutual Variation

Mutual Variation of Itô Stochastic Integrals

Nearest Neighbour Condition

Nearest Neighbour Vector Quantizer (NNVQ)

Negative Binomial RV

No Arbitrage

Non-Explosive

Nonsingular Code

Norm-like Function

Null Recurrence

Number of Types

Occupation Time

Operational Channel Capacity

Operational Channel Capacity (Continuous)

Optimal Bit Allocation

Optimal Compressor

Optimal Distortion

Optimal Power Allotment

Optimal Quantizer

Optimal Vector Quantizer

Orthogonal

Orthogonality Principle

Parametric Family

Passage Time

Penalty Lemma

Performance Analysis of Quantizers

Petite Set

Pointwise Convergence

Poisson Process

Poisson Random Variable

Portfolio

Portmanteau's Theorem

Positive Harris Recurrent

Positive Recurrent

Predictable Stochastic Intervals

Prediction Error Upper Bound

Predictive Quantization vs Scalar Quantization

Prefix Code

Prefix Code Redundancy

Prior

Probability Density Function

Probability Mass Function

Probability Measure

Probability of Decoding Error

Probability Simplex

Probability Space

Prokhorov's Theorem

Proposal Kernel

Q-Matrix

Quadratic Variation

Quadratic Variation is CAII

Quantizer Cell

R(D) is Convex in D

Random Time

Random Variable

Random Vector

Rate

Rate Distortion Function

Rate Distortion Theorem

Recurrent

Redundancy

Regular Quantizer

Regular Vector Quantizer

Relatively Sequentially Compact

Renyi Divergence

Renyi Entropy

Replicating Portfolio

Reproduction

Same Finite-Dimensional Distribution

Sample Space

Scalar Quantizer

Scheffé's Theorem

Self-Financing

Self-Information

Semimartingale

Semimartingale Integral

Semimartingale Properties

Set of Types with Denominator n

Shannon Coding Theorem

Shannon Limit

Shannon-Fano-Elias Code

Shannon's Channel Coding Theorem for AWGN

Shannon's Channel Coding Theorem for the DMC

Signal to Noise Ratio

Simplification of the Norm

Size of Type Class

Skohorod's Theorem

Solution to SDE

Source

Source Entropy

Source-Channel Fixed-Length Code

Standard Normal Random Variable

State

Stationary

Stationary

Stationary & Ergodic Source

Stationary Distribution

Stochastic Interval

Stochastic Kernel

Stochastic Process

Stochastic Realization

Stopped Process is also R.C. Martingale

Stopping Time

Stopping Time Integral

Submartingale

Summary of MATH 895

Supermartingale

Symmetric Channel

System SNR

Tail Field

Tangent Plane for LM

Theorems on Convergence

Tight

Transform Coding Distortion

Transform Coding with Scalar Quantization

Transient

Transition Kernel

Type

Type Class

Typical Set

Typical Set

Uniform Quantization of Real-Valued Source

Uniform Quantizer

Uniform Random Variable

Uniquely Decodable

Uniquely Ergodic

Uniqueness of invariant measure

Universal Code

Universal Code PMF Q

Upcrossings

Upper Bound on Channel Capacity

Variable Length Code

Variance

Vector Quantizer

Volume

Voronoi Regions

Wide Sense Stationary Process

WSS Predictor Coefficients

Zero Penalty iff Universal Code

σ(X)