"Nice" Topological Spaces

(A) Ring Generated by R

(Cumulative) Distribution Function

(Ɛ) Simple Predicable Processes

(n-μ)-small

(P) Predictable σ-algebra

(R) Set of Predictable Rectangles

(Λ) Set of Predictable Locally Integrable Processes

1-Norm

2-Norm

A Summary of MATH 891

Abelian Group

Absolute Continuity

Absolute Continuity of Lebesgue Integral

Acceptance Rate

Accessible

Achievability of the Rate Distortion Function

Achievable

Achievable (Continuous)

Adapted

Additive

Adjoint

Admissible

Admissible Control

Admissible Policy

AEP For Continuous IID RVs

Affine

Affine Control System

Algebra

All Bases have same size

All Numbers Have Prime Divisors

All Numbers have Prime Factorization

Almost Everywhere

Almost Sure Convergence

Almost surely

Aperiodic

Arbitrary assignment of Eigenvalues

Arithmetic Function

Arithmetic-Geometric Mean Inequality

Arzelà-Ascoli

Asian Call Option

Associate

Asymptotic Equipartition Property

Atlas

Atom

Atomic Measure

Average Code Rate

Average Cost Constraint

Average Infinite Horizon Optimization

Average Power Constraint

Average Probability of Error (Channel Code)

Average Probability of Error (Source-Channel Code)

Axioms of Probability

b-ary Representation Theorem

Banach Space

Banach's Fixed Point

Barrier Certificate

Basic Cell

Basis

Bayes' Rule

Bayesian Statistics

Belief MDP

Bellman's Optimality Principle

Beppo Levi Theorem

Bernoulli Trial

Bichteler-Dellacherie Theorem

Bijective

Binary Erasure Channel

Binary Function (f_b)

Binary Symmetric Channel

Binary Symmetric Erasure Channel

Binomial Random Variable

Binomial Theorem

Bit Allocation Problem

Blackwell's Irrelevant Information Theorem

Block Codes for the Gaussian Channel

Borel function

Borel measure

Borel space

Borel subset

Borel σ-algebra

Borel-Cantelli Lemma

Bounded Function

Bounded-Lipschitz

Brownian Motion

Bucklew and Wise

Càdlàg

Capacity of Correlated Parallel Gaussian Channels

Capacity of Uncorrelated Parallel Gaussian Channels

Capacity with Decoder-Side Information (DSI)

Capacity with Full-Side Information (FSI)

Carathéodory Theorem

Carmichael Number

Cauchy Sequence

Cauchy-Schwartz Inequality

Cauchy-Schwarz Inequality

Central Limit Theorem

Centroid Condition

Centroid Condition for MSE

Cesaro-Mean Theorem

Chain rule (derivative)

Change of Variable Formula

Channel Coding Operation

Chapman Kolmogorov Equation

Characteristic

Characteristic of F

Chebyshev Inequality

Chinese Remainder Theorem

Class Κ

Closed

Closed-loop Prediction Gain

Closed-loop Predictive Quantization

Closed-loop Predictor Coefficients

Closure

Closure (Algebra)

Closure of Measurability

Code Rate

Codebook

Codeword

Coding Gain

Coin Tossing Probability Space

Communication

Compact

Companding Quantization

Compatible

Complement of Subspace Generates Direct Sum

Complementary Subspaces

Complete

Complete Measure Space

Completion

Completion of Squares

Complex Conjugate

Composite Number

Composition

Composition of Measurable Functions

Composition Rules for Matrices

Concave Function

Concavity of Channel Capacity in P

Concentrated

Conditional Expectation

Conditional Independence

Conditional Probability Density Function

Conditional Probability Mass Function

Conditional Probability of Events

Conditional Variance

Conditions for Diagonalizability

Conditions for PBP Optimality to be Global

Congruence

Conjugate Exponents

Connected

Connected component

Consequence of the AEP

Construction of Invariant measure

Construction of Outer Measure from Pre-measure

Continuity of Probability

Continuous

Continuous

Continuous and Compactly supported functions dense in Lp

Continuous Martingale with Finite Variation is Constant

Continuous Memoryless Source

Continuous Time Markov Chain

Contraction Mapping

Control Barrier Function

Controllability & Observability w.r.t. Riccati

Controllability Canonical Form

Controllability Gramian

Controllability Matrix

Controllability Normal Form

Controllable

Controlled Markov Chain

Convergence

Convergence

Convergence in Distribution

Convergence in Expectation

Convergence to Equilibrium

Convex

Convex Polytope

Convex Set

Convexity implies continuity

Convexity or Concavity of Information Measure

Convolution

Coprime

Coset

Countable Additivity is Necessary for MCT

Countable Sub-additivity of Events

Counting measure

Covariance

Covariance Matrix

Criterion for a Basis

Criterion for Direct Sum

Criterion for Invertibility using Upper Triangular

Criterion for Measurability

Criterion for Subspace

Criterion for Upper Triangular Matrix

Cross Variation

Cross-Entropy

Cyclic Group

Cyclic Subgroup

Darboux Sums

Data Processing Inequality

De Finetti's Theorem

Decreasing Events

Degree is Closed under Addition

Dense

Density of Ɛ

Dependence Lemma

Derivative

Design of Observer-based controllers

Detailed Balance

Detectable

Determinants for Linearly Transformed Autocorrelation Matrices

Diagonal

Diagonalizable

Diffeomorphism

Difference Quantization

Differential Cross-Entropy

Differential Divergence

Differential Entropy

Differentiation

Dimensionality & Linear Maps

Dirac Distribution

Dirac Measure

Direct Sum

Dirichlet Mixture Distribution

Discounted Cost

Discounted Infinite Horizon Optimization

Discrete Communication Channel

Discrete Memoryless Channel

Discrete Memoryless Source

Discrete topology

Discrete-Time Continuous Memoryless Channels

Discrete-Time Memoryless Gaussian Channel (AWGN Channel)

Disintegration

Distinct eigenvalues have linearly independent eigenvectors

Distortion

Distortion Measure

Distortion Rate Function

Distribution

Divergence

Divergence Bound on KT

Division Algorithm for Polynomials

Division of Polynomials

Dobrushin's Ergodic Coefficient

Doléans Measure

Dominated Convergence Theorem

Doob's Forward Convergence Theorem

Doob's Maximal Inequalities

Doob's Optional Sampling Theorem

Doob's Upcrossing Inequality

Double Complement Returns the Original Subspace

Driftless Control System

Dynamic

Dynamic Programming

Eigendecomposition of a Matrix

Eigenvalue assignment for LTIC systems

Eigenvalues are Diagonal Elements of Upper Triangular

Eigenvector

Elementary Set

Encoder Decoder Structure of Quantization

Enough Diagonal Elements Imply Diagonalizability

Entropy

Entropy Rate

Equicontinuous

Equivalence between Density and Radon-Nikodym Derivative

Equivalence Relation

Equivalent Bounds

Equivalent Properties in Discrete & Continuous IT

Equivalent Realization

Ergodic

Ergodic Theorem

Essential bound

Estimation Error and differential entropy

Euclidean Algorithm

Euclidean topology

Euler's Criterion

Euler's Theorem

Euler's Totient Function in Terms of Mobius Function

Euler's Totient Function ϕ

European Call Option

European Put Option

Event Space

Every Measure Space has a Completion

Every Metric Space is a Topological Space

Every Normed Vector Space and Inner Product Space is a Metric Space

Every Spanning Set Contains a Basis

Exchangeable

Excursion Time

Existence for N-agent Team

Existence of Càdlàg Version

Existence of Companding Realization

Existence of Eigenvalues on Complex Spaces

Existence of Invariant measure

Existence of Sequences of Independent rvs

Existence of Smallest σ-algebra

Existence of Uniform Measure

Existence of Upper Triangular Matrices on Complex Spaces

Expectation

Expectation of a Function of a Random Variable

Exponential Random Variable

Extended Kalman Filter

Extended Real Line

Extension Theorem

Fading Channel

Fatou's Lemma

Feller condition for invariant measure

Feller Property

Fermat's Little Theorem

Field

Filter Stability

Filtration

Finite Basis

Finite Dimensional

Finite Field

Finite Field of Order p

Finite Fields of Same Cardinality = Isomorphic

Finite Horizon Optimization

Finite Subgroups are Cyclic

Finite Variance = Autocorrelation symmetric + positive semidefinite

Finite-Horizon Optimal Control Problem

FIR Filter

First-countable

Fixed endpoint problem

Fixed-Length (Block) Code

Fixed-Length Codes for Discrete Channels

Forward Contract

Forward Invariant

Foster-Lyapunov Theorems

Four Hypotheses

Free endpoint problem

Fubini Theorem (For indicator functions)

Fubini-Tonelli

Function of real measurable functions is measurable

Functional

Fundamental Inequality

Fundamental Theorem of Arithmetic

Fσ set

Gain of Transform Coding

Gamma Random Variable

Gauss Sum

Gauss's Formula

Gaussian Noise Minimizes Capacity of Additive-Noise Channel

Gaussian Process

Gaussian Random Variable

Gaussian Rate Distortion Function

Gaussian Source Maximizes Rate Distortion Function

Generalized Chinese Remainder Theorem

Generator

Geometric Random Variable

Global Asymptotic Stability

Greatest Common Divisor

Group

Gδ set

h-interval

Hadamard's Inequality

Hahn-Jordan Theorem

Hamilton-Jacobi-Bellman Equation

Hamming Distortion

Hamming Rate Distortion Function

Harris Recurrent

Hausdorff

Heine-Borel Theorem

High Resolution Optimality of KLT

High-Resolution Conditions

High-Resolution Conditions LVQ

High-Resolution Optimal Vector Quantizer

Hilbert Space

Hitting Time

Hölder's Inequality

Holding Time

Homeomorphism

Homomorphism

Homotopy

Hopf's Extension Theorem

Huffman Lemma

Hurwitz Matrix

Identity Map

If a Prime divides ab it divides one of a or b

If an Irreducible Polynomial divides ab then it divides a or b

iid

Image

In Probability Convergence

Increasing Events

Independence of Bounded RV

Independent

Index

Indistinguishable

Indistinguishable

Induced Topology

Infimum

Infinity Norm

Information Capacity

Information Capacity Calculation

Information Capacity with Input Cost

Information Function

Information Signal

Information Structure

Injective

Injectivity, surjectivity, and isomorphism are equivalent when Dimension is the Same

Inner Product

Inner Product Space

Integrable

Integrable = Vector Space

Integrals of Functions that are Zero a.e.

Interior Point

Intersection of Sets

Intro to Safe Control

Invariant

Invariant Distribution

Invariant Distribution ↔ Positive Recurrence

Invariant Distribution for CTMC

Invariant Measure

Invariant probability measure

Invariant Set

Invariant Subspace

Inverse Property of Matrix of Linear Map

Inverse to a Linear Map is Unique

Inversion Method

Invertibility

Ionescu Tulcea Theorem

Irreducible

Irreducible (Polynomial)

Irreducible Factorization

Isometry

Isomorphic Vector Spaces have the Same Dimension

Isomorphism

Isomorphism is a Bijection

Itô Isometry

Itô Isometry v2

Itô Isometry v3

Itô Representation Theorem

Itô Stochastic Integral

Itô Stochastic Integral is a Local Martingale

Itô Stochastic Integral on Ɛ is a R.C. Martingale

Itô Stochastic Integral on Λ is a R.C. Martingale

Itô's Formula

Jacobi Prop 1

Jacobi Prop 2

Jacobi Symbol

Jensen-Shannon Divergence

Jensen's Inequality

Jensen's Inequality

Joint AEP

Joint Differential Entropy of Multivariate Gaussian

Joint Distribution Function

Joint Entropy

Joint Markov Chain

Joint Probability Density Function

Joint Probability Mass Function

Jordan Canonical Form

Jump Chain

Jump Time

Kac's Lemma

Kac's Lemma (Stationary Ergodic)

Kalman Filter

Karhunen-Loeve Transform

Karush-Kuhn-Tucker Conditions (KKT)

Kernel

Kernel and Image are Subspaces

KL Transform Decorrelates X

Kolmogorov 0-1 Law

Kolmogorov Extension Theorem

Kraft Inequality

Kraft Inequality for UD VLCs

Krichevsky and Trofimov Coding Distribution

L infinity

Lagrange Corollary

Lagrange Lemma

Lagrange Multiplier

Lagrange Multipliers Technique

Lagrange Theorem

Laplace Transform

Lattice

Lattice Vector Quantizer

Law

Law of Large Numbers

Law of Total Probability

lcm Order Lemma

Least Common Multiple

Lebesgue Integral

Lebesgue Integral is a Measure

Lebesgue Integral Triangle Inequality

Lebesgue Measurable

Lebesgue Measurable σ-algebra

Lebesgue Measure

Lebesgue Outer Measure

Lebesgue Point

Lebesgue σ-algebra

Lebesgue-Stieltjes Integral

Lebesgue-Stieltjes Measure

Left Continuous Adapted = Predictable

Legendre Prop 3

Legendre Symbol

Legendre Symbol Properties

Lemma 3.1

Lemma 3.2

Lempel-Ziv Coding

Level Set

Lévy's Convergence Theorems

Lie Group

Likelihood

Limit Inferior

Limit of Measurable Functions is Measurable

Limit Superior

Limits of Events

Lindëlof Space

Linear Combination

Linear Independence

Linear Independent Sets are Smaller than Spanning Sets

Linear Map

Linear Map is Injective iff Kernel is 0

Linear maps are defined on a basis

Linear Maps are Isomorphic to their Matrices

Linear Operator

Linear Prediction

Linear Quadratic Problem

Linear Time Invariant Control System

Linear Time Varying Control System

Linear Transform for Autocorrelation Matrices

Linearity Properties of Matrices

Linearization of Nonlinear Control System

Linearly Independent Sets Generate Bases

Lipschitz Continuous

Lloyd-Max Algorithm

Lloyd-Max Quantizer

Local Martingale

Locally Compact

Locally constant

Locally Euclidean Space

Log-likelihood

Log-Sum Inequality

Loop

Lossless Joint Source-Channel Coding Theorem

Lossy Source-Channel Coding Theorem

Lp Convergence

Lp is Banach

Lp Martingale

Lp Norm

Lp Space

Luenberger Observer

Marginal Probability Density Function

Marginal Probability Mass Function

Market Model

Markov chain

Markov Decision Process

Markov Policy

Markov Policy induces Markov Chain

Markov Policy is Good Enough

Markov Property

Markov's Inequality

Martingale

Martingale Convergence Theorem

Martingale Equivalence for Stopping Times

Matrix of Linear Map

Max Redundancy on KT

Maximal Likelihood Estimator

MCMC

Mean Squared Error Distortion

Measurability Criterion for Topological Codomain

Measurability of Continuous Functions

Measurable Function

Measurable Rectangle

Measurable Selection Conditions

Measurable Space

Measure

Measure Space

Memoryless Property 1

Mesh

Metric

Metric Space

Metric Topology

Metrizable

Metropolis Hastings Algorithm

Min poly. dividing poly. of same root

Minimal

Minimal Polynomial

Minkowski's Inequality

Möbius Function

Möbius Inversion Formula

Modification

Moment

Monic

Monotone Class

Monotone Convergence Theorem

Monte Carlo Method

Moving from small to petite sets

Multidimensional Itô Formula

Multiplicative

Multivariate Gaussian

Mutual Information

Mutual Variation

Mutual Variation of Itô Stochastic Integrals

Mutually Singular

Nash Equilibrium

Near Optimal

Nearest Neighbour Condition

Nearest Neighbour Vector Quantizer (NNVQ)

Necessary

Negative Binomial RV

Neighbourhood

No Arbitrage

Non-Explosive

Nonsingular Code

Nonzero Constant Polynomials

Norm

Norm-like Function

Norm-Preserving Matrix

Normed Vector Space

Notions of Compactness

Null Recurrence

Number of Types

Observability Gramian

Observability Matrix

Observability Normal Form

Observability Problem

Observable

Occupation Time

Open

Operational Channel Capacity

Operational Channel Capacity (Continuous)

Operator Norm

Optimal

Optimal Bit Allocation

Optimal Compressor

Optimal Distortion

Optimal Power Allotment

Optimal Quantizer

Optimal Vector Quantizer

Optimality of Q Learning Algorithm

Order

Orthogonal

Orthogonal

Orthogonal Complement

Orthogonal Decomposition

Orthogonal Group

Orthogonal Matrices have Determinant 1

Orthogonal Matrix

Orthogonal Vector

Orthogonality Principle

Orthonormal

Orthonormal Matrix

Outer Measure

Outer Measure

p-adic Valuation (v_p)

Paracompact

Parallelogram Equality

Parametric Family

Parity

Partially Observable Markov Decision Process

Passage Time

Path connected

Penalty Lemma

Perfect Square

Performance Analysis of Quantizers

Petite Set

Picard-Lindelöf Theorem

Pointwise Convergence

Pointwise Convergence

Poisson Process

Poisson Random Variable

Policy

Policy Relationships

Polish space

Polynomials

Portfolio

Portmanteau's Theorem

Positive Definite

Positive Definite = Positive Eigenvalues

Positive Divisor Counter Function (d)

Positive Harris Recurrent

Positive Recurrent

Positive Semidefinite

Positive Semidefinite has dim Eigenvectors

Pre-measure

Precompact

Predictable Stochastic Intervals

Prediction Error Upper Bound

Predictive Quantization vs Scalar Quantization

Prefix Code

Prefix Code Redundancy

Preimage

Prime Number

Primitive

Primitive Root

Principle of Uniform Boundedness

Prior

Probabilistic Safety

Probability Density Function

Probability Mass Function

Probability Measure

Probability of Decoding Error

Probability Simplex

Probability Space

Product Measure

Product σ-algebra

Product σ-algebra is the smallest monotone class containing collection of elementary sets

Prokhorov's Theorem

Proper

Proper Divisor

Proper Function

Properties of LGQT Cost

Properties of Limit Inferior & Limit Superior

Properties of Linear Maps

Properties of Measures

Properties of Orthogonal Complements

Proposal Kernel

Pseudoprime

Pushforward Measure

Pythagorean Triple

Q-Learning

Q-Matrix

Quadratic Gaussian Team

Quadratic Reciprocity of Jacobi

Quadratic Reciprocity of Legendre

Quadratic Residue

Quadratic Variation

Quadratic Variation is CAII

Quantizer Cell

R(D) is Convex in D

Radner Krainak Theorem

Radon-Nikodym Derivative

Radon-Nikodym Theorem

Random Time

Random Variable

Random Vector

Rank

Rank Nullity Theorem

Rate

Rate Distortion Function

Rate Distortion Theorem

Rational Function

Realizable

Realization

Realization Problem

Recurrent

Redundancy

Regular Quantizer

Regular Vector Quantizer

Relatively Compact

Relatively Prime

Relatively Sequentially Compact

Renyi Divergence

Renyi Entropy

Replicating Portfolio

Reproduction

Riccati Equation

Riemann Integral

Riemann integral existence

Riemann-Stieltjes Integral

Right Continuous

Ring

Ring of Polynomials of Order p

Ring of Sets

Root

Root of Unity

S is dense in Lp

Safe Set

Same Finite-Dimensional Distribution

Sample Space

Scalar Quantizer

Scheffé's Theorem

Second-countable

Self-Financing

Self-Information

Semialgebra

Semicontinuous

Semigroup

Semimartingale

Semimartingale Integral

Semimartingale Properties

Semiring

Separable

Sequential

Sequential Covering Class

Sequentially Compact

Set of all Integrable Functions

Set of Linear Maps

Set of Types with Denominator n

Shannon Coding Theorem

Shannon Limit

Shannon-Fano-Elias Code

Shannon's Channel Coding Theorem for AWGN

Shannon's Channel Coding Theorem for the DMC

Shrink nicely

Signal to Noise Ratio

Signed Measure

Simple Function

Simplification of the Norm

Simply connected

Singular

Size of Type Class

Skohorod's Theorem

Slices of P stay in their respective σ-algebras

Slices of product measurable function are in measurable in resultant σ-algebras

Smallest σ-algebra

Smooth

Smooth Manifold

Solution to Congruence

Solution to SDE

Source

Source Entropy

Source-Channel Fixed-Length Code

Span

Span of Vectors is a Subspace of the Vector Space

Special Orthogonal Group

Square of Gauss Sum

Squarefree Number

Stabilizable

Standard Normal Random Variable

Standard Quadratic Team

Standard topology

State

State Observer Problem

Static

Static Quadratic Team

Stationary

Stationary

Stationary & Ergodic Source

Stationary Conditions for Team Optimality

Stationary Distribution

Stationary Policy

Stationary Radner Krainak Theorem

Stationary Team Policy

Stochastic Interval

Stochastic Kernel

Stochastic Process

Stochastic Realization

Stopped Process is also R.C. Martingale

Stopping Time

Stopping Time Integral

Strictly Causal

Submartingale

Subspace

Sufficient

Sum of Subsets

Sum of Subspaces is a Subspace

Sum of Subspaces is Smallest Subspace Containing their Union

Summary of MATH 895

Supermartingale

Support

Supremum

Supremum & Infimum Preserve Measurability

Surjective

Symmetric Channel

System SNR

Tail Field

Tangent Plane for LM

Taylor Expansion

Taylor Series

Team

Team-Optimal Solution

The Orthogonal Complement is a Complementary Subspace

The Set of Residue Classes mod p

Theorem 1.1

Theorem 1.16

Theorem 1.17

Theorem 1.18

Theorem 1.19

Theorem 1.22

Theorem 1.6

Theorem 3.1

Theorems on Convergence

Tight

Time Consistent

Topological Manifold

Topological Space

Topology

Total Variation

Total Variation Measure

Totally Bounded

Trace

Trace and Determinant with Eigenvalues

Transfer Function

Transform Coding Distortion

Transform Coding with Scalar Quantization

Transient

Transition Kernel

Transition Matrix

Trivial topology

Type

Type Class

Typical Set

Typical Set

Uniform Integrability

Uniform Quantization of Real-Valued Source

Uniform Quantizer

Uniform Random Variable

Uniformly Bounded

Union of Sets

Unique Factorization Theorem for Polynomials

Uniquely Decodable

Uniquely Ergodic

Uniqueness for N-agent Team

Uniqueness of GCD

Uniqueness of invariant measure

Unit

Universal Code

Universal Code PMF Q

Upcrossings

Upper Bound on Channel Capacity

Upper-Triangular

Value Function

Value Iteration Algorithm

Variable Length Code

Variance

Variation

Vector Quantizer

Vector Space

Vee

Volume

Voronoi Regions

Wasserstein Convergence

Wasserstein distance metrizes Wasserstein space

Wasserstein metric

Wasserstein space

Weak convergence

Weak convergence in Wasserstein space

Wedge

Weierstrass Theorem

Weighting Pattern

Wide Sense Stationary Process

Wilson's Theorem

Witsenhausen's Intrinsic Model

Writing Primitive Pythagorean Triples

WSS Predictor Coefficients

Zero Penalty iff Universal Code

ε-Optimal

σ-algebra

σ-algebra containing all Borel subsets

σ-algebra of subsets of

σ-compact

σ-finite

σ(X)