NAVIGATION

Home

Research

Bookshelf

Garden

FIND ME ON

GitHub

LinkedIn

Home

Research

Bookshelf

Garden

Stationary & Ergodic Source

šŸŒ±

Definition
InfoTheory

A stationary ergodic source is a generalization of a DMS such that the WLLN holds which consequently provides a generalization of the AEP such that for a stationary ergodic source {Xi}i=1āˆž\{X_i\}_{i=1}^\infty with pmf pXp_X and alphabet X\mathcal{X} we have āˆ’1nlogā”2(pXn(Xn))ā†’nā†’āˆžH(X)\mboxinprobability-\frac{1}{n}\log_2(p_{X^n}(X^n))\xrightarrow{n\to\infty}H(\mathcal{X})\mbox{ in probability}

For a stationary and ergodic source 1. Its sample averages converge to a constant given by the expected value almost surely 1nāˆ‘i=1nf(Xi)ā†’nā†’āˆžE[f(X1)]Ā Ā a.s.\frac{1}{n}\sum\limits_{i=1}^{n}f(X_{i})\xrightarrow{n\to\infty}E[f(X_{1})] \ \ a.s.. 2. This shows that the WLLN holds for stationary ergodic sources. 3. A generalized AEP then holds for such sources with the source entropy replaced with entropy rate.

Given a stationary ergodic source then the entropy rate is as follows HD(X)=infā”{R:Ā R\mboxachievable}H_{D}(\mathcal{X})=\inf\{R: \ R\mbox{ achievable}\}

- A DMS is stationary ergodic - An irreducible stationary MC is ergodic

Linked from

Redundancy

Kac's Lemma (Stationary Ergodic)

Lempel-Ziv Coding

Lossless Joint Source-Channel Coding Theorem

Stationary & Ergodic Source

Asymptotic Equipartition Property