NAVIGATION

Home

Research

Bookshelf

Garden

FIND ME ON

GitHub

LinkedIn

Home

Research

Bookshelf

Garden

Discrete Memoryless Channel

🌱

Definition
InfoTheory

A discrete memoryless channel (DMC) is a discrete channel whose sequence of nn-dimensional transition distributions {PYn∣Xn}\{P_{Y^{n}|X^{n}}\} satisfies PYn∣Xn(bn∣an)=Πi=1∞PY∣X(bi∣ai)(**)\tag{**}P_{Y^{n}|X^{n}}(b^{n}|a^{n})=\Pi_{i=1}^{\infty}P_{Y|X}(b_{i}|a_{i})∀n≥1, an∈Xn, bn∈Yn\forall n\ge1, \ a^{n}\in\mathcal{X}^{n}, \ b^{n}\in \mathcal{Y}^{n} where PY∣XP_{Y|X} is a fixed (time-invariant) conditional distribution on X×Y\mathcal{X}\times\mathcal{Y}. In other words, a DMC is fully described by the triplet (X,Y,Q=[PXY])(\mathcal{X},\mathcal{Y},Q=[P_{XY}]) where QQ is called the channel’s transition matrix and PXY:=PY∣X(y∣x), x∈X, y∈YP_{XY}:=P_{Y|X}(y|x), \ x\in\mathcal{X}, \ y\in\mathcal{Y}.

- Matrix is row-stochastic (i.e. rows sum to one) - It can be verified that a DMC satisfies the consistency property

The DMC Property (∗∗)(**) is equivalent to these two conditions: 1. Output Memoryless Feature: PYn∣Xn,Yn−1(bn∣an,bn−1)=PY∣X(bn∣an)P_{Y_{n}|X^{n},Y^{n-1}}(b_{n}|a^{n},b^{n-1})=P_{Y|X}(b_{n}|a_{n}) ∀n≥1, an∈Xn, bn∈Yn\forall n\ge1, \ a^{n}\in\mathcal{X}^{n}, \ b^{n}\in \mathcal{Y}^{n}. i.e. The current output is conditionally independent of past output given the current input. 2. Non-Anticipatory Feature: PYn−1∣Xn(bn−1∣an)=PYn−1∣Xn−1(bn−1∣an−1)P_{Y^{n-1}|X^{n}}(b^{n-1}|a^{n})=P_{Y^{n-1}|X^{n-1}}(b^{n-1}|a^{n-1}) ∀n≥2, an∈Xn, bn−1∈Yn−1\forall n\ge2, \ a^{n}\in\mathcal{X}^{n}, \ b^{n-1}\in \mathcal{Y}^{n-1}. i.e. The current output is conditionally independent of future input given the current and past inputs.

Linked from

Binary Erasure Channel

Binary Symmetric Channel

Information Capacity

Operational Channel Capacity

Symmetric Channel

Lossless Joint Source-Channel Coding Theorem

Shannon's Channel Coding Theorem for the DMC

Lossy Source-Channel Coding Theorem

Information Capacity Calculation

Reinforcement Learning for Zero-Delay Coding