FIND ME ON

GitHub

LinkedIn

Data Processing Inequality

🌱

Theorem
InfoTheory

If RVs, X,Y,ZX,Y,Z form a Markov chain, X→Y→ZX\to Y\to Z, then I(X;Y)≄I(X;Z)I(X;Y)\ge I(X;Z)

1. Equality holds iff I(X;Y∣Z)=0ā€…ā€ŠāŸŗā€…ā€ŠX→Z→YI(X;Y|Z)=0 \iff X\to Z\to Y 2. Can be similarly shown that I(Y;Z)≄I(X;Z)I(Y;Z)\ge I(X;Z) 3. Conditioning Reduces Mutual Information: I(X;Y)≄I(X;Y∣Z)ā€…ā€ŠāŸŗā€…ā€ŠX→Y→ZI(X;Y)\ge I(X;Y|Z) \iff X\to Y\to Z

Linked from