FIND ME ON

GitHub

LinkedIn

Equivalent Properties in Discrete & Continuous IT

🌱

Theorem
InfoTheory

1. Conditioning Reduces Entropy

h(X)h(XY)h(X)\ge h(X|Y)with equality if X ⁣ ⁣ ⁣YX\perp \!\!\! \perp Y.

2. Data Processing Inequality

For continuous RVs X,Y,ZX,Y,Z such that XYZX\to Y\to Z, I(X;Y)I(X;Z)I(X;Y)\ge I(X;Z)