NAVIGATION
Home
Research
Bookshelf
Garden
FIND ME ON
GitHub
LinkedIn
š±
Xā¼fX,Yā¼fYX\sim f_{X},Y\sim f_{Y}Xā¼fXā,Yā¼fYā with SXāSYāRS_{X}\subset S_{Y}\subset \mathbb{R}SXāāSYāāR, the differential cross-entropy between fXf_{X}fXā and fYf_{Y}fYā is h(fX;fY)=ā«SXfX(t)logā”21fY(t)dth(f_{X};f_{Y})=\int_{S_{X}}f_{X}(t)\log_{2} \frac{1}{f_{Y}(t)}dth(fXā;fYā)=ā«SXāāfXā(t)log2āfYā(t)1ādt
D(fXā„fY)=āh(X)+h(fX;fY)ā„0D(f_{X}\|f_{Y})=-h(X)+h(f_{X};f_{Y})\ge0D(fXāā„fYā)=āh(X)+h(fXā;fYā)ā„0ā“\thereforeā“ h(fX;fY)ā„h(X)h(f_{X};f_{Y})\ge h(X)h(fXā;fYā)ā„h(X)