NAVIGATION
Home
Research
Bookshelf
Garden
FIND ME ON
GitHub
LinkedIn
Email
Definition (Differential cross-entropy)
X∼fX,Y∼fYX\sim f_{X},Y\sim f_{Y}X∼fX,Y∼fY with SX⊂SY⊂RS_{X}\subset S_{Y}\subset \mathbb{R}SX⊂SY⊂R, the differential cross-entropy between fXf_{X}fX and fYf_{Y}fY is h(fX;fY)=∫SXfX(t)log21fY(t)dth(f_{X};f_{Y})=\int_{S_{X}}f_{X}(t)\log_{2} \frac{1}{f_{Y}(t)}dth(fX;fY)=∫SXfX(t)log2fY(t)1dt
Remark
D(fX∥fY)=−h(X)+h(fX;fY)≥0D(f_{X}\|f_{Y})=-h(X)+h(f_{X};f_{Y})\ge0D(fX∥fY)=−h(X)+h(fX;fY)≥0∴\therefore∴ h(fX;fY)≥h(X)h(f_{X};f_{Y})\ge h(X)h(fX;fY)≥h(X)