NAVIGATION
Home
Research
Bookshelf
Garden
FIND ME ON
GitHub
LinkedIn
🌱
Given parameter α>0, α≠1\alpha>0, \ \alpha\not=1α>0, α=1, a RV X∼pXX\sim p_XX∼pX, then the Renyi entropy with parameter α\alphaα is Hα(X)=11−αlog2(∑a∈XpX(a)α)=α1−αlog2(∥pX∥α)\begin{align*} H_\alpha(X)&=\frac{1}{1-\alpha}\log_2\left(\sum_{a\in\mathscr{X}}p_X(a)^\alpha\right)\\ &=\frac{\alpha}{1-\alpha}\log_2\left(\|p_X\|_\alpha\right) \end{align*} Hα(X)=1−α1log2(a∈X∑pX(a)α)=1−ααlog2(∥pX∥α) where ∥pX∥α\|p_X\|_\alpha∥pX∥α is the “α\alphaα-norm of pXp_XpX” or ∥pX∥α=[∑a∈XpX(a)α]1α\|p_X\|_\alpha=\left[\sum_{a\in\mathscr{X}}p_X(a)^\alpha\right]^{\frac{1}{\alpha}}∥pX∥α=[a∈X∑pX(a)α]α1
limα→1Hα(X)=H(X)\lim_{\alpha\to1}H_\alpha(X)=H(X)α→1limHα(X)=H(X)
Renyi Divergence