Definition (Renyi Divergence)
Given two pmfs p and q with common support X, and parameter α>0, α=1, the Renyi divergence of parameter α between p and q is: Dα:=α−11log2(a∈X∑pα(a)q1−α(a))
Lemma (Renyi’s Divergence reduces to Divergence)
α→1limDα(p∥q)=D(p∥q)