Given two pmfs $p$ and $q$ on the same alphabet $\mathscr{X}$, the divergence between $p$ and $q$, denoted by $D(p\|q)$ is given by: $$\begin{align*} D(p\|q)&=\sum_{a\in\mathscr{X}}p(a)\log_2\left(\frac{p(a)}{q(a)}\right)\\ &=E_p\left[\log_2\left(\frac{p(X)}{q(X)}\right)\right] \end{align*}$$ where $$\log_2\left(\frac{p(X)}{q(X)}\right)$$ is called the “log-likelihood ratio”.

Let $p_X$ be a pmf on $\mathscr{X}$ and let $p_{Y|X}$ and $q_{Y|X}$ be two different conditional pmfs on $\mathscr{Y}\times\mathscr{X}$. Then the conditional divergence between $p_{Y|X}$ and $q_{Y|X}$ given $p_X$ is $$\begin{align*}D(p_{Y|X}\|q_{Y|X}|p_X):&=E_{p_{Y|X}p_X}\left[\log_2\left(\frac{p_{Y|X}(Y|X)}{q_{Y|X}(Y|X)}\right)\right]\\&=\sum_{a\in\mathscr{X}}\sum_{b\in\mathscr{Y}}p_X(a)p_{Y|X}(b|a)\log_2\frac{p_{Y|X}(b|a)}{q_{Y|X}(b|a)}\\&=\sum_{a\in\mathscr{X}}p_X(a)\sum_{b\in\mathscr{Y}}p_{Y|X}(b|a)\log_2\frac{p_{Y|X}(b|a)}{q_{Y|X}(b|a)}\\&=E_{p_X}\left[D(p_{Y|X=X}\|q_{Y|X=X})\right]\end{align*}$$