Cross-Entropy is a quantity intimately connected to divergence: $$\begin{align*} D(p\|q)&=\sum_{a\in\mathscr{X}}p(a)\log_2\left(\frac{p(a)}{q(a)}\right)\\ &=\sum_{a\in\mathscr{X}}p(a)\log_2p(a)+\sum_{a\in\mathscr{X}}p(a)\log_2\frac{1}{q(a)}\\ &=-H(X)+H(p;q) \end{align*}$$ # Intuition Where divergence essentially tells us the difference in entropy between using $p$ to encode $p$ and using $q$ to encode $p$. Cross-Entropy tells us the total number of expected bits (or entropy) needed to when symbols are generated from $p$ but $q$ is used to encode them.