Theorem (Convexity or Concavity of Information Measure)

1. The divergence, $D(p\|q)$ is convex in the pair $(p,q)$ i.e. if $(p_1,q_1$ and $(p_2,q_2)$ are two pairs of pmfs defined on $\mathcal{X}$ then $$D(\lambda p_1+(1-\lambda)p_2\|\lambda q_1+(1-\lambda)q_2)\le\lambda D(p_1\|q_1)+(1-\lambda)D(p_2\|q_2)$$ $\forall\lambda\in[0,1]$.
2. If $X\sim p_X$, then $H(X)=H(p_X)$ is concave in
3. If $(X,Y)\sim p_Xp_{Y|X}$, then $I(X;Y)=I(p_X,p_{Y|X})$ is concave in $p_X$ for fixed $p_{Y|X}$ and convex in $p_{Y|X}$ for fixed $p_X$