FIND ME ON

GitHub

LinkedIn

Information Capacity Calculation

🌱

InfoTheory

Problem

Given a DMC (X,Y,Q=[PXY])(\mathcal{X},\mathcal{Y},Q=[P_{XY}]), determine the Information Capacity C=maxpXI(X;Y)C=\max_{p_{X}}I(X;Y)First note that necessary and sufficient conditions for finding CC can be shown since I(X;Y)=I(pX,pYX)I(X;Y)=I(p_{X},p_{Y|X}) is concave in pXp_{X}; but they might be hard to use as they require “guessing” a priori the optimal input distribution.

\therefore In general, CC does not admit a closed-form expression (in terms of channel parameters), unless if the channel exhibits certain “symmetry” properties.