🌱
Given a DMC , its Information Capacity, , is defined as: where maximization is over all possible input distributions .
1. is well-defined since is a concave and continuous function of (by concavity of information measure) and the set of is compact (closed and bounded) in ; thus admits a maximum. Also since then 2. Will see via Shannon’s Channel Coding Theorem for the DMC that is the DMC’s “operational capacity”
Binary Erasure Channel
Binary Symmetric Channel
Binary Symmetric Erasure Channel
Symmetric Channel
Lossless Joint Source-Channel Coding Theorem
Shannon's Channel Coding Theorem for the DMC
Shannon Limit
Lossy Source-Channel Coding Theorem
Information Capacity with Input Cost
Capacity of Uncorrelated Parallel Gaussian Channels
Upper Bound on Channel Capacity
Information Capacity Calculation