Introduction

After investigating the Capacity of Parallel Gaussian Channels we would like to look at the problem of optimal power allotment $$P^{K}=(P_{1},\ldots,P_{K})$$subject to the constraint $$\sum\limits_{i=1}^{K}P_{i}\le P$$that maximizes $C(P)$ in $$\tag{$*$}C(P)=\sum\limits_{i=1}^{K} \frac{1}{2}\log\left(1+ \frac{P_{i}}{\sigma_{i}^{2}}\right)$$We first note that $\log_{2}\left(1+ \frac{P_{i}}{\sigma^{2}_{i}}\right)$ is increasing in $P_{i}$ hence $C(P)$ is maximized by setting $\sum\limits_{i=1}^{K}P_{i}=P$. So now we want to maximize $*$ over $P^{K}=(P_{1},\ldots,P_{K})$ subject to the constraints $$\tag{$**$}\sum\limits_{i=1}^{K}P_{i}=P \ \mbox{and} \ P_{i}\ge0, \ i=1,\ldots,K$$

Problem

Want to find $$\max_{P^{K}=(P_{1},\ldots,P_{K}):P_{i}\ge0,i=1,\ldots,K\mbox{ and }\sum\limits_{i=1}^{K}P_{i}=P} C(P)$$where $C(P)=\sum\limits_{i=1}^{K} \frac{1}{2}\log_{2}\left(1+ \frac{P_{i}}{\sigma_{i}^{2}}\right)$.

Solution

Using the Lagrange Multipliers Technique, set

• $n=K$ with $x^{n}=P^{n}$ (i.e. $x_{i}=P_{i}, \ i=1,\ldots,n$)
• $f(x^{n})=-C(P)=-\sum\limits_{i=1}^{n} \frac{1}{2}\log_{2}\left(1+ \frac{x_{i}}{\sigma_{i}^{2}}\right)$
• $g_{i}(x^{n})=-x_{i}\le0, \ i=1,\ldots,n$ (i.e. $m=n=K$)
• $h(x^{n})=\sum\limits_{i=1}^{n}x_{i}-P=0$ (i.e. $l=1$) After some math stuff we get the following solution $$C(P)=\sum\limits_{i=1}^{K} \frac{1}{2}\log_{2}\left(1+ \frac{(\Theta-\sigma_{i}^{2})^{+}}{\sigma_{i}^{2}} \right)$$where $(\Theta-\sigma_{i}^{2})^{+}=\max\{0,\Theta-\sigma_{i}^{2}\}$ (i.e. ReLU function) and $\Theta$ is chosen so that $\sum\limits_{i=1}^{K}(\Theta-\sigma_{i}^{2})^{+}=P$ where $P_{i}=(\Theta-\sigma_{i}^{2})^{+}$ i.e. the “Water-filling principle”.