ℹ

Lagrange Multiplier

Theorem (Lagrange multiplier)

Assume f:Rk→Rf:\mathbb{R}^{k}\to \mathbb{R} and g:Rk→Rg:\mathbb{R}^{k}\to \mathbb{R} are continuously differentiable functions. We want to solve the following constrained minimization problem (CMP): minimize f(x)subject to g(x)=0\begin{align*} &\text{minimize }f(\mathbf{x})\\ &\text{subject to }g(\mathbf{x})=0 \end{align*}Let ∇u=(∂∂x1u…,∂∂xku)\nabla u=\left( \frac{ \partial }{ \partial x_{1} }{u}\dots,\frac{ \partial }{ \partial x_{k} } u \right) denote the gradient of a u:Rk→Ru:\mathbb{R}^{k}\to \mathbb{R}. We say now if x∗\mathbf{x}^{*} is a solution of the CMP and ∇g(x∗)≠0\nabla g(\mathbf{x}^{*})\not=0, then ∃λ∈R∗\exists\lambda\in\mathbb{R}^{*} called the Lagrange multiplier such that ∇f(x∗)+λ∇g(x∗)=0\nabla f(\mathbf{x}^{*})+\lambda \nabla g(\mathbf{x}^{*})=0

Remark

The theorem yields the following equations: ∂∂xj(f(x)+λg(x))∣x=x∗=0g(x∗)=0 \begin{align*} \left.\frac{ \partial }{ \partial x_{j} } (f(\mathbf{x})+\lambda g(\mathbf{x}))\right|_{\mathbf{x}=\mathbf{x}^{*}}&=0\\ g(\mathbf{x}^{*})&=0 \end{align*} this gives us k+1k+1 equations for k+1k+1 unknowns.

Lemma (Tangent Plane for LM)

If ∇g(x∗)≠0\nabla g(\mathbf{x}^{*})\not=\mathbf{0}, then the tangent plane of SS at x∗\mathbf{x}^{*} is given by T={y:∇g(x∗)y=0}T=\{ \mathbf{y}:\nabla g(\mathbf{x}^{*})\mathbf{y}=0 \}