Method of Lagrange multipliers

We have studied how to identify extreme values of two-variable functions $f(x,y)$. In practice, we may have additional constraints. For example,

Find the extreme values of $f(x,y)$ subject to the constraint $g(x,y)=0$.

We can use the method of Lagrange multipliers to solve this type of problem.

Let's consider the following example.

Problem. If $(x,y)$ moves on the unit circle $x^2+y^2=1$, find the extreme values of $f(x,y)=x^2+xy+y^2$. □

This problem can be restated as follows: 

Problem (restated). Let $g(x,y)=x^2+y^2-1$. Find the extreme values of $f(x,y)=x^2+xy+y^2$ subject to the constraint $g(x,y)=0$. □

Let's solve this problem using an "explicit" method.

Solution 1 (Explicit method). Consider the implicit function $y=\sqrt{1-x^2}$ of $g(x,y)$ on the open interval $(-1,1)$ and substitute it to $f(x,y)$ to have

$$h(x)=f(x,y(x))=x^2+x\sqrt{1-x^2}+(\sqrt{1-x^2}{)}^2=x\sqrt{1-x^2}+1.$$

Since

$$h^{\prime }(x)=\frac{1-2x^2}{\sqrt{1-x^2}},$$

$h(x)$ has a local minimum at $x=-\frac{1}{\sqrt{2}}$ and a local maximum at $x=\frac{1}{\sqrt{2}}$. Therefore, $f(x,y)$ has a local minimum value $\frac{1}{2}$ at point $(-\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}})$, and a local maximum value $\frac{3}{2}$ at point $(\frac{1}{\sqrt{2}},\frac{1}{\sqrt{2}})$ on the unit circle.

Similarly, using the implicit function $y=-\sqrt{1-x^2}$ on the open interval $(-1,1)$, we can define $h(x)=-x\sqrt{1-x^2}+1$ and find that $f(x,y)$ has a local minimum value $\frac{1}{2}$ at $(\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}})$ and a local maximum value $\frac{3}{2}$ at $(-\frac{1}{\sqrt{2}},-\frac{1}{\sqrt{2}})$.

Next, we examine the neighbors of the points $(1,0)$ and $(-1,0)$. In these cases, we consider the implicit functions $x=\sqrt{1-y^2}$ (the branch passing through $(1,0)$) and $x=-\sqrt{1-y^2}$ (the branch passing through $(-1,0)$), and find the same points as above to give local minimum and maximum.

In summary, $f(x,y)$ has a local minimum value $\frac{1}{2}$ at $(x,y)=(\pm \frac{1}{\sqrt{2}},\mp \frac{1}{\sqrt{2}})$, and a local maximum value $\frac{3}{2}$ at $(x,y)=(\pm \frac{1}{\sqrt{2}},\pm \frac{1}{\sqrt{2}})$. □

The above solution is explicit and easy to understand. However, this method applies only when the implicit functions of $g(x,y)=0$ can be obtained explicitly. If that is not the case, the following method of Lagrange (undetermined) multipliers provides an effective approach. We first show how to solve the above problem using the method of Lagrange multipliers. We will prove the method in general later.

Solution 2 (Lagrange multiplier). First, let us define a new function

$$F(x,y,\lambda )=f(x,y)-\lambda g(x,y)=(x^2+xy+y^2)-\lambda (x^2+y^2-1)$$

where $\lambda$ is a new variable (undetermined constant) called the Lagrange multiplier. Note that we have $F(x,y)=f(x,y)$ if the constraint $g(x,y)=0$ is satisfied. We now consider the extreme value problem of this function.

Next, we differentiate $F(x,y)$ with respect to $x$ and $y$ and solve

$$\begin{array}{}\text{(Eq:lagx)}& F_x(x,y,\lambda )& =& 2(1-\lambda )x+y=0,\text{(Eq:lagy)}& F_y(x,y,\lambda )& =& x+2(1-\lambda )y=0.\end{array}$$

Furthermore,

$$\begin{array}{}\text{(Eq:lambda)}& F_{\lambda }(x,y,\lambda )=-(x^2+y^2-1)=0\end{array}$$

should also be satisfied. The latter equation implies, in particular, that $(x,y)\ne (0,0)$ (i.e., $x$ and $y$ cannot be simultaneously equal to 0). However, from (Eq:lagx) and (Eq:lagy) above, if $x=0$, then $y=0$ and vice versa, which is a contradiction. Thus, both $x$ and $y$ are non-zero. Eliminating, for example, $y$ from (Eq:lagx) and (Eq:lagy) and some rearrangement gives

$$x(2\lambda -1)(2\lambda -3)=0.$$

Since $x\ne 0$, we have $\lambda =\frac{1}{2},\frac{3}{2}$. For each of these values of $\lambda$, we solve the simultaneous equations (Eq:lagx) and (Eq:lagy) for $x$ and $y$:

• For $\lambda =\frac{1}{2}$, we have $x=-y$.
• For $\lambda =\frac{3}{2}$, we have  $x=y$.

Combining with (Eq:lambda), we have

• For $\lambda =\frac{1}{2}$, we have $(x,y)=(\pm \frac{1}{\sqrt{2}},\mp \frac{1}{\sqrt{2}})$.
• For $\lambda =\frac{3}{2}$, we have  $(x,y)=(\pm \frac{1}{\sqrt{2}},\pm \frac{1}{\sqrt{2}})$.

In order to see if $f(x,y)$ indeed has extreme values at these points, let us examine its behavior in the neighbor of these points $(a,b)=(\pm \frac{1}{\sqrt{2}},\mp \frac{1}{\sqrt{2}}),(\pm \frac{1}{\sqrt{2}},\pm \frac{1}{\sqrt{2}})$.

Since $b\ne 0$, each point on the unit circle $g(x,y)=0$ can be expressed as $(x,\phi (x))$ where $y=\phi (x)$ is an implicit function of $g(x,y)=0$. Thus, we need to solve the extreme value problem of 

$$h(x)=f(x,\phi (x)).$$

Differentiating the equation of constraint $g(x,\phi (x))=0$, or

$$x^2+\{\phi (x){\}}^2-1=0$$

with respect to $x$, we have

$$2x+2\phi (x){\phi }^{\prime }(x)=0.$$

From this, we have

$${\phi }^{\prime }(x)=-\frac{x}{\phi (x)}.$$

Using the multivariate chain rule, we obtain

$$\begin{array}{rcl}{h}^{\prime }(x)& =& f_x(x,\phi (x))+f_y(x,\phi (x))(-\frac{x}{\phi (x)})=\frac{1-2x^2}{\phi (x)},\\ h^{″}(x)& =& \frac{-4x\phi (x)-(1-2x^2){\phi }^{\prime }(x)}{\{\phi (x){\}}^2}=\frac{-2x^3+[1-4\{\phi (x){\}}^2]x}{\{\phi (x){\}}^3}.\end{array}$$

If $(x,\phi (x))=(\pm \frac{1}{\sqrt{2}},\mp \frac{1}{\sqrt{2}})$ (corresponding to $\lambda =\frac{1}{2}$),  we have $h^{\prime }(x)=0$ and $h^{″}(x)=4>0$ so $h(x)$ has a local minimum value $\frac{1}{2}$.

If $(x,\phi (x))=(\pm \frac{1}{\sqrt{2}},\pm \frac{1}{\sqrt{2}})$ (corresponding to $\lambda =\frac{3}{2}$),  we have $h^{\prime }(x)=0$ and $h^{″}(x)=-4<0$ so $h(x)$ has a local maximum value $\frac{3}{2}$. □

It should be stressed that using the method of Lagrange multipliers, we did not need to know the explicit form of the implicit function $\phi (x)$. That is, the implicit function remains, in fact, implicit. The mere existence of the implicit function $\phi (x)$ is sufficient. Thus, this method is applicable even when the implicit function is hard (if possible) to find.

Remark. In general, constrained extreme value problems can be solved by the following procedure:

1. Enumerate the possible extreme points by the method of Lagrange multipliers.
2. For each possible extreme point, consider the implicit function in the neighbor of the point and evaluate the first and second derivatives.

□

We now give the method of Lagrange multipliers as a theorem.

Theorem [Method of Lagrange multipliers]

Let $f(x,y)$ and $g(x,y)$ be functions of class $C^1$. With a new variable $\lambda$, let us define

$$F(x,y,\lambda )=f(x,y)-\lambda g(x,y).$$

Suppose the point $(a,b)$ satisfies the following conditions:

1. The function $f(x,y)$ has an extreme value at $(x,y)=(a,b)$ subject to the constraint $g(x,y)=0$ (in particular, $g(a,b)=0$);
2. $(a,b)$ is a regular point of $g(x,y)=0$ (i.e., it is not the case that $g_x(a,b)=g_y(a,b)=0$).

Then, there exists $\alpha \in \mathbb{R}$ such that

$$F_x(a,b,\alpha )=F_y(a,b,\alpha )=F_{\lambda }(a,b,\alpha )=0.$$

Proof. By condition (1), $F_{\lambda }(a,b,\alpha )=-g(a,b)=0$. Thus, it suffices to show that there exists $\alpha \in \mathbb{R}$ such that

$$\begin{array}{}\text{(Eq:lagax)}& F_x(a,b,\alpha )& =& f_x(a,b)-\alpha g_x(a,b)=0,\text{(Eq:lagay)}& F_y(a,b,\alpha )& =& f_y(a,b)-\alpha g_y(a,b)=0.\end{array}$$

By condition (2), $g_x(a,b)\ne 0$ or $g_y(a,b)\ne 0$. Without loss of generality (what does this mean?), we may assume $g_y(a,b)\ne 0$. By the Implicit Function Theorem, there exists a function $y=\phi (x)$ in a neighbor of $x=a$ such that $b=\phi (a)$ and $g(x,\phi (x))=0$. In the neighbor of $(a,b)$, any point satisfying $g(x,y)=0$ is of the form $(x,\phi (x))$. Therefore, we need to solve the extreme value problem of the univariate function $f(x,\phi (x))$ in the neighbor of $x=a$. Note that

$${\phi }^{\prime }(x)=-\frac{g_x(x,\phi (x))}{g_y(x,\phi (x))}.$$

Differentiating $f(x,\phi (x))$ with respect to $x$, we have

$$f_x(x,\phi (x))+f_y(x,\phi (x)){\phi }^{\prime }(x)=f_x(x,\phi (x))-f_y(x,\phi (x))\frac{g_x(x,\phi (x))}{g_y(x,\phi (x))}.$$

By assumption, $f(x,\phi (x))$ has an extreme value at $x=a$,

$$f_x(a,\phi (a))=f_y(a,\phi (a))=0.$$

But $b=\phi (a)$ so that

$$f_x(a,b)-f_y(a,b)\frac{g_x(a,b)}{g_y(a,b)}=0.$$

It follows that

$$f_x(a,b)g_y(a,b)-f_y(a,b)g_x(a,b)=0.$$

Based on this, let us define

$$\alpha =\frac{f_y(a,b)}{g_y(a,b)}.$$

Clearly, this $\alpha$ satisfies (Eq:lagax}) and (Eq:lagay). ■