maths.free › Multivariable Calculus › 3. Derivatives of Multivariable Functions › Constrained Optimization: Lagrange Multipliers

Constrained Optimization: Lagrange Multipliers

This section requires the optimization tools from and the interpretations of the gradient from and .

Constrained Optimization: Lagrange Multipliers

This section requires the optimization tools from and the interpretations of the gradient from and . This section is omitted for time by some instructors but a requirement at some institutions with engineering programs that need this topic covered. This section can likely be covered in a single class, but that depends a lot on students' algebraic proficiency, as the nonlinear systems that arise in these problems can be challenging for students. Students tend to find the fact that there is not one method that will solve all of these systems frustrating, so giving them a variety of problems to practice on is important for instructors who will be giving anything but the most routine Lagrange multiplier problem on an exam. We have placed the geometric aspect (parallel gradients) prominently throughout the section, which allows for asking graphical questions that do not require intensive algebra questions a viable conceptual method for testing students on this material.

Introduction

We previously considered how to find the extreme values of functions on both unrestricted domains and closed, bounded domains. Other types of optimization problems involve maximizing or minimizing a quantity subject to an external constraint. In these cases the extreme values frequently won't occur at the points where the gradient is zero, but rather at other points that satisfy an important geometric condition. These problems are called constrained optimization problems and can be solved with the method of Lagrange Multipliers, which we study in this section. This method also provides an alternative approach to looking for maxima or minima on the boundary of a closed, bounded domain and can be useful in cases where parameterizing the boundary is more difficult than the situation we saw in .

Condensed — the full section is in Boelkins et al., Active Calculus Multivariable.

Constrained Optimization and Lagrange Multipliers

We will further examine our work in Preview Activity to motivate and understand the general method for solving constrained optimization problems. In Preview Activity, we considered an optimization problem where there is an external constraint on the variables, namely that the girth plus the length of the package cannot exceed 12 feet.

You saw in your plots that the direction to move in the $(x,y)$-plane to increase the volume would take you off of the constraint line. For points $P_1$ and $P_2$ you saw how moving down on the constraint line would move you at least a little bit in the direction of greatest increase (as measured by $\nabla f$). Similarly, at points $P_3$ and $P_4$ you could move up the constraint line to increase the value of the objective function. So somewhere between $P_2$ and $P_3$ there has to be a place where moving down the constraint line will no longer increase the output of our objective function.

Ultimately, we observed that the gradient of the objective function $f$ and the gradient of the constraint $g$ must be parallel when we maximize $f$ subject to the constraint. This is because the only way to increase the value of the objective function at such a point is to move off of the constraint curve, which is not allowed. In , we show the contour $f(x,y)=x^2y = 16$, the constraint $g(x,y) = 4x+y=12$, and the point $(2,4)$ at which these two intersect. We have also included the gradients at this point, illustrating that the gradients are parallel.

After doing the preview activity and looking at , you may be wondering if there are other contours that intersect the constraint line and have a point where the gradient to the objective function and the gradient to the constraint function are parallel. In the next example, we will set up an algebraic approach to finding all points on the constraint where the gradients of the objective and constraint functions are parallel and then find where maximum and minimum values occur for the objective function.

We summarize the process of Lagrange multipliers as follows.

Condensed — the full section is in Boelkins et al., Active Calculus Multivariable.

Practice (3)

Try each one on paper first. Reveal the answer to check; verified ones can be opened in the solver for every step.

The Cobb-Douglas production function is used in economics to model production levels based on labor and equipment. Suppose we have a specific Cobb-Douglas function of the form \[\begin{aligned}\end{aligned}\] where $x$ is the dollar amount spent on labor and $y$ the dollar amount spent on equipment. Use the method of Lagrange multipliers to determine how much should be spent on labor and how much on equipment to maximize productivity if we have a total of 1.5 million dollars to invest in labor and equipment.

Otkrij odgovor

To find the values of $\lambda$ that satisfy $\nabla f = \lambda \nabla g$ for our Cobb-Douglas function, we calculate both $\nabla f$ and $\nabla g$. Now \[\begin{aligned}\end{aligned}\], so we need a value of $\lambda$ so that \[\begin{aligned}\end{aligned}\].
Equating components and the original constraint give us the three equations \[\begin{aligned}20y^{0.6} \amp = \lambda x^{0.6} \\ 30x^{0.4} \amp = \lambda y^{0.4} \\ x+y \amp = 1.5\end{aligned}\] in the three unknowns $x$, $y$, and $\lambda$. Dividing both sides of by the corresponding sides of yields \[\begin{aligned}\frac{2}{3} \frac{y^{0.6}}{x^{0.4}} \amp = \frac{x^{0.6}}{y^{0.4}} \\ \frac{2}{3} y \amp = x\end{aligned}\]
Substituting into gives us \[\begin{aligned}\end{aligned}\] or \[\begin{aligned}\end{aligned}\].
Then $x = \frac{3}{5}$ and $\lambda \approx 25.51$. Therefore we maximize our output when we spend $0.6 million on labor and $0.9 million on equipment.
Use the method of Lagrange multipliers to find the point on the line $x-2y=5$ that is closest to the point $(1,3)$. To do so, respond to the following prompts.
1. Write the function $f=f(x,y)$ that measures the square of the distance from $(x,y)$ to $(1,3)$. (The extrema of this function are the same as the extrema of the distance function, but $f(x,y)$ is simpler to work with.)
2. What is the constraint $g(x,y) = c$?
3. Write the equations resulting from $\nabla f = \lambda \nabla g$ and the constraint. Find all the points $(x,y)$ satisfying these equations.
4. Test all the points you found to determine the extrema.
Otkrij odgovor
1. The square of the distance from $(x,y)$ to $(1,3)$ is given by $f(x,y) = (x-1)^2 + (y-3)^2$.
2. The constraint is that the point $(x,y)$ lies on the line $x-2y=5$, so $g(x,y) = x-2y$.
3. Since $\nabla f = \langle 2(x-1) + 2(y-3)$ and $\nabla g = \langle 1, -2 \rangle$, the equations resulting from $\nabla f = \lambda \nabla g$ and the constraint are \[\begin{aligned}2(x-1) \amp = \lambda \\ 2(y-3) \amp = -2\lambda \\ x-2y \amp = 5\end{aligned}\]. Substituting $2(x-1)$ for $\lambda$ in the second equation yields $2(y-3) = -4(x-1)$ or $4x+2y=10$. Adding corresponding sides of this new equation with the third equation gives us $5x = 15$ or $x=3$. Then $y = -1$.
4. This single critical point gives us an extreme distance of $\sqrt{f(3,-1)} = \sqrt{20}$. This is the only extreme value and since points on the plane can be found as far as possible from the point $(1,3)$, we have found the minimum distance.
Apply the Method of Lagrange Multipliers to solve each of the following constrained optimization problems.
1. Determine the absolute maximum and absolute minimum values of $f(x,y) = (x-1)^2 + (y-2)^2$ subject to the constraint that $x^2 + y^2 = 16$.
2. Determine the points on the sphere $x^2 + y^2 + z^2 = 4$ that are closest to and farthest from the point $(3,1,-1)$. (As in the preceding exercise, you may find it simpler to work with the square of the distance formula, rather than the distance formula itself.)
3. Find the absolute maximum and minimum of $f(x,y,z) = x^2 + y^2 + z^2$ subject to the constraint that $(x-3)^2 + (y+2)^2 + (z-5)^2 \le 16$. (Hint: here the constraint is a closed, bounded region. Use the boundary of that region for applying Lagrange Multipliers, but don't forget to also test any critical values of the function that lie in the interior of the region.)
Otkrij odgovor
1. Here we have $g(x,y) = x^2+y^2$, so the system of equations resulting from $\nabla f = \lambda \nabla g$ and the constraint is \[\begin{aligned}2(x-1) \amp = 2\lambda x \\ 2(y-2) \amp = 2\lambda y \\ x^2 + y^2 \amp = 16\end{aligned}\]. Dividing corresponding sides of the first equation by the second shows that $\frac{x-1}{y-2} = \frac{x}{y}$ or $(x-1)y = x(y-2)$. Simplifying yields $y=2x$. Substituting into the constraint equation gives us $x^2+4x^2 = 16$ or $x = \pm \frac{4}{\sqrt{5}}$. Checking the values of $f$ at these critical points we see that \[\begin{aligned}f\left(\frac{4}{\sqrt{5}}, \frac{8}{\sqrt{5}}\right) \amp = 21-8\sqrt{5} \approx 3.11 \\ f\left(-\frac{4}{\sqrt{5}}, -\frac{8}{\sqrt{5}}\right) \amp = 21+8\sqrt{5} \approx 38.89\end{aligned}\]. So the maximum value of $f$ subject to this constraint is $21+8\sqrt{5}$ and occurs at the point $\left(\frac{4}{\sqrt{5}}, \frac{8}{\sqrt{5}}\right)$ while the minimum value is $21-8\sqrt{5}$ and occurs at $\left(-\frac{4}{\sqrt{5}}, -\frac{8}{\sqrt{5}}\right)$.
2. The square of the distance from $(x,y,z)$ to $(3,1,-1)$ is given by $f(x,y) = (x-3)^2 + (y-1)^2 + (z+1)^2$. The constraint is $g(x,y) = x^2+y^2+z^2=4$. The equations resulting from $\nabla f = \lambda \nabla g$ and the constraint are \[\begin{aligned}2(x-3) \amp = 2\lambda x \\ 2(y-1) \amp = 2\lambda y \\ 2(z+1) \amp = 2\lambda z \\ x^2+y^2+z^2 \amp = 4\end{aligned}\]. Dividing corresponding sides of the first equation by the second shows that $\frac{x-3}{y-1} = \frac{x}{y}$ or $(x-3)y = x(y-1)$. Simplifying yields $x=3y$. Dividing corresponding sides of the second equation by the third shows that $\frac{y-1}{z+1} = \frac{y}{z}$ or $(y-1)z = y(z+1)$. Simplifying yields $z=-y$. Substituting into the constraint equation gives us $11y^2=4$ or $y = \pm \frac{2}{\sqrt{11}}$. Checking the values of $f$ at these critical points we see that \[\begin{aligned}f\left(\frac{6}{\sqrt{11}}, \frac{2}{\sqrt{11}}, -\frac{2}{\sqrt{11}} \right) \amp = 15-4\sqrt{11} \approx 1.73 \\ f\left(-\frac{6}{\sqrt{11}}, -\frac{2}{\sqrt{11}}, \frac{2}{\sqrt{11}} \right) \amp = 15+4\sqrt{11} \approx 28.27\end{aligned}\]. So the point on the sphere closest to $(3,1,-1)$ is $\left(\frac{6}{\sqrt{11}}, \frac{2}{\sqrt{11}}, -\frac{2}{\sqrt{11}} \right)$ and the point on the sphere farthest from $(3,1,-1)$ is $\left(-\frac{6}{\sqrt{11}}, -\frac{2}{\sqrt{11}}, \frac{2}{\sqrt{11}} \right)$.
3. First we find the critical points of $f$ inside the sphere. We have $\nabla f = 0$ when \[\begin{aligned}\end{aligned}\], yielding the single critical point $(0,0,0)$. Next we determine the critical points of $f$ on the boundary of the sphere $(x-3)^2 + (y+2)^2 + (z-5)^2 \le 16$, using the constraint $g(x,y,z) = (x-3)^2 + (y+2)^2 + (z-5)^2 = 16$. The equations resulting from $\nabla f = \lambda \nabla g$ and the constraint are \[\begin{aligned}2x \amp = 2\lambda (x-3) \\ 2y \amp = 2\lambda (y+2) \\ 2z \amp = 2\lambda (z-5) \\ (x-3)^2 + (y+2)^2 + (z-5)^2 \amp = 16\end{aligned}\]. Dividing corresponding sides of the first equation by the second shows that $\frac{x}{y} = \frac{x-3}{y+2}$ or $x(y+2)=y(x-3)$. Simplifying yields $x=-\frac{3}{2}y$. Dividing corresponding sides of the second equation by the third shows that $\frac{y}{z}=\frac{y+2}{z-5}$ or $y(z-5) = z(y+2)$. Simplifying yields $z=-\frac{5}{2}y$. Substituting into the constraint equation gives us $\frac{19}{2}y^2+38y+38=16$. The quadratic formula reveals that $y = -2 \pm \frac{4}{19} \sqrt{38}$. Checking the values of $f$ at these critical points we see that \[\begin{aligned}f\left(-\frac{3}{2}\left(-2 +\frac{4}{19} \sqrt{38}\right), -2 + \frac{4}{19} \sqrt{38}, -\frac{5}{2}\left(-2 +\frac{4}{19} \sqrt{38}\right) \right) \amp = 54-8\sqrt{38} \approx 4.68 \\ f\left(-\frac{3}{2}\left(-2 -\frac{4}{19} \sqrt{38}\right), -2 - \frac{4}{19} \sqrt{38}, -\frac{5}{2}\left(-2 -\frac{4}{19} \sqrt{38}\right) \right) \amp = 54+8\sqrt{38} \approx 103.32 \\ f(0,0,0) \amp = 0\end{aligned}\]. So the absolute maximum value of $f$ on the domain $(x-3)^2 + (y+2)^2 + (z-5)^2 \le 16$ is $54+8\sqrt{38}$ and the absolute minimum value is 0.

Symbols used here

\nabla f

gradient (nabla, del)
Vector of partial derivatives; points uphill.

\lambda

lambda (eigenvalue)
The factor by which an eigenvector is stretched: Av = λv.

\leq,\ \geq

less/greater than or equal
Inequalities that allow equality; < and > exclude it.

\frac{\partial f}{\partial x}

partial derivative
Derivative with respect to x, holding the other variables fixed.

\int f(x)\,dx,\ \int_a^b

integral
Antiderivative (indefinite) or signed area from a to b (definite).

\iint,\ \oint

double / contour integral
Integral over a region of the plane; integral around a closed curve.

\mathbf{v},\ \vec{v}

vector
A quantity with magnitude and direction; a column of numbers.

\mathbf{u} \cdot \mathbf{v},\ \|\mathbf{v}\|

dot product, norm
Σ u_i v_i; the length of v, √(v·v).

How to: Constrained Optimization: Lagrange Multipliers

What geometric condition enables us to optimize a function f=f(x,y) subject to a constraint given by g(x,y) = k, where k is a constant?
How can we exploit this geometric condition to find the extreme values of a function subject to a constraint?

Questions people ask

What is a partial derivative?

The ordinary derivative with respect to one variable while every other variable is frozen — the slope of the surface in one coordinate direction.

What does the gradient point at?

Uphill: the direction of steepest increase, with length equal to that steepest slope. It is perpendicular to the level curves.

Pokušaj i ti

Parts of this page are adapted from Boelkins et al., Active Calculus Multivariable (CC BY-SA 4.0). Condensed and re-explained here; errors are ours.

Više u Multivariable Calculus

Functions of several variables Partial derivatives and the gradient Optimisation in several variables Double and triple integrals Vector fields, line integrals and the big theorems