Lagrange Multipliers: Constrained Optimisation Explained

Optimising with a Restriction

A farmer wants the largest rectangular field possible with a fixed length of fencing. A company wants the lowest production cost while producing a fixed number of units. A tin manufacturer wants a can holding 500 cm³ with the least metal. Each problem asks for a maximum or minimum of one quantity subject to a constraint on others. For this kind of constrained optimisation with two or more variables, the standard tool is the method of Lagrange multipliers. This tutorial explains the idea behind the method using gradients, sets out the steps, works through several examples from geometry, economics and engineering, and explains what the multiplier λ actually means.

The Problem

We want to find the maximum or minimum of a function f(x, y) (the objective), subject to a constraint g(x, y) = c.

Sometimes you can solve the constraint for one variable and substitute, reducing the problem to one variable. But that is often messy or impossible, and it gets worse with three or more variables. Lagrange's method avoids it.

The Idea: Parallel Gradients

Picture the contour lines of f (curves along which f is constant) and the curve g(x, y) = c. As you walk along the constraint curve, f rises and falls as you cross its contours. At a maximum or minimum along the constraint, the constraint curve just touches a contour of f without crossing it: the two curves are tangent.

The gradient of a function, ∇f = (∂f/∂x, ∂f/∂y), is perpendicular to its contours. If the curves are tangent, their perpendicular directions are parallel, so

∇f = λ∇g for some number λ, called the Lagrange multiplier.

The Method of Lagrange Multipliers Step by Step

Write down the equations ∂f/∂x = λ ∂g/∂x and ∂f/∂y = λ ∂g/∂y.
Add the constraint g(x, y) = c. You now have three equations in three unknowns: x, y and λ.
Solve them. A good strategy is to eliminate λ from the first two equations to get a relationship between x and y, then use the constraint.
Evaluate f at every candidate point. The largest value is the maximum and the smallest is the minimum (if the constraint curve is closed and bounded, both are guaranteed to exist).

Equivalently, form the Lagrangian L(x, y, λ) = f(x, y) − λ[g(x, y) − c] and set all three partial derivatives to zero.

Example 1: The Largest Rectangle

Find the dimensions of the rectangle with the greatest area that can be enclosed by 100 m of fencing.

Maximise f = xy subject to g = 2x + 2y = 100.

∂f/∂x = y and ∂g/∂x = 2, so y = 2λ.
∂f/∂y = x and ∂g/∂y = 2, so x = 2λ.

So x = y. Then 4x = 100, giving x = y = 25. The best rectangle is a 25 m × 25 m square with area 625 m². Here λ = 12.5.

Example 2: Extreme Values on a Circle

Find the maximum and minimum of f(x, y) = 3x + 4y on the circle x² + y² = 25.

3 = λ × 2x, so x = 3/(2λ).
4 = λ × 2y, so y = 2/λ = 4/(2λ).

So y/x = 4/3, giving y = 4x/3. Substitute into the constraint: x² + 16x²/9 = 25, so 25x²/9 = 25, x² = 9 and x = ±3.

x = 3, y = 4: f = 9 + 16 = 25 (maximum).
x = −3, y = −4: f = −9 − 16 = −25 (minimum).

Geometrically, the lines 3x + 4y = k touch the circle at these two points.

Example 3: The Cheapest Tin Can

A closed cylindrical can must hold V = 500 cm³. Find the radius r and height h that use the least metal.

Minimise the surface area A = 2πr² + 2πrh subject to πr²h = 500.

∂A/∂r = 4πr + 2πh = λ × 2πrh, which simplifies to 2r + h = λrh.
∂A/∂h = 2πr = λ × πr², which gives λ = 2/r.

Substitute λ = 2/r into the first equation: 2r + h = 2h, so h = 2r. The best can is as tall as it is wide. Then πr²(2r) = 500, so r³ = 250/π and r ≈ 4.30 cm, h ≈ 8.60 cm. (Real cans are often taller, because other costs such as the rims and seams matter too.)

Example 4: Three Variables

Find the point on the plane x + 2y + 2z = 18 closest to the origin.

Minimise the squared distance f = x² + y² + z² (easier than the distance itself, with the same minimum point) subject to x + 2y + 2z = 18.

2x = λ, 2y = 2λ, 2z = 2λ.
So x = λ/2, y = λ and z = λ.

Substitute: λ/2 + 2λ + 2λ = 18, so 4.5λ = 18 and λ = 4. The closest point is (2, 4, 4), at distance √(4 + 16 + 16) = 6. This agrees with the distance formula from a point to a plane, |−18|/√(1 + 4 + 4) = 18/3 = 6.

Example 5: Economics (Cobb–Douglas Production)

A firm's output is Q = 10K^0.5 L^0.5, where K is capital and L is labour. Capital costs ₦20 per unit and labour ₦5 per unit, and the budget is ₦400. How should the budget be split to maximise output?

Maximise Q subject to 20K + 5L = 400.

∂Q/∂K = 5K^(−0.5) L^0.5 = 20λ.
∂Q/∂L = 5K^0.5 L^(−0.5) = 5λ.

Divide the first equation by the second: L/K = 4, so L = 4K. Then 20K + 20K = 400, giving K = 10 and L = 40. Half the budget goes to each input, and the maximum output is Q = 10√(10 × 40) = 10 × 20 = 200 units.

What Does λ Mean?

The multiplier is not just a helper variable. At the optimum, λ is the rate at which the best value of f changes as the constraint value c increases: approximately, increasing c by 1 changes the optimal f by λ.

In Example 1, λ = 12.5: one extra metre of fencing adds about 12.5 m² of area. (Check: with 101 m, the best square has side 25.25 m and area 637.56 m², an increase of 12.56 m².)
In Example 5, λ = ∂Q/∂L ÷ 5 = (5 × √10/√40)/5 = 0.5: each extra naira of budget gives about half a unit of extra output. Economists call this the shadow price of the constraint.

Common Pitfalls with Lagrange Multipliers

Forgetting to include the constraint equation when solving.
Dividing by a variable that might be zero, which can lose candidate points. Consider the case "x = 0" separately.
Assuming every solution is a maximum. Compare the value of f at all candidates; on an unbounded constraint there may be no maximum at all.
The method also misses points where ∇g = 0 on the constraint; check these separately in unusual problems.

Summary

At a constrained extremum, ∇f = λ∇g: the contours of f are tangent to the constraint curve.
Solve ∇f = λ∇g together with g = c, usually by eliminating λ first.
Evaluate f at every candidate point to find the maximum and minimum.
λ measures how much the optimum changes per unit change in the constraint, the shadow price.
The method extends directly to three or more variables.