Geometric Representation of Method of Lagrange Multipliers
This Demonstration gives a geometric representation of the method of Lagrange multipliers. The initial view shows the red point iteratively moving toward a minimum of a specified function. At each iteration, the point takes a small step in the direction, shown by the red arrow, that causes the greatest reduction in the value of the function, i.e. the direction of steepest descent. This direction varies from point to point and can be visualized by the "field" of blue streamlines.[more]
Next, consider what will happen if the point is constrained to move along the circumference of a fixed circle. Check the box labeled "Constrain to circle" to find out. You can explore numerous ways of interacting with this figure, or read the Details section for further information on the method of Lagrange multipliers.[less]
1. Click anywhere in the figure to reset the point's initial position. As before, the point will move following the path of steepest descent toward a local minimum of the function.
2. When you check the option "Constrain to circle", a circle is displayed on the figure and the program attempts to seek a minimum while keeping the point constrained to the circumference of the circle. Also, a short segment of the tangent to the circle is shown in red.
3. When you check the option "Constrain to circle", you can drag the center of the circle to move the circle. Click elsewhere to reset the initial position of the point on the circle.
4. Use the "Pause" button to temporarily halt the animation. Click again to resume.
5. When the option "Show simple example" is checked, the original function is replaced by a simple function for which the direction of steepest descent is always pointing straight down.
The process of minimization is easiest to understand when you check both the options "Constrain to circle" and "Show simple example". By analogy of a bead acted upon by gravity and sliding along a hoop of wire, the "natural" location of the minimum is at the bottom of the circle. It is easy to see that the tangent at this minimum point must be horizontal, that is, perpendicular to the direction of gravity. If the tangent were not horizontal, that naturally would imply that the point could slide along the wire in the direction of the tangent and further minimize the function.
This same insight carries over to the more complex constrained minimization problem. When a minimum is found, the direction of steepest descent (which acts as the "direction of local gravity") must be perpendicular to the tangent of the constraint curve at that point.
Joseph-Louis Lagrange (1736–1813) prided himself on keeping his work free of distracting diagrams. Nevertheless, I hope that this Demonstration makes his beautiful application of the calculus more accessible.