Calculus of Variations

Introduction

Calculus of variations is a field of mathematical analysis that deals with maximizing or minimizing functionals. Unlike standard calculus, which focuses on analyzing functions that take numbers as inputs and produce numbers as outputs, calculus of variations analyzes functionalsmappings from a space of functions to the real numbers.

The fundamental problem in calculus of variations is to find a function y(x) that makes a functional J[y] extremal (maximum or minimum) among all functions satisfying certain boundary conditions. This can be expressed as:

Where f is a given function, and y(x) is the unknown function we wish to determine.

This diagram illustrates a functional y(x) between two points (x, y) and (x, y), which is typical in variational problems.

What sets calculus of variations apart is that we're not just looking for specific numerical optima, but rather for entire functions that optimize certain properties overall. Problems in mechanics, physics, economics, and many other fields often naturally take this form.

Historical Overview

The calculus of variations emerged in the 17th century through problems that classical calculus could not solve. One of the earliest problems that led to its development was the brachistochrone problem posed by Johann Bernoulli in 1696, which asked for the curve of fastest descent between two points.

Key contributors to the development of calculus of variations include:

• The Bernoulli brothers: Johann and Jacob Bernoulli who posed and solved early variational problems
• Leonhard Euler: Developed systematic methods and laid the groundwork for the theory
• Joseph-Louis Lagrange: Introduced the calculus of variations in its modern form
• Carl Friedrich Gauss: Applied variational methods to physics and astronomy
• Karl Weierstrass: Provided rigorous foundations for the subject
• David Hilbert: Made significant contributions to direct methods in the calculus of variations

The subject matured through the 18th and 19th centuries, with applications extending from pure mathematics to physics, particularly in mechanics and field theory.

Fundamental Concepts

The calculus of variations builds upon some key concepts that differentiate it from ordinary calculus:

Functionals

A functional is a mapping from a space of functions to the real numbers. While ordinary functions take numbers as inputs and produce numbers as outputs, functionals take entire functions as inputs and produce numbers as outputs. Common functionals include integrals of the form:

Variations

The variation of a functional J is analogous to the differential of a function in ordinary calculus. It represents how the value of the functional changes when the argument function is slightly varied.

Stationarity

A function y is said to make a functional J[y] stationary if the first variation J[y,] vanishes for all admissible variations . This is analogous to the condition that the derivative of a function is zero at an extremum in ordinary calculus.

Admissible Functions

In variational problems, we typically constrain our search to functions that satisfy certain conditions. At a minimum, these functions must satisfy the given boundary conditions, but may also need to satisfy smoothness, differentiability, or other requirements.

The Euler-Lagrange Equation

The cornerstone of the calculus of variations is the Euler-Lagrange equation, which provides a necessary condition for a function to be an extremal of a given functional. Developed jointly by Euler and Lagrange in the 1750s, this equation serves as the primary tool for solving variational problems.

For a functional of the form:

With fixed boundary conditions y(x) = y and y(x) = y, the Euler-Lagrange equation is:

This second-order differential equation must be satisfied by any function y(x) that makes the functional J[y] stationary. Solving this differential equation with the given boundary conditions yields candidate extremals.

Derivation of the Euler-Lagrange Equation

The derivation of the Euler-Lagrange equation involves considering a small variation (x) to the function y(x) and examining how the functional changes. The condition for stationarity is that the first variation of J[y] vanishes for all such variations (x) that are zero at the boundary.

Through integration by parts and using the boundary conditions, we arrive at the Euler-Lagrange equation. This derivation illustrates how the calculus of variations connects the optimization of functionals with differential equations.

Special Cases and Extensions

The Euler-Lagrange equation can be extended in several ways:

• For functionals involving higher derivatives, higher-order Euler-Lagrange equations result.
• For functionals involving multiple independent variables, we obtain partial differential equations.
• For problems with constraints, we use Lagrange multipliers in the variational framework.
• For isoperimetric problems (with integral constraints), we introduce additional terms to account for the constraint.

The Beltrami Identity

When the integrand f(x, y, y') does not explicitly depend on x, the Euler-Lagrange equation can be integrated once to yield the Beltrami identity:

This first integral can significantly simplify the solution process in many problems.

Applications of Calculus of Variations

Calculus of variations finds applications across numerous scientific and engineering disciplines. Here are some notable examples:

Classical Mechanics

One of the most profound applications is in classical mechanics, where Hamilton's principle states that the actual path taken by a mechanical system between two configurations is the one that makes the action functional stationary. The action is defined as:

Where L = T - V is the Lagrangian, with T being kinetic energy and V being potential energy. Applying the Euler-Lagrange equation to this functional yields Lagrange's equations of motion, which are equivalent to Newton's laws but often more convenient for complex systems.

Optics and Geodesics

Fermat's principle in optics states that light travels between two points along the path that takes the least time. This is naturally expressed as a variational problem. A similar principle applies to geodesicsthe shortest paths on curved surfaceswhich find applications in general relativity and differential geometry.

Control Theory

Optimal control problems, which involve finding control policies that optimize system behavior, are naturally formulated as variational problems. The Pontryagin maximum principle, a key result in control theory, can be viewed as an extension of the Euler-Lagrange equation to control problems.

Quantum Mechanics

The Feynman path integral formulation of quantum mechanics expresses the probability amplitude for a particle to move from one point to another as an integral over all possible paths, with each path weighted by an exponential involving the classical action. This provides a deep connection between classical and quantum mechanics through variational principles.

Economics

In economics, variational methods are used in optimal growth theory, resource extraction problems, and other dynamic optimization scenarios where decisions made over time must optimize some objective.

Direct Method of Calculus of Variations

The direct method is a powerful approach in the calculus of variations that doesn't rely on differential equations like the Euler-Lagrange equation. Instead, it works directly with the functional itself, often using functional analysis and topology.

Bolzano-Weierstrass Theorem

The theoretical foundation of the direct method relies on the Bolzano-Weierstrass theorem, which states that every bounded sequence in has a convergent subsequence. In infinite-dimensional spaces of functions, we need analogous results, which leads to the concept of weak convergence and weak compactness.

Key Steps in the Direct Method

1. Extract a minimizing sequence: Consider a sequence of functions {y} such that J[y] converges to the infimum of J.
2. Show the sequence is bounded: Prove that the functions y are bounded in an appropriate norm.
3. Obtain a convergent subsequence: Use compactness arguments to extract a subsequence that converges in some sense (often weak convergence).
4. Show the limit is admissible: Prove that the limit function satisfies the required boundary conditions and other constraints.
5. Show the limit minimizes the functional: Demonstrate that J[y] = lim J[y] = inf J, which typically requires some form of lower semicontinuity.

Direct Method in Practice

The direct method is particularly valuable for problems where the Euler-Lagrange equation is difficult to solve or may not have classical solutions. It's especially useful in problems with non-differentiable functionals or constraints that make traditional approaches challenging.

Non-Existence of Solutions

Not every variational problem has a solution. A classic counterexample is the problem of minimizing

subject to the constraint

One can construct a sequence of functions {y} with J[y] 0 that do not converge to a function in the admissible class. This highlights the importance of appropriate function spaces and conditions for existence.

Modern Developments and Computational Approaches

Calculus of variations remains an active field of research with modern developments in both theory and computational methods.

Convex Analysis

The connection between calculus of variations and convex analysis has been extensively developed in recent decades. When the integrand is convex with respect to the derivative term, the variational problem has particularly nice properties, including the existence of solutions and uniqueness under certain conditions.

-Convergence

-convergence is a notion of convergence for functionals introduced by De Giorgi. It has proven to be a powerful tool for approximating complex variational problems by simpler ones and ensuring that minimizers of the approximating problems converge to minimizers of the original problem.

Homogenization

Homogenization theory addresses problems with highly oscillatory coefficients, common in composite materials and other heterogeneous media. Variational methods are essential for deriving effective models that capture the macroscopic behavior while accounting for microscopic structure.

Computational Variational Methods

The computational solution of variational problems involves several powerful techniques:

• Finite element methods: Used to discretize the domain and solve the Euler-Lagrange equations numerically.
• Ritz method: Approximates the solution using a linear combination of basis functions and finds coefficients that minimize the functional directly.
• Galerkin method: Similar to the Ritz method but more broadly applicable to problems that don't arise from minimization.
• Gradient flow and descent methods: Iteratively improve an approximate solution by moving in directions that reduce the functional value.

Mechanics Materials and Defects

In materials science, variational methods are used to model phase transitions, fractures, and defects. These applications often involve non-convex energy landscapes and require sophisticated mathematical techniques.

Image Processing

Variational methods are increasingly used in image processing for tasks like denoising, segmentation, and inpainting. These problems are often formulated as variational problems with regularization terms to impose smoothness or other desirable properties on the solution.

Resources for Further Study

For those interested in exploring calculus of variations further, the following resources provide excellent starting points:

Textbooks

• Gelfand, I. M., and Fomin, S. V., Calculus of Variations A classic introduction with many worked examples.
• Weinstock, R., Calculus of Variations: With Applications to Physics and Engineering A comprehensive text focusing on applications.
• Dacorogna, B., Direct Methods in the Calculus of Variations A modern treatment of the direct method.
• Evans, L. C., Partial Differential Equations Contains a solid chapter on the calculus of variations and its connection to PDEs.
• Jost, J., and Li-Jost, X., Calculus of Variations A mathematically rigorous treatment suitable for graduate students.

Online Resources

• MIT OpenCourseWare Calculus of Variations Lecture notes and problem sets from MIT.
• Stanford Engineering Everywhere Variational Methods Video lectures on variational methods in engineering.
• Wolfram MathWorld Calculus of Variations A concise overview with several solved examples.
• arXiv Optimization and Control Preprints of current research in variational methods and optimization.

Related Topics

To gain a deeper understanding of calculus of variations, it's beneficial to study related fields:

• Functional Analysis and Sobolev Spaces
• Optimization Theory
• Control Theory
• Differential Geometry
• Classical Mechanics