What is Entropy?
Entropy originated in thermodynamics as a measure of the irreversibility of physical processes. In statistical mechanics, it quantifies the number of microscopic configurations compatible with a macroscopic state. Mathematically, entropy is often represented by a convex functional\( \eta : \mathbb{R}^n \to \mathbb{R} \) that satisfies certain monotonicity properties.
Beyond physics, the concept has been adopted in information theory (Shannon entropy), probability, and dynamical systems. In each setting, entropy measures disorder, uncertainty, or the loss of information. The common thread is the principle that natural processes tend to increase entropy, a statement known as the second law of thermodynamics.
Partial Differential Equations
A partial differential equation (PDE) involves an unknown function of several variables and its partial derivatives. PDEs model a wide range of phenomena: heat conduction, fluid flow, electromagnetic fields, quantum mechanics, and many others.
Classic examples include:
- Heat equation: \(\displaystyle u_t = \alpha \Delta u\)
- Wave equation: \(\displaystyle u_{tt} = c^2 \Delta u\)
- Laplaces equation: \(\displaystyle \Delta u = 0\)
- NavierStokes equations: \(\displaystyle \partial_t \mathbf{u} + (\mathbf{u}\!\cdot\!\nabla)\mathbf{u} = -\nabla p + \nu \Delta\mathbf{u}\)
Each PDE comes with its own set of existence, uniqueness, and regularity questions. The analytical toolkit includes energy estimates, maximum principles, Fourier analysis, and functionalanalytic methods.
Entropy in PDEs
When studying nonlinear PDEs, especially conservation laws and diffusion equations, entropy plays a central role. A typical setting is a system of conservation laws
where \(\mathbf{u}:\mathbb{R}^d\times[0,\infty)\to\mathbb{R}^n\) is the vector of conserved quantities and \(\mathbf{f}\) is the flux. Classical solutions rarely exist beyond smooth initial data; instead, one seeks weak solutions that satisfy the integral form of the equation.
To single out physically relevant weak solutions, an additional inequality is imposed: the entropy condition. Let \(\eta(\mathbf{u})\) be a convex entropy function and \(\mathbf{q}(\mathbf{u})\) its associated entropy flux, satisfying \(\nabla\eta \cdot \nabla \mathbf{f} = \nabla\mathbf{q}\). The entropy inequality reads
For scalar conservation laws (e.g., Burgers equation), the condition eliminates nonphysical shock waves and guarantees uniqueness via the CrandallLaxOleinik theory. In multidimensional systems, entropy pairs are essential for deriving a priori estimates, proving compactness, and establishing convergence of numerical schemes.
Entropy Solutions for Diffusion Equations
Consider the porousmedium equation
One can define an entropy functional \(\displaystyle \mathcal{E}(u) = \int_{\mathbb{R}^d} \frac{u^{m+1}}{m+1}\,dx\). Direct computation shows that \(\mathcal{E}(u(t))\) is nonincreasing in time, yielding an entropy dissipation inequality
This dissipation is a powerful tool for establishing smoothing effects and longtime asymptotics (e.g., convergence to Barenblatt profiles). Similar ideas appear in the analysis of the FokkerPlanck equation, the fast diffusion equation, and many gradientflow structures.
Entropy Methods for Kinetic Equations
The Boltzmann equation, governing the statistical behavior of dilute gases, possesses a celebrated entropy functionalthe Boltzmann \(H\)functional:
where \(f(t,x,v)\) is the particle density in phase space. The celebrated \(H\)theorem states that \(\frac{d}{dt}H(f) \le 0\). This monotonicity underlies the trend toward thermodynamic equilibrium and drives modern renormalization techniques.
Applications of EntropyBased PDE Analysis
Fluid Dynamics
Entropy inequalities are indispensable in the rigorous theory of compressible NavierStokes equations. They provide control over density oscillations and enable the passage to the limit in approximation schemes. In turbulent flow modelling, entropy concepts appear in subgridscale closures.
Image Processing
Variational models for denoising and deblurring often minimise an energy that includes an entropylike term (e.g., total variation). The resulting EulerLagrange equations are PDEs whose solutions inherit the entropy dissipation property, ensuring stability and edge preservation.
Machine Learning
Gradient flows in the space of probability measures, governed by the Wasserstein metric, can be written as PDEs (e.g., the JKO scheme). Entropy functionals serve as the driving energy, linking optimal transport, diffusion, and deep learning algorithms.
Geometric Flows
Meancurvature flow and Ricci flow are geometric evolution equations that can be interpreted as entropygradient flows. Perelmans entropy functional for Ricci flow was pivotal in proving the Poincar conjecture.
Further Reading
- Evans, L. C.*Partial Differential Equations*. AMS, 2nded., 2010.
- LeVeque, R. J.*Finite Volume Methods for Hyperbolic Problems*. Cambridge UP, 2002.
- Villani, C.*Topics in Optimal Transportation*. AMS, 2003.
- Brenier, Y.Polar factorization and monotone rearrangement. *Communications on Pure and Applied Mathematics*, 1991.
- Dafermos, C.*Hyperbolic Conservation Laws in Continuum Physics*. Springer, 2005.
These texts provide deeper insight into the interplay between entropy and PDEs, ranging from foundational theory to modern research frontiers.
