Numerical Optimization Using MATLAB

Introduction

Optimization is the process of finding the best solutionaccording to a defined criterionamong all feasible alternatives. In engineering, finance, data science, and many other fields, numerical optimization provides the backbone for model fitting, design, resource allocation, and decisionmaking.

MATLAB offers a comprehensive toolbox for both simple and advanced optimization problems. It combines highlevel language constructs, builtin solvers, and visualisation capabilities that let users prototype, test, and refine algorithms quickly.

Why MATLAB for Optimization?

• Unified Environment: MATLAB integrates matrix algebra, plotting, and scripting, making it easy to define objective functions and constraints.
• Extensive Solver Suite: The Optimization Toolbox contains algorithms for linear programming (LP), mixedinteger programming (MIP), quadratic programming (QP), nonlinear programming (NLP), and more.
• Automatic Differentiation: Functions such as gradients and hessian can be approximated numerically, reducing the burden of handcoding derivatives.
• Parallel & GPU Support: Largescale problems can exploit multicore CPUs or GPUs with minimal code changes.
• Rich Documentation & Examples: Every solver includes a set of example scripts and detailed help pages.

Core Functions and Workflow

The typical workflow for a numerical optimization project in MATLAB follows these steps:

1. Define the objective function f(x) (and optionally its gradient and Hessian).
2. Specify constraints (linear, bound, or nonlinear).
3. Select an appropriate solver (e.g., fmincon, linprog, ga).
4. Configure options via optimoptions to control tolerances, iteration limits, and output.
5. Run the solver and analyse the results.

Key Solver Functions

• fminunc Unconstrained nonlinear minimisation.
• fmincon Constrained nonlinear minimisation.
• linprog Linear programming.
• quadprog Quadratic programming.
• intlinprog Mixedinteger linear programming.
• ga Genetic algorithm (global optimisation).
• patternsearch Direct search method for derivativefree problems.
• lsqnonlin Nonlinear leastsquares.

Setting Options

Most solvers accept an optimoptions object. For example, to increase the maximum number of iterations and request a plot of the objective value during optimisation:

options = optimoptions('fmincon', ...    'MaxIterations', 500, ...    'Display', 'iter', ...    'PlotFcn', @optimplotfval);

Types of Optimization Problems

1. Linear Programming (LP)

Linear programs have a linear objective and linear constraints. In MATLAB they are solved with linprog:

c = [-1; -2];                % Minimise -x - 2y   maximise x+2yA = [1, 1];b = 5;lb = zeros(2,1);              % x  0, y  0[x, fval] = linprog(c, A, b, [], [], lb, []);

2. Quadratic Programming (QP)

Quadratic programs contain a quadratic term in the objective. They are useful for portfolio optimisation, control, and ridge regression.

H = [4, 1; 1, 2];f = [-8; -6];A = [1, 2];b = 3;lb = [0; 0];[x, fval] = quadprog(H, f, A, b, [], [], lb, []);

3. Unconstrained Nonlinear Optimization

Use fminunc when there are no constraints. Provide gradients for speed when possible.

obj = @(x) (x(1)-2)^2 + (x(2)+3)^2;grad = @(x) [2*(x(1)-2); 2*(x(2)+3)];options = optimoptions('fminunc','GradObj','on','Display','iter');[x, fval] = fminunc(obj, [0;0], options);

4. Constrained Nonlinear Optimization

fmincon handles bound, linear, and nonlinear constraints.

obj = @(x) sin(x(1))*cos(x(2));nonlin = @(x) deal([], x(1)^2 + x(2)^2 - 1);   % Equality: x^2+y^2 = 1lb = [-5; -5];ub = [5; 5];options = optimoptions('fmincon','Display','iter','Algorithm','sqp');[x,fval] = fmincon(obj, [1;1], [], [], [], [], lb, ub, nonlin, options);

5. Global Optimization

When multiple local minima exist, stochastic methods such as genetic algorithms (ga) or pattern search can locate a global optimum.

obj = @(x) (x(1)^2-10)^2 + (x(2)^2-10)^2;lb = [-10 -10];ub = [10 10];options = optimoptions('ga','Display','iter','PopulationSize',70);[x,fval] = ga(obj,2,[],[],[],[],lb,ub,[],options);

Complete Example: Parameter Estimation for a Nonlinear Model

Suppose we have experimental data that follows the model y = aexp(bt) + c. We want to estimate the parameters a, b, and c that minimise the sum of squared residuals.

Step 1 Generate synthetic data (for illustration)

t = linspace(0,5,50)';trueParams = [2.5, -0.8, 0.3];yTrue = trueParams(1)*exp(trueParams(2)*t) + trueParams(3);rng(1); % reproducibilityyNoisy = yTrue + 0.05*randn(size(yTrue));

Step 2 Define objective (leastsquares) function

obj = @(p) sum((p(1)*exp(p(2)*t) + p(3) - yNoisy).^2);

Step 3 Choose starting guess and run lsqnonlin

p0 = [1, -1, 0];   % initial guessoptions = optimoptions('lsqnonlin','Display','iter','PlotFcn',@optimplotresnorm);[pEst, resnorm] = lsqnonlin(@(p) p(1)*exp(p(2)*t) + p(3) - yNoisy, p0, [], [], options);

Step 4 Visualise the fit

yFit = pEst(1)*exp(pEst(2)*t) + pEst(3);figure;plot(t, yNoisy, 'ko', 'MarkerSize',5);hold on;plot(t, yFit, 'r-', 'LineWidth',2);plot(t, yTrue, 'b--', 'LineWidth',1);legend('Noisy data','Estimated model','True model');xlabel('Time (t)');ylabel('Response (y)');title('Nonlinear Parameter Estimation');

Running the script returns an estimate close to [2.48, -0.79, 0.31], demonstrating how MATLAB can solve realistic optimisation tasks with only a few lines of code.

Practical Tips & Further Resources

• Scale your variables. Poorly scaled problems often cause solvers to stall. Use norm or explicit scaling factors.
• Provide analytical gradients when possible. Supplying grad and hessian can reduce iterations dramatically.
• Check feasibility. Before calling a solver, verify that initial guesses satisfy bounds and simple constraints.
• Use optimplot functions. Visual diagnostics (objective value, constraints violation) help detect problems early.
• Leverage parallel computing. For MonteCarlo studies or large population algorithms, wrap solvers in parfor loops.

Additional learning material:

1. MATLAB Documentation Optimization Toolbox
2. Book: Practical Optimization: Algorithms and Engineering Applications (by Gill, Murray, Wright)
3. Online Course Coursera Mathematical Optimization in MATLAB.
4. MATLAB Central File Exchange many usercontributed examples.

By mastering these tools, engineers and scientists can translate complex mathematical models into actionable designs, making MATLAB an indispensable platform for numerical optimisation.