Chapter 5 Robust Performance Tailoring with Tuning - SSL - MIT

major iteration an approximation of the Hessian of the Lagrangian function is made
using the quasi-Newton updating method of Broyden, Fletcher, Goldfarb and Shanno
(BFGS). The Hessian approximation is then used along with first-order Taylor series
approximations of the nonlinear constraints to generate a quadratic programming
(QP) subproblem:
min
d∈Rn 1
2 dT Hkd + ∇f (xk) T d
∇gi (xk) T d + gi (xk) =0 i =1,...me (2.40)
∇gi (xk) T d + gi (xk) ≤ 0 i = me +1,...m
where H is the Hessian approximation, gi are the constraint equations and m and me
are the total number of constraints and the number of equality constraints, respec-
tively. The solution, dk, is obtained through an active set quadratic programming
strategy and is used to form a new major iterate:
xk+1 = xk + αkdk
(2.41)
The step length, αk is found through a line search that requires sufficient decrease in
a particular merit function.
If a problem is well-behaved and properly scaled, then gradient-based algorithms,
such as SQP, are likely to find a global optimum as long as the objective function and
constraints are convex. However, if the problem is non-convex, i.e. there are multiple
solutions in the space that are locally optimal, then the algorithm may converge to a
local minima, instead of the global one. In fact, the solution that is obtained depends
on the initial guess chosen by the user. Non-convexity poses a difficult problem since
there is no known way to prove definitively that a global optimum has been found
instead of simply a local one. Therefore, there is an entire body of heuristic methods
that can be employed to search for a global optimum. One simple heuristic is to
randomly chose some number of initial guesses and run SQP, or some other gradient-
based algorithm, from each of these starting points. Each resulting solution is then
56