What Is Optimization Toolbox?

Trust-Region Methods for Nonlinear MinimizationTrust-Region Methods for Nonlinear MinimizationMany of the methods used in Optimization Toolbox are based on trust regions,a simple yet powerful concept in optimization.To understand the trust-region approach to optimization, consider theunconstrained minimization problem,, where the functiontakes vector arguments and returns scalars. Suppose you are at a point inn-space and you want to improve, i.e., move to a point with a lower functionvalue. The basic idea is to approximate with a simpler function whichreasonably reflects the behavior of function in a neighborhood around thepoint x. This neighborhood is the trust region. Atrialstep is computed byminimizing (or approximately minimizing) over N. Thisisthetrust-regionsubproblem,(4-1)The current point is updated to be if ; otherwise, the currentpoint remains unchanged and N, the region of trust, is shrunk and the trialstep computation is repeated.The key questions in defining a specific trust-region approach to minimizingare how to choose and compute the approximation (defined at thecurrent point ),howtochooseandmodifythetrustregionN, and howaccurately to solve the trust-region subproblem. This section focuses on theunconstrained problem. Later sections discuss additional complications dueto the presence of constraints on the variables.In the standard trust-region method ([8]), the quadratic approximationis defined by the first two terms of the Taylor approximation to at x; theneighborhood is usually spherical or ellipsoidal in shape. Mathematicallythe trust-region subproblem is typically stated(4-2)where is the gradient of at the current point x, is the Hessian matrix (thesymmetric matrix of second derivatives), is a diagonal scaling matrix, is apositive scalar, and || . || is the 2-norm. Good algorithms exist for solving4-3