What Is Optimization Toolbox?

Quasi-Newton ImplementationMixed Cubic and Quadratic Polynomial MethodThe cubic interpolation/extrapolation method has proved successful for a largenumber of optimization problems. However, when analytic derivatives are notavailable, evaluating finite difference gradients is computationally expensive.Therefore, another interpolation/extrapolation method is implemented sothat gradients are not needed at every iteration. The approach in thesecircumstances, when gradients are not readily available, is to use a quadraticinterpolation method. The minimum is generally bracketed using some formof bisection method. This method, however, has the disadvantage that allthe available information about the function is not used. For instance, agradient calculation is always performed at each major iteration for theHessian update. Therefore, given three points that bracket the minimum, it ispossible to use cubic interpolation, which is likely to be more accurate thanusing quadratic interpolation. Further efficiencies are possible if, instead ofusing bisection to bracket the minimum, extrapolation methods similar tothose used in the cubic polynomial method are used.Hence, the method that is used in lsqnonlin, lsqcurvefit, andfsolve is tofind three points that bracket the minimum and to use cubic interpolationto estimate the minimum at each line search. The estimation of step lengthat each minor iteration, j, is shown in the following graphs for a number ofpoint combinations. The left-most point in each graph represents the functionvalue and univariate gradient obtained at the last update. Theremaining points represent the points accumulated in the minor iterationsof the line search procedure.The terms and alpha sub c refer to the minimum obtained from arespective quadratic and cubic interpolation or extrapolation. For highlynonlinear functions, and can be negative, in which case they are set to avalue of so that they are always maintained to be positive. Cases 1 and 2use quadratic interpolation with two points and one gradient to estimate athird point that brackets the minimum. If this fails, cases 3 and 4 representthe possibilities for changing the step length when at least three points areavailable.When the minimum is finally bracketed, cubic interpolation is achieved usingone gradient and three function evaluations. If the interpolated point isgreater than any of the three used for the interpolation, then it is replacedwith the point with the smallest function value. Following the line search3-15