What Is Optimization Toolbox?

3 Standard AlgorithmsNonlinear Least-Squares ImplementationFor a general survey of nonlinear least-squares methods, see Dennis [10].Specific details on the Levenberg-Marquardt method can be found inMoré [30]. Both the Gauss-Newton method and the Levenberg-Marquardtmethod are implemented in Optimization Toolbox. Details of theimplementations are discussed below:• “Gauss-Newton Implementation” on page 3-22• “Levenberg-Marquardt Implementation” on page 3-22Gauss-Newton ImplementationThe Gauss-Newton method is implemented using polynomial line searchstrategies similar to those discussed for unconstrained optimization. Insolving the linear least-squares problem, you can avoid exacerbation of theconditioning of the equations by using the QR decomposition of andapplying the decomposition to (using the MATLAB \ operator). Thisis in contrast to inverting the explicit matrix,,whichcancauseunnecessary errors to occur.Robustness measures are included in the method. These measures consist ofchanging the algorithm to the Levenberg-Marquardt method when either thestep length goes below a threshold value (1e-15 in this implementation) orwhen the condition number of is below 1e-10. The condition number is aratio of the largest singular value to the smallest.Levenberg-Marquardt ImplementationThe main difficulty in the implementation of the Levenberg-Marquardtmethod is an effective strategy for controlling the size of at each iterationso that it is efficient for a broad spectrum of problems. The method used inthis implementation is to estimate the relative nonlinearity of using alinear predicted sum of squares andacubiclyinterpolatedestimateofthe minimum .Inthiswaythesizeof is determined at each iteration.3-22