What Is Optimization Toolbox?

3 Standard AlgorithmsGauss-Newton MethodIn the Gauss-Newton method, a search direction, , is obtained at each majoriteration, k, that is a solution of the linear least-squares problem.(3-18)The direction derived from this method is equivalent to the Newton directionwhen the terms of Q(x) can be ignored. The search direction can be usedas part of a line search strategy to ensure thatateachiterationthefunctionf(x) decreases.Consider the efficiencies that are possible with the Gauss-Newton method.Gauss-Newton Method on Rosenbrock’s Function on page 3-20 shows the pathto the minimum on Rosenbrock’s function when posed as a least-squaresproblem. The Gauss-Newton method converges after only 48 functionevaluations using finite difference gradients, compared to 140 iterations usingan unconstrained BFGS method.Figure 3-3:Gauss-Newton Method on Rosenbrock’s FunctionThe Gauss-Newton method often encounters problems when the second-orderterm Q(x) is significant. A method that overcomes this problem is theLevenberg-Marquardt method.3-20