What Is Optimization Toolbox?

Least-Squares OptimizationLevenberg-Marquardt MethodThe Levenberg-Marquardt [27], and [29] method uses a search direction thatis a solution of the linear set of equations(3-19)where the scalar controls both the magnitude and direction of .Whenis zero, the direction is identical to that of the Gauss-Newton method.As tends to infinity, tends toward a vector of zeros and a steepestdescent direction. This implies that for some sufficiently large ,thetermholds true. The term can therefore be controlled toensure descent even when second-order terms, which restrict the efficiency ofthe Gauss-Newton method, are encountered.The Levenberg-Marquardt method therefore uses a search direction that isa cross between the Gauss-Newton direction and the steepest descent. Thisis illustrated in Figure 3-4, Levenberg-Marquardt Method on Rosenbrock’sFunction. The solution for Rosenbrock’s function converges after 90 functionevaluations compared to 48 for the Gauss-Newton method. The poorerefficiency is partly because the Gauss-Newton method is generally moreeffective when the residual is zero at the solution. However, such informationis not always available beforehand, and the increased robustness of theLevenberg-Marquardt method compensates for its occasional poorer efficiency.Figure 3-4:Levenberg-Marquardt Method on Rosenbrock’s Function3-21