What Is Optimization Toolbox?

3 Standard AlgorithmsLeast-Squares Optimization• “Introduction” on page 3-18• “Gauss-Newton Method” on page 3-20• “Levenberg-Marquardt Method” on page 3-21• “Nonlinear Least-Squares Implementation” on page 3-22IntroductionThe line search procedures used in conjunction with a quasi-Newton methodare used as part of the nonlinear least-squares (LS) optimization routines,lsqnonlin and lsqcurvefit. In the least-squares problem a function f(x) isminimized that is a sum of squares.Problems of this type occur in a large number of practical applications,especially when fitting model functions to data, i.e., nonlinear parameterestimation. They are also prevalent in control where you want the output,y(x,y), to follow some continuous model trajectory, , for vector x andscalar t. This problem can be expressed as(3-14)where and are scalar functions.When the integral is discretized using a suitable quadrature formula, theabove can be formulated as a least-squares problem:(3-15)where and include the weights of the quadrature scheme. Note that inthis problem the vector F(x) is(3-16)3-18