What Is Optimization Toolbox?

4 Large-Scale AlgorithmsLinear Least-SquaresIn this case the function f(x) tobesolvedisThe algorithm generates strictly feasible iterates converging, in the limit, toa local solution. Each iteration involves the approximate solution of a largelinear system (of order n, wheren is the length of x). The iteration matriceshave the structure of the matrix C. In particular, the method of preconditionedconjugate gradients is used to approximately solve the normal equations, i.e.,although the normal equations are not explicitly formed.The subspace trust-region method is used to determine a search direction.However, instead of restricting the stepto(possibly)onereflectionstep,as in the nonlinear minimization case, a piecewisereflectivelinesearchisconducted at each iteration, as in the quadratic case. See [5] for details of theline search. Ultimately, the linear systems represent a Newton approachcapturing the first-order optimality conditions at the solution, resulting instrong local convergence rates.4-14