exercise sheet - Institut fuer Mathematik - HU Berlin

HUMBOLDT–UNIVERSITÄT ZU BERLINMathematisch-Naturwissenschaftliche Fakultät IIInstitut für MathematikProf. Andreas Griewank PhDSebastian WalterHumboldt-Universität zu Berlin, Institut für Mathematik, Unter den Linden 6, D-10099 BerlinBerlin, May 29, 2008Exercises for the lecture “Nonlinear Optimization”Assignment 2 (Return Theoretical Part on Wednesday, May 28th 2008)(Return Implementation Part on Wednesday, June 4th 2008)1 Verification of the Sherman-Morrison-Woodbury FormulaVerify the Sherman-Morrison formula Proposition 2.4 (ii) from the lecture. I.e.• Show if B + = B + UCV T and det(C −1 + V T B −1 U) ≠ 0 thenB −1+ = B−1 − B −1 U(C −1 + V T B −1 U) −1 V T B −1 ,where B ∈ R n×n , U,V ∈ R n×p and C ∈ R p×p .2 Elementary Properties of Convergence Rates• Show that for any convergent sequence x k → x ∗ ∈ R n with x k ≠ x ∗ ∀kwhereR({x k } ∞ k=0 ) ≤ Q({x k} ∞ k=0 ) ,] 1k[R({x k } ∞ ‖xk − x ∗ ‖k=0 ) := lim supk→∞ ‖x 0 − x ∗ ‖[Q({x k } ∞ ‖xk+1 − x ∗ ‖k=0 ) := lim supk→∞ ‖x k − x ∗ ‖I.e. you show that Q-[super]linear convergence implies R-[super]linear convergence.3 Comparison Between BFGS and CGConsider the quadratic functionwhere x,b ∈ R n , A ∈ R n×n and c ∈ R.f(x) = 1 2 xT Ax + b T x + c , (3.1)• Show that the BFGS method with B 0 = I n generates iteratesx k ∈ x 0 + K k (g 0 ,A) ,where K k (g 0 ,A) := span{g 0 ,Ag 0 ,...,A k−1 g 0 } is the Krylov space.Hint: Check by induction that H k = I + ∆H k = B −1kwith ∆H k a symmetric, and thusself-adjoint operator of finite rank, whose range is contained in K k+1 (g 0 ,A).].1

4 PRACTICAL PART4 Practical Part• Implement the conjugate gradient (CG) method.• Compare BFGS, CG and steepest descent. Use these methods to solve the linear systemAx = b, whereA = U T diag(λ 1 ,... ,λ n )U .U = (I n − 2uu T ) is an orthonormal matrix for ‖u‖ = 1. Pick a u with nonzero elementsu i ≠ 0. Select the right-hand side to b = [1,... ,1] T . Tryλ i = 1 + i/nλ i = 1 + 1/iλ i = i .Use n = 100. Calculate the minimizer x ∗ by hand and and plot ‖x k − x ∗ ‖, ‖x k − x ∗ ‖ 1/kand (‖∇f(x k )‖) 1/k . Interpret the plots and explain the relation to the definition of the R-convergence rate. Compare the iterates x k of CG, BFGS and steepest descent. Interpretthe results.• Implement Fletcher-Reeves (FR-)CG and Polak-Ribière-Polyak (PRP-)CG for nonquadraticfunctions.• Test FR-CG and PRP-CG on the Rosenbrock function defined on the last exercise sheet.phone: 030/2093-5820 fax: 030/2093-5848e-mail: griewank@math.hu-berlin.deInternet-Seite: http://www.math.hu-berlin.de/∼gaggle/S08/NLOPTwalter@math.hu-berlin.de2