The Steepest Descent Algorithm for Unconstrained Optimization and ...
The Steepest Descent Algorithm for Unconstrained Optimization and ...
The Steepest Descent Algorithm for Unconstrained Optimization and ...
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
1 T<br />
= 1 x T Qx + αd T Qx + α 2 d T Qd + q x + αq T d<br />
2 2<br />
1<br />
= f (x) − αd T d + α 2 d T Qd.<br />
2<br />
Optimizing the value of α in this last expression yields<br />
d T d<br />
α = d T Qd ,<br />
<strong>and</strong> the next iterate of the algorithm then is<br />
<strong>and</strong><br />
′ d T d<br />
x = x + αd = x + d,<br />
d T Qd<br />
1 (d T d) 2<br />
f (x ′ )= f (x + αd) = f (x) − αd T d + 1 α 2 d T Qd = f (x) − .<br />
2 2 d T Qd<br />
<strong>The</strong>re<strong>for</strong>e,<br />
f (x ′ ) − f (x ∗ ) f (x) − 1 (d T d) 2<br />
2<br />
− f (x )<br />
d = T Qd<br />
f (x) − f (x ∗ ) f (x) − f (x ∗ )<br />
∗<br />
=1 −<br />
=1 −<br />
1 (d T d) 2<br />
2 d T Qd<br />
1 x T Qx + q T x + 1<br />
2<br />
2 qT Q −1 q<br />
1 (d T d) 2<br />
2 d T Qd<br />
1<br />
2 (Qx + q)T Q −1 (Qx + q)<br />
(d T d) 2<br />
=1 − (d T Qd)(d T Q −1 d)<br />
1<br />
=1 − β<br />
where<br />
(d T Qd)(d T Q −1 d)<br />
β = .<br />
(d T d) 2<br />
6