The Steepest Descent Algorithm for Unconstrained Optimization and ...

1 T
= 1 x T Qx + αd T Qx + α 2 d T Qd + q x + αq T d
2 2
1
= f (x) − αd T d + α 2 d T Qd.
2
Optimizing the value of α in this last expression yields
d T d
α = d T Qd ,
and the next iterate of the algorithm then is
and
′ d T d
x = x + αd = x + d,
d T Qd
1 (d T d) 2
f (x ′ )= f (x + αd) = f (x) − αd T d + 1 α 2 d T Qd = f (x) − .
2 2 d T Qd
Therefore,
f (x ′ ) − f (x ∗ ) f (x) − 1 (d T d) 2
2
− f (x )
d = T Qd
f (x) − f (x ∗ ) f (x) − f (x ∗ )
∗
=1 −
=1 −
1 (d T d) 2
2 d T Qd
1 x T Qx + q T x + 1
2
2 qT Q −1 q
1 (d T d) 2
2 d T Qd
1
2 (Qx + q)T Q −1 (Qx + q)
(d T d) 2
=1 − (d T Qd)(d T Q −1 d)
1
=1 − β
where
(d T Qd)(d T Q −1 d)
β = .
(d T d) 2
6