The Steepest Descent Algorithm for Unconstrained Optimization and ...

It is elementary to show that
Proposition 6.1 h ′ (0) < 0.
q.e.d.
′
h (α) = ∇f (¯ x + αd) ¯ T d. ¯
Because h(α) is a convex function of α, wealsohave:
Proposition 6.2 h ′ (α) is a monotone increasing function α. of
q.e.d.
Figure 5 shows an example of convex function of two variables to be
optimized. Figure 6 shows the function h(α) obtained by restricting the
function of Figure 5 to the line shown in that figure. Note from Figure 6 that
h(α) is convex. Therefore its first derivative h ′ (α) will be a monotonically
increasing function. This is shown in Figure 7.
Because h ′ (α) is a monotonically increasing function, we can approxiα,
¯ the point that satisfies h ′ (¯ α) = 0, by a suitable bisection
mately compute
method. Suppose that we know a value α ˆ that h ′ (ˆ α) > 0. Since h ′ (0) < 0
and h ′ (ˆ α) > 0, the mid-value α ˜ = 0+ 2α ˆ is a suitable test-point. Note the
following:
• If h ′ (˜α) = 0, we are done.
• If h ′ (˜ α) > 0, we can now bracket α ¯ in the interval (0, α). ˜
• If h ′ (˜ α) < 0, we can now bracket α ¯ in the interval ( α, ˜ α). ˆ
This leads to the following bisection algorithm for minimizing h(α) = f (¯x +
¯ αd) by solving the equation h ′ (α) ≈ 0.
Step 0. Set k = 0. Set α l := 0 and α u := ˆα.
α = αu+α l
Step k. Set ˜ and compute h ′ 2
(˜ α).
18