Introduction to Unconstrained Optimization - Scilab

for all x, y ∈ C, then f is convex over the convex set C. This proof is given in a slightly more
complex form in [2], chapter 10 ”Conditions d’optimalité”, section 10.1.2 ”Différentiabilité”.
Assume that x, y are two points in the convex set C. Since C is a convex set, the point
x + θ(y − x) = θy + (1 − θ)x is also in C, for all θ so that 0 ≤ θ ≤ 1. Let us define the following
function
φ(θ) = f(x + θ(y − x)), (84)
for all θ so that 0 ≤ θ ≤ 1.
The idea of the proof is based on the fact that the convexity of φ with respect to θ ∈ [0, 1] is
equivalent to the convexity of f with respect to x ∈ C. We will prove that
φ ′ (t) − φ ′ (s) ≥ 0, (85)
for all real values s and t so that t ≥ s. That inequality means that φ ′ is an increasing function,
which property is associated with a convex function. Then we will integrate the inequality over
well chosen ranges of s and t so that the convexity of φ appear.
To prove the inequality 85, let us define two real values s, t ∈ R. We apply the inequality 83
with the two points x + t(y − x) and x + s(y − x). By the convexity of C, these points are in C
so that we get
[g(x + t(y − x)) − g(x + s(y − x))] T (x + t(y − x) − x − s(y − x) ≥ 0. (86)
The left hand side can be simplified into (t−s)(φ ′ (t)−φ ′ (s)) ≥ 0. For t ≥ s, the previous inequality
leads to 85.
We now integrate the inequality 85 for t ∈ [θ, 1] and s ∈ [0, θ] so that the order t ≥ s is satisfied
and we get
∫ ∫
(φ ′ (t) − φ ′ (s)) dtds ≥ 0. (87)
t∈[θ,1]
s∈[0,θ]
We first compute the following two integrals
∫ ∫
φ ′ (t)dtds =
s∈[0,θ]
t∈[θ,1]
∫
s∈[0,θ]
(φ(1) − φ(θ))ds (88)
= θ(φ(1) − φ(θ)), (89)
and
∫ ∫
∫
φ ′ (s)dsdt = (φ(θ) − φ(0))dt (90)
t∈[θ,1] s∈[0,θ]
t∈[θ,1]
= (1 − θ)(φ(θ) − φ(0)). (91)
By plugging the two integrals 89 and 91 into the inequality 87, we get
which simplifies to
θ(φ(1) − φ(θ)) − (1 − θ)(φ(θ) − φ(0)) ≥ 0, (92)
θφ(1) + (1 − θ)φ(0) − φ(θ) ≥ 0. (93)
which proves that φ is convex. If we plug the definition 84 of φ and the equalities φ(0) = f(x) and
φ(1) = f(y), we find the convexity inequality for f
which concludes the proof.
θf(y) + (1 − θ)f(x) ≤ f(θy + (1 − θ)x), (94)
Answer of Exercise 2.6 (Hessian of Rosenbrock’s function) Let us prove that the Hessian
matrix of Rosenbrock’s function is positive definite at the point x = (1, 1) T . In the following script,
we define the point x and use the rosenbrock function to compute the associated Hessian matrix.
Then we use the spec function in order to compute the eigenvalues.
39