Introduction to Unconstrained Optimization - Scilab

By hypothesis x = θy 1 + (1 − θ)y 2 , so that the inequality can be simplified again
and can be written as:
θf(y 1 ) + (1 − θ)f(y 2 ) ≥ f(θy 1 + (1 − θ)y 2 ), (41)
which shows that the function f is convex and concludes the proof.
We shall find another form of the first order condition for convexity with simple
algebraic manipulations. Indeed, the inequality 31 can be written g(x) T (y − x) ≤
f(y) − f(x). Multiplying both terms by −1 yields
g(x) T (x − y) ≥ f(x) − f(y). (42)
If we interchange the role of x and y in 42, we get g(y) T (y − x) ≥ f(y) − f(x). The
left hand side can be transformed so that the term x − y appear and we find that:
− g(y) T (x − y) ≥ f(y) − f(x). (43)
We can now take the sum of the two inequalities 42 and 43 and finally get:
(g(x) − g(y)) T (x − y) ≥ 0. (44)
This last derivation proved one part of the following proposition.
Proposition 2.6. (First order condition of differentiable convex function - second
form) Let f be continously differentiable. Then f is convex over a convex set C if
and only if
for all x, y ∈ C.
(g(x) − g(y)) T (x − y) ≥ 0 (45)
The second part of the proof is given as an exercise. The proof is based on an
auxiliary function that we are going to describe in some detail.
Assume that x, y are two points in the convex set C. Since C is a convex set,
the point x + θ(y − x) = θy + (1 − θ)x is also in C, for all θ so that 0 ≤ θ ≤ 1. Let
us define the following function
φ(θ) = f(x + θ(y − x)), (46)
for all θ so that 0 ≤ θ ≤ 1.
The function φ should really be denoted by φ x,y , because it actually depends
on the two points x, y ∈ C. Despite this lack of accuracy, we will keep our current
notation for simplicity reasons, assuming that the points x, y ∈ C are kept constant..
By definition, the function φ is so that the two following equalities are satisfied:
φ(0) = f(x), φ(1) = f(y). (47)
The function f is convex over the convex set C if and only if the function φ
satisfies the following inequality
φ(θ) ≥ θφ(1) + (1 − θ)φ(0), (48)
23