Introduction to Unconstrained Optimization - Scilab

f(x)
y 1
x y 2
Figure 17: Proof of the first order condition on convex functions - part 2
By the convexity hypothesis of f, we deduce:
f(θy + (1 − θ)x) ≤ θf(y) + (1 − θ)f(x). (32)
The left hand side can be written as f(y + θ(x − y)) while the right hand side of the
previous inequality can be written in the form f(x) + θ(f(y) − f(x)). This leads to
the following inequality:
f(x + θ(y − x)) ≤ f(x) + θ(f(y) − f(x)). (33)
We move the term f(x) to the left hand side, divide by θ > 0 and get the inequality:
f(x + θ(y − x)) − f(x)
θ
≤ f(y) − f(x). (34)
We can take the limit as θ → 0 since, by hypothesis, the function f is continuously
differentiable. Therefore,
g(x) T (y − x) ≤ f(y) − f(x), (35)
which concludes the first part of the proof.
In the second part of the proof, let us assume that the function f satisfies the
inequality 31. Let us prove that f is convex.
Let y 1 , y 2 be two points in C and let θ be a scalar such that 0 ≤ θ ≤ 1. The
idea is to use the inequality 31, with the carefully chosen point x = θy 1 + (1 − θ)y 2 .
This idea is presented in figure 17.
The first order condition 31 at the two points y 1 and y 2 gives:
f(y 1 ) ≥ f(x) + g(x) T (y 1 − x) (36)
f(y 2 ) ≥ f(x) + g(x) T (y 2 − x) (37)
We can form a convex combination of the two inequalities and get:
θf(y 1 ) + (1 − θ)f(y 2 ) ≥ θf(x) + (1 − θ)f(x) (38)
The previous inequality can be simplified into:
+g(x) T (θ(y 1 − x) + (1 − θ)(y 2 − x)) . (39)
θf(y 1 ) + (1 − θ)f(y 2 ) ≥ f(x) + g(x) T (θy 1 + (1 − θ)y 2 − x) . (40)
22