Introduction to Unconstrained Optimization - Scilab

Proof. We assume that the Hessian matrix is positive definite. Let us prove that
the function f is convex. Since f is twice continuously differentiable, we can use the
following Taylor expansion of f. By the proposition 1.7, there exists a θ satisfying
0 ≤ θ ≤ 1 so that
f(y) = f(x) + g(x ⋆ ) T (y − x) + 1 2 (y − x)T H(x + θ(y − x))(y − x), (51)
for any x, y ∈ C. Since H is positive definite, the scalar (y−x) T H(x+θ(y−x))(y−x)
is positive, which leads to the inequality
f(y) ≥ f(x) + g(x ⋆ ) T (y − x). (52)
By proposition 2.5, this proves that f is convex on C.
Assume that f is convex on the convex set C and let us prove that the Hessian
matrix is positive definite. We assume that the Hessian matrix is not positive definite
and show that it leads to a contradiction. The hypothesis that the Hessian matrix
is not positive definite implies that there exists a point x ∈ C vector p ∈ C such
that p T H(x)p < 0. Let us define y = x + p. By the proposition 1.7, there exists
a θ satisfying 0 ≤ θ ≤ 1 so that the equality 51 holds. Since C is open and f is
twice continuously differentiable, this inequality is also true in a neighbourhood of
x. More formaly, this implies that we can choose the vector p to be small enough
such that p T H(x + θp)p < 0 for any θ ∈ [0, 1]. Therefore, the equality 51 implies
f(y) < f(x) + g(x ⋆ ) T (y − x). (53)
By proposition 2.5, the previous inequality contradicts the hypothesis that f is
convex. Therefore, the Hessian matrix H is positive definite, which concludes the
proof.
2.5 Examples of convex functions
In this section, we give several examples of univariate and multivariate convex functions.
We also give an example of a nonconvex function which defines a convex
set.
Example 2.1 Consider the quadratic function f : R n → R given by
f(x) = f 0 + g T x + 1 2 xT Ax, (54)
where f 0 ∈ R, g ∈ R n and A ∈ R n×n . For any x ∈ R n , the Hessian matrix H(x) is
equal to the matrix A. Therefore, the quadratic function f is convex if and only if
the matrix A is positive definite. The quadratic function f is strictly convex if and
only if the matrix A is strictly positive definite. We will review quadratic functions
in more depth in the next section.
There are many other examples of convex functions. Obviously, any linear function
is convex. The exp(ax) function is convex on R, for any a ∈ R. The function x a
is convex on R for any a > 0. The function − log(x) is convex on R ++ = {x > 0}.
The function x log(x) is convex on R ++ . The following script produces the two plots
presented in figure 19.
25