Introduction to Unconstrained Optimization - Scilab

spec
Computes the eigenvalue and the eigenvectors.
D=spec(A) Computes the eigenvalues D of A.
[R,D]=spec(A) Computes the right eigenvectors R of A.
Figure 21: The spec function.
-->t = 0.5;
-->c = t*a + (1 -t)*b;
-->fa = f(a(1) ,a (2));
-->fb = f(b(1) ,b (2));
-->fc = f(c(1) ,c (2));
-->[ fc t*fa +(1 -t)* fb]
ans =
1. 0.5
Therefore, the function f is nonconvex, although it defines a convex set.
2.6 Quadratic functions
In this section, we consider specific examples of quadratic functions and analyse the
sign of their eigenvalues to see if a stationnary point exist and is a local minimum.
Let us consider the following quadratic function
f(x) = b T x + 1 2 xT Hx, (56)
where x ∈ R n , the vector b ∈ R n is a given vector and H is a n × n matrix.
In the following examples, we use the spec function which is presented in figure
21. This function allows to compute the eigenvalues, i.e. the spectrum, and the
eigenvectors of a given matrix. This function allows to access to several Lapack
routines which can take into account for symetric or non-symetric matrices.
In the context of numerical optimization, we often compute the eigenvalues of
the Hessian matrix H of the objective function, which implies that we consider only
real symetric matrices. In this case, the eigenvalues and the eigenvectors are real
matrices (as opposed to complex matrices).
Example 2.2 (A quadratic function for which the Hessian has positive eigenvalues.)
Assume that b = 0 and the Hessian matrix is
( ) 5 3
H =
(57)
3 2
The eigenvalues are approximately equal to (λ 1 , λ 2 ) ≈ (0.1458980, 6.854102) and are
both positive. Hence, this quadratic function is strictly convex.
The following script produces the contours of the corresponding quadratic function.
This produces the plot presented in figure 22.
function f = quadraticdefpos ( x1 , x2 )
x = [x1 x2]’
H = [5 3; 3 2]
f = x.’ * H * x;
28