Introduction to Unconstrained Optimization - Scilab

A subclass of optimization problems is the problem where the inequality constraints
are made of bounds. The bound-constrained optimization problem is the
following,
min f(x)
x∈Rn (7)
s.t. x i,L ≤ x i ≤ x i,U , i = 1, m. (8)
where {x i,L } i=1,m (resp. {x i,U } i=1,m ) are the lower (resp. upper) bounds. Many
physical problems are associated with this kind of problem, because physical parameters
are generaly bounded.
1.3 Rosenbrock’s function
We now present a famous optimization problem, used in many demonstrations of
optimization methods. We will use this example consistently thoughout this document
and will analyze it with Scilab. Let us consider Rosenbrock’s function [13]
defined by
f(x) = 100(x 2 − x 2 1) 2 + (1 − x 1 ) 2 . (10)
The classical starting point is x 0 = (−1.2, 1) T where the function value is f(x 0 ) =
24.2. The global minimum of this function is x ⋆ = (1, 1) T where the function value
is f(x ⋆ ) = 0. The gradient is
( )
−400x1 (x
g(x) =
2 − x 2 1) − 2(1 − x 1 )
200(x 2 − x 2 (11)
(
1)
)
400x
3
=
1 − 400x 1 x 2 + 2x 1 − 2
−200x 2 . (12)
1 + 200x 2
(9)
The Hessian matrix is
H(x) =
( 1200x
2
1 − 400x 2 + 2 −400x 1
−400x 1 200
)
. (13)
The following rosenbrock function computes the function value f, the gradient
g and the Hessian matrix H of Rosenbrock’s function.
function [f, g, H] = rosenbrock ( x )
f = 100*( x(2) -x (1)^2)^2 + (1 -x (1))^2
g (1) = - 400*( x(2) -x (1)^2)* x (1) - 2*(1 - x (1))
g (2) = 200*( x(2) -x (1)^2)
H (1 ,1) = 1200* x (1)^2 - 400* x (2) + 2
H (1 ,2) = -400* x (1)
H (2 ,1) = H (1 ,2)
H (2 ,2) = 200
endfunction
It is safe to say that the most common practical optimization issue is related to a
bad implementation of the objective function. Therefore, we must be extra careful
7