Introduction to Unconstrained Optimization - Scilab

1.2 What is an optimization problem ?
In the current section, we present the basic vocabulary of optimization.
Consider the following general constrained optimization problem.
min f(x)
x∈Rn (2)
s.t. h i (x) = 0, i = 1, m ′ , (3)
h i (x) ≥ 0, i = m ′ + 1, m. (4)
The following list presents the name of all the variables in the optimization
problem.
• The variables x ∈ R n can be called the unknowns, the parameters or, sometimes,
the decision variables. This is because, in many physical optimization
problems, some parameters are constants (the gravity constant for example)
and only a limited number of parameters can be optimized.
• The function f : R n → R is called the objective function. Sometimes, we also
call it the cost function.
• The number m ≥ 0 is the number of constraints and the functions h are the
constraints function. More precisely The functions {h i } i=1,m ′ are the equality
constraints functions and {h i } i=m ′ +1,m are inequality constraints functions.
Some additionnal notations are used in optimization and the following list present
the some of the most useful.
• A point which satisfies all the constraints is feasible. The set of points which
are feasible is called the feasible set. Assume that x ∈ R n is a feasible point.
Then the direction p ∈ R n is called a feasible direction if the point x+αp ∈ R n
is feasible for all α ∈ R.
• The gradient of the objective function is denoted by g(x) ∈ R n and is defined
as
( ∂f
g(x) = ∇f(x) = , . . . , ∂f ) T
. (5)
∂x 1 ∂x n
• The Hessian matrix is denoted by H(x) and is defined as
H ij (x) = (∇ 2 f) ij =
∂2 f
∂x i ∂x j
. (6)
If f is twice differentiable, then the Hessian matrix is symetric by equality of
mixed partial derivatives so that H ij = H ji .
The problem of finding a feasible point is the feasibility problem and may be
quite difficult itself.
6