Introduction to Unconstrained Optimization - Scilab

f(x)
strong local
optimum
weak local
optimum
strong global
optimum
x
Figure 5: Different types of optimum – strong local, weak local and strong global.
These types of optimum are presented in figure 5.
Definition 1.1. (Strong local minimum) The point x ⋆ is a strong local minimum
of the constrained optimization problem 2–4 if there exists δ > 0 such that the two
following conditions are satisfied :
• f(x) is defined on N(x ⋆ , δ), and
• f(x ⋆ ) < f(y), ∀y ∈ N(x ⋆ , δ), y ≠ x ⋆ .
Definition 1.2. (Weak local minimum) The point x ⋆ is a weak local minimum of
the constrained optimization problem 2–4 if there exists δ > 0 such that the three
following conditions are satisfied
• f(x) is defined on N(x ⋆ , δ),
• f(x ⋆ ) ≤ f(y), ∀y ∈ N(x ⋆ , δ),
• x ⋆ is not a strong local minimum.
Definition 1.3. (Strong global minimum) The point x ⋆ is a strong global minimum
of the constrained optimization problem 2–4 if there exists δ > 0 such that the two
following conditions are satisfied :
• f(x) is defined on the set of feasible points,
• f(x ⋆ ) < f(y), for all y ∈ R n feasible and y ≠ x ⋆ .
Most algorithms presented in this document are searching for a strong local
minimum. The global minimum may be found in particular situations, for example
when the cost function is convex. The difference between weak and strong local
minimum is also of very little practical use, since it is difficult to determine what
are the values of the function for all points except the computed point x ⋆ .
In practical situations, the previous definitions does not allow to get some insight
about a specific point x ⋆ . This is why we will derive later in this document first
order and second order necessary conditions, which are computable characteristics
of the optimum.
13