Introduction to Unconstrained Optimization - Scilab

More documents

Recommendations

Info

1 Overview In this section, we analyze optimization problems and define the associated vocabulary. We introduce level sets and separate local and global optimums. We emphasize the use of contour plots in the context of unconstrained and constrained optimization. 1.1 Classification of optimization problems In this document, we consider optimization problems in which we try to minimize a cost function min f(x) (1) x∈Rn with or without constraints. Several properties of the problem to solve may be taken into account by the numerical algorithms: • The unknown may be a vector of real or integer values. • The number of unknowns may be small (from 1 to 10 - 100), medium (from 10 to 100 - 1 000) or large (from 1 000 - 10 000 and above), leading to dense or sparse linear systems. • There may be one or several cost functions (multi-objective optimization). • The cost function may be smooth or non-smooth. • There may be constraints or no constraints. • The constraints may be bounds constraints, linear or non-linear constraints. • The cost function can be linear, quadratic or a general non linear function. An overview of optimization problems is presented in figure 1. In this document, we will be concerned mainly with continuous parameters and problems with one objective function only. From that point, smooth and nonsmooth problems require very different numerical methods. It is generally believed that equality constrained optimization problems are easier to solve than inequality constrainted problems. This is because computing the set of active constraints at optimum is a difficult problem. The size of the problem, i.e. the number n of parameters, is also of great concern with respect to the design of optimization algorithms. Obviously, the rate of convergence is of primary interest when the number of parameters is large. Algorithms which require n iterations, like BFGS’s methods for example, will not be efficient if the number of iterations to perform is much less than n. Moreover, several algorithms (like Newton’s method for example), require too much storage to be of a practical value when n is so large that n × n matrices cannot be stored. Fortunately, good algorithms such as conjugate gradient methods and BFGS with limited memory methods are specifically designed for this purpose. 4
Optimization Discrete Parameters Continuous Parameters Several Objectives One Objective Non Smooth Smooth Without Constraints With Constraints Bounds Constraints Linear Constraints Non-linear Constraints Linear Objective Quadratic Objective Non-linear Objective 1-10 Unknowns 10-100 Unknowns >100 Unknowns Figure 1: Classes of optimization problems 5
Page 1 and 2: Introduction to Unconstrained Optim
Page 3: Copyright c○ 2008-2010 - Michael
Page 7 and 8: A subclass of optimization problems
Page 9 and 10: -->[ f , g , H ] = rosenbrock ( x0
Page 11 and 12: 3000 2500 2000 Z 1500 1000 500 0 2.
Page 13 and 14: f(x) strong local optimum weak loca
Page 15 and 16: The previous definition intuitively
Page 17 and 18: x 1 x 2 x 1 x 2 Figure 7: Convex se
Page 19 and 20: 0 0 Figure 11: Conic hulls f(x) f(y
Page 21 and 22: T f(x)+g(x) (y-x) f(x) x Figure 15:
Page 23 and 24: By hypothesis x = θy 1 + (1 − θ
Page 25 and 26: Proof. We assume that the Hessian m
Page 27 and 28: 5 4 3 2 1 Z 0 -1 -2 -3 -4 -5 5 4 3
Page 29 and 30: 2.0 1.5 1.0 20 0.5 10 0.0 5 2 2 5 -
Page 31 and 32: φ(α) α 2.0 1.5 -20 φ(α) -10 1.
Page 33 and 34: 6 20 4 2 0 -5 0 5 -2 -4 20 -6 -10 -
Page 35 and 36: 3 Answers to exercises 3.1 Answers
Page 37 and 38: C x 2 x 2 x 1 x 3 x 3 Figure 28: Co
Page 39 and 40: for all x, y ∈ C, then f is conve
Page 41 and 42: [2] Grégoire Allaire. Analyse num

Introduction to Unconstrained Optimization - Scilab

Create successful ePaper yourself

Delete template?

Save as template?