Introduction to Unconstrained Optimization - Scilab

More documents

Recommendations

Info

f(x) f(y) x y Figure 13: Concave function f(x) x Figure 14: Nonconvex function Proposition 2.5. (First order condition of differentiable convex function) Let f be continously differentiable. Then f is convex over a convex set C if and only if for all x, y ∈ C. f(y) ≥ f(x) + g(x) T (y − x) (31) The proposition says that the graph of a convex function is above a line which slope is equal to the first derivative of f. The figure 15 presents a graphical analysis of the proposition. The first order condition can be associated with the Taylor expansion at the point x. The proposition tells that the first order Taylor approximation is under the graph of f, i.e. from a local approximation of f, we can deduce a global behaviour of f. As we shall see later, this makes convex as perfect function candidates for optimization. Proof. In the first part of the proof, let us assume that f is convex over the convex set C. Let us prove that the inequality 31 is satisfied. Assume that x, y are two point in C. Let θ be a scalar such that 0 < θ ≤ 1 (notice that θ is chosen strictly positive). Because C is a convex set, the point θy + (1 − θ)x is in the set C. This point can be written as x + θ(y − x). The idea of the proof is to compute the value of f at the two points x + θ(y − x) and x, and to form the factor (f(x + θ(y − x)) − f(x)) /θ. This fraction can be used to derive the inequality by making θ → 0, so that we let appear the dot product g(x) T (y − x). This situation is presented in figure 16. 20
T f(x)+g(x) (y-x) f(x) x Figure 15: First order condition on convex functions f(x) f(x+θ(y-x)) x x+θ(y-x) Figure 16: Proof of the first order condition on convex functions - part 1 21
Page 1 and 2: Introduction to Unconstrained Optim
Page 3 and 4: Copyright c○ 2008-2010 - Michael
Page 5 and 6: Optimization Discrete Parameters Co
Page 7 and 8: A subclass of optimization problems
Page 9 and 10: -->[ f , g , H ] = rosenbrock ( x0
Page 11 and 12: 3000 2500 2000 Z 1500 1000 500 0 2.
Page 13 and 14: f(x) strong local optimum weak loca
Page 15 and 16: The previous definition intuitively
Page 17 and 18: x 1 x 2 x 1 x 2 Figure 7: Convex se
Page 19: 0 0 Figure 11: Conic hulls f(x) f(y
Page 23 and 24: By hypothesis x = θy 1 + (1 − θ
Page 25 and 26: Proof. We assume that the Hessian m
Page 27 and 28: 5 4 3 2 1 Z 0 -1 -2 -3 -4 -5 5 4 3
Page 29 and 30: 2.0 1.5 1.0 20 0.5 10 0.0 5 2 2 5 -
Page 31 and 32: φ(α) α 2.0 1.5 -20 φ(α) -10 1.
Page 33 and 34: 6 20 4 2 0 -5 0 5 -2 -4 20 -6 -10 -
Page 35 and 36: 3 Answers to exercises 3.1 Answers
Page 37 and 38: C x 2 x 2 x 1 x 3 x 3 Figure 28: Co
Page 39 and 40: for all x, y ∈ C, then f is conve
Page 41 and 42: [2] Grégoire Allaire. Analyse num

Introduction to Unconstrained Optimization - Scilab

Create successful ePaper yourself

Delete template?

Save as template?