v2009.01.01 - Convex Optimization
v2009.01.01 - Convex Optimization v2009.01.01 - Convex Optimization
76 CHAPTER 2. CONVEX GEOMETRY A set of nonempty interior that has a supporting hyperplane at every point on its boundary, conversely, is convex. A convex set C ⊂ R n , for example, can be expressed as the intersection of all halfspaces partially bounded by hyperplanes supporting it; videlicet, [215, p.135] C = ⋂ a∈R n { y | a T y ≤ σ C (a) } (122) by the halfspaces theorem (2.4.1.1.1). There is no geometric difference between supporting hyperplane ∂H + or ∂H − or ∂H and 2.22 an ordinary hyperplane ∂H coincident with them. 2.4.2.6.2 Example. Minimization on a hypercube. Consider minimization of a linear function on a hypercube, given vector c minimize c T x x subject to −1 ≼ x ≼ 1 (123) This convex optimization problem is called a linear program 2.23 because the objective of minimization is linear and the constraints describe a polyhedron (intersection of a finite number of halfspaces and hyperplanes). Applying graphical concepts from Figure 22, Figure 24, and Figure 25, an optimal solution can be shown to be x ⋆ = − sgn(c) but is not necessarily unique. Because an optimal solution always exists at a hypercube vertex (2.6.1.0.1) regardless of the value of nonzero vector c [82], mathematicians see this geometry as a means to relax a discrete problem (whose desired solution is integer or combinatorial, confer Example 4.2.3.1.1). [206,3.1] [207] 2.4.2.6.3 Exercise. Unbounded below. Suppose instead we minimize over the unit hypersphere in Example 2.4.2.6.2; ‖x‖ ≤ 1. What is an expression for optimal solution now? Is that program still linear? Now suppose we instead minimize absolute value in (123). Are the following programs equivalent for some arbitrary real convex set C ? 2.22 If vector-normal polarity is unimportant, we may instead signify a supporting hyperplane by ∂H . 2.23 The term program has its roots in economics. It was originally meant with regard to a plan or to efficient organization of some industrial process. [82,2]
2.4. HALFSPACE, HYPERPLANE 77 (confer (459)) minimize |x| x∈R subject to −1 ≤ x ≤ 1 x ∈ C ≡ minimize x + + x − x + , x − subject to 1 ≥ x − ≥ 0 1 ≥ x + ≥ 0 x + − x − ∈ C (124) Many optimization problems of interest and some methods of solution require nonnegative variables. The method illustrated below splits a variable into its nonnegative and negative parts; x = x + − x − (extensible to vectors). Under what conditions on vector a and scalar b is an optimal solution x ⋆ negative infinity? minimize x + − x − x + ∈ R , x − ∈ R subject to x − ≥ 0 x + ≥ 0 [ ] a T x+ = b x − Minimization of the objective function 2.24 entails maximization of x − . 2.4.2.7 PRINCIPLE 3: Separating hyperplane (125) The third most fundamental principle of convex geometry again follows from the geometric Hahn-Banach theorem [215,5.12] [17,1] [110,I.1.2] that guarantees existence of a hyperplane separating two nonempty convex sets in R n whose relative interiors are nonintersecting. Separation intuitively means each set belongs to a halfspace on an opposing side of the hyperplane. There are two cases of interest: 1) If the two sets intersect only at their relative boundaries (2.6.1.3), then there exists a separating hyperplane ∂H containing the intersection but containing no points relatively interior to either set. If at least one of the two sets is open, conversely, then the existence of a separating hyperplane implies the two sets are nonintersecting. [53,2.5.1] 2) A strictly separating hyperplane ∂H intersects the closure of neither set; its existence is guaranteed when the intersection of the closures is empty and at least one set is bounded. [173,A.4.1] 2.24 The objective is the function that is argument to minimization or maximization.
- Page 25 and 26: ases (biorthogonal expansion). We e
- Page 27 and 28: 27 Figure 7: These bees construct a
- Page 29 and 30: that establish its membership to th
- Page 31 and 32: 31 appendices Provided so as to be
- Page 33 and 34: Chapter 2 Convex geometry Convexity
- Page 35 and 36: 2.1. CONVEX SET 35 2.1.2 linear ind
- Page 37 and 38: 2.1. CONVEX SET 37 2.1.6 empty set
- Page 39 and 40: 2.1. CONVEX SET 39 2.1.7.1 Line int
- Page 41 and 42: 2.1. CONVEX SET 41 (a) R 2 (b) R 3
- Page 43 and 44: 2.1. CONVEX SET 43 This theorem in
- Page 45 and 46: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 47 and 48: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 49 and 50: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 51 and 52: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 53 and 54: 2.2. VECTORIZED-MATRIX INNER PRODUC
- Page 55 and 56: 2.3. HULLS 55 Figure 16: Convex hul
- Page 57 and 58: 2.3. HULLS 57 The affine hull of tw
- Page 59 and 60: 2.3. HULLS 59 2.3.2 Convex hull The
- Page 61 and 62: 2.3. HULLS 61 In case k = N , the F
- Page 63 and 64: 2.3. HULLS 63 2.3.2.0.3 Exercise. C
- Page 65 and 66: 2.3. HULLS 65 Figure 20: A simplici
- Page 67 and 68: 2.4. HALFSPACE, HYPERPLANE 67 H + a
- Page 69 and 70: 2.4. HALFSPACE, HYPERPLANE 69 1 1
- Page 71 and 72: 2.4. HALFSPACE, HYPERPLANE 71 Recal
- Page 73 and 74: 2.4. HALFSPACE, HYPERPLANE 73 C H
- Page 75: 2.4. HALFSPACE, HYPERPLANE 75 2.4.2
- Page 79 and 80: 2.5. SUBSPACE REPRESENTATIONS 79 Ra
- Page 81 and 82: 2.5. SUBSPACE REPRESENTATIONS 81 If
- Page 83 and 84: 2.6. EXTREME, EXPOSED 83 2.6 Extrem
- Page 85 and 86: 2.6. EXTREME, EXPOSED 85 A B C D Fi
- Page 87 and 88: 2.7. CONES 87 2.6.1.3.1 Definition.
- Page 89 and 90: 2.7. CONES 89 0 Figure 30: Boundary
- Page 91 and 92: 2.7. CONES 91 2.7.2 Convex cone We
- Page 93 and 94: 2.7. CONES 93 Then a pointed closed
- Page 95 and 96: 2.7. CONES 95 A pointed closed conv
- Page 97 and 98: 2.8. CONE BOUNDARY 97 So the ray th
- Page 99 and 100: 2.8. CONE BOUNDARY 99 2.8.1.1 extre
- Page 101 and 102: 2.8. CONE BOUNDARY 101 2.8.2 Expose
- Page 103 and 104: 2.8. CONE BOUNDARY 103 From Theorem
- Page 105 and 106: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 107 and 108: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 109 and 110: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 111 and 112: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 113 and 114: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 115 and 116: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 117 and 118: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 119 and 120: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 121 and 122: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 123 and 124: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
- Page 125 and 126: 2.9. POSITIVE SEMIDEFINITE (PSD) CO
2.4. HALFSPACE, HYPERPLANE 77<br />
(confer (459))<br />
minimize |x|<br />
x∈R<br />
subject to −1 ≤ x ≤ 1<br />
x ∈ C<br />
≡<br />
minimize x + + x −<br />
x + , x −<br />
subject to 1 ≥ x − ≥ 0<br />
1 ≥ x + ≥ 0<br />
x + − x − ∈ C<br />
(124)<br />
Many optimization problems of interest and some methods of solution<br />
require nonnegative variables. The method illustrated below splits a variable<br />
into its nonnegative and negative parts; x = x + − x − (extensible to vectors).<br />
Under what conditions on vector a and scalar b is an optimal solution x ⋆<br />
negative infinity?<br />
minimize x + − x −<br />
x + ∈ R , x − ∈ R<br />
subject to x − ≥ 0<br />
x + ≥ 0<br />
[ ]<br />
a T x+<br />
= b<br />
x −<br />
Minimization of the objective function 2.24 entails maximization of x − . <br />
2.4.2.7 PRINCIPLE 3: Separating hyperplane<br />
(125)<br />
The third most fundamental principle of convex geometry again follows from<br />
the geometric Hahn-Banach theorem [215,5.12] [17,1] [110,I.1.2] that<br />
guarantees existence of a hyperplane separating two nonempty convex sets<br />
in R n whose relative interiors are nonintersecting. Separation intuitively<br />
means each set belongs to a halfspace on an opposing side of the hyperplane.<br />
There are two cases of interest:<br />
1) If the two sets intersect only at their relative boundaries (2.6.1.3), then<br />
there exists a separating hyperplane ∂H containing the intersection but<br />
containing no points relatively interior to either set. If at least one of<br />
the two sets is open, conversely, then the existence of a separating<br />
hyperplane implies the two sets are nonintersecting. [53,2.5.1]<br />
2) A strictly separating hyperplane ∂H intersects the closure of neither<br />
set; its existence is guaranteed when the intersection of the closures is<br />
empty and at least one set is bounded. [173,A.4.1]<br />
2.24 The objective is the function that is argument to minimization or maximization.