Chapter 3 Geometry of convex functions - Meboo Publishing ...
Chapter 3 Geometry of convex functions - Meboo Publishing ... Chapter 3 Geometry of convex functions - Meboo Publishing ...
222 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONS Then (498) is equivalent to minimize c c∈R , x∈R n , a∈R 2n subject to x = [I −I ]a a T 1 = c a ≽ 0 Ax = b ≡ minimize a∈R 2n ‖a‖ 1 subject to [A −A ]a = b a ≽ 0 (500) where x ⋆ = [I −I ]a ⋆ . (confer (495)) Significance of this result: (confer p.375) Any vector 1-norm minimization problem may have its variable replaced with a nonnegative variable of twice the length. All other things being equal, nonnegative variables are easier to solve for sparse solutions. (Figure 69, Figure 70, Figure 97) The compressed sensing problem becomes easier to interpret; e.g., for A∈ R m×n minimize ‖x‖ 1 x subject to Ax = b x ≽ 0 ≡ minimize 1 T x x subject to Ax = b x ≽ 0 (501) movement of a hyperplane (Figure 26, Figure 30) over a bounded polyhedron always has a vertex solution [92, p.22]. Or vector b might lie on the relative boundary of a pointed polyhedral cone K = {Ax | x ≽ 0}. In the latter case, we find practical application of the smallest face F containing b from 2.13.4.3 to remove all columns of matrix A not belonging to F ; because those columns correspond to 0-entries in vector x . 3.3.0.0.2 Exercise. Combinatorial optimization. A device commonly employed to relax combinatorial problems is to arrange desirable solutions at vertices of bounded polyhedra; e.g., the permutation matrices of dimension n , which are factorial in number, are the extreme points of a polyhedron in the nonnegative orthant described by an intersection of 2n hyperplanes (2.3.2.0.4). Minimizing a linear objective function over a bounded polyhedron is a convex problem (a linear program) that always has an optimal solution residing at a vertex. What about minimizing other functions? Given some nonsingular matrix A , geometrically describe three circumstances under which there are
3.3. PRACTICAL NORM FUNCTIONS, ABSOLUTE VALUE 223 likely to exist vertex solutions to minimize x∈R n ‖Ax‖ 1 subject to x ∈ P optimized over some bounded polyhedron P . 3.10 (502) 3.3.1 k smallest entries Sum of the k smallest entries of x∈ R n is the optimal objective value from: for 1≤k ≤n n∑ n∑ π(x) i = minimize x T y π(x) i = maximize k t + 1 T z y∈R n i=n−k+1 z∈R n , t∈R subject to 0 ≼ y ≼ 1 ≡ subject to x ≽ t1 + z 1 T y = k z ≼ 0 (503) i=n−k+1 k∑ i=1 which are dual linear programs, where π(x) 1 = max{x i , i=1... n} where π is a nonlinear permutation-operator sorting its vector argument into nonincreasing order. Finding k smallest entries of an n-length vector x is expressible as an infimum of n!/(k!(n − k)!) linear functions of x . The sum ∑ π(x)i is therefore a concave function of x ; in fact, monotonic (3.7.1.0.1) in this instance. 3.3.2 k largest entries Sum of the k largest entries of x∈ R n is the optimal objective value from: [59, exer.5.19] π(x) i = maximize x T y y∈R n subject to 0 ≼ y ≼ 1 1 T y = k ≡ k∑ π(x) i = i=1 minimize k t + 1 T z z∈R n , t∈R subject to x ≼ t1 + z z ≽ 0 (504) 3.10 Hint: Suppose, for example, P belongs to an orthant and A were orthogonal. Begin with A=I and apply level sets of the objective, as in Figure 65 and Figure 68. [ Or rewrite ] x the problem as a linear program like (492) and (494) but in a composite variable ← y . t
- Page 1 and 2: Chapter 3 Geometry of convex functi
- Page 3 and 4: 3.1. CONVEX FUNCTION 213 f 1 (x) f
- Page 5 and 6: 3.1. CONVEX FUNCTION 215 Rf (b) f(X
- Page 7 and 8: 3.3. PRACTICAL NORM FUNCTIONS, ABSO
- Page 9 and 10: 3.3. PRACTICAL NORM FUNCTIONS, ABSO
- Page 11: 3.3. PRACTICAL NORM FUNCTIONS, ABSO
- Page 15 and 16: 3.3. PRACTICAL NORM FUNCTIONS, ABSO
- Page 17 and 18: 3.4. INVERTED FUNCTIONS AND ROOTS 2
- Page 19 and 20: 3.5. AFFINE FUNCTION 229 3.4.1.3 po
- Page 21 and 22: 3.5. AFFINE FUNCTION 231 3.5.0.0.2
- Page 23 and 24: 3.6. EPIGRAPH, SUBLEVEL SET 233 q(x
- Page 25 and 26: 3.6. EPIGRAPH, SUBLEVEL SET 235 To
- Page 27 and 28: 3.6. EPIGRAPH, SUBLEVEL SET 237 con
- Page 29 and 30: 3.6. EPIGRAPH, SUBLEVEL SET 239 3.6
- Page 31 and 32: 3.7. GRADIENT 241 2 1.5 1 0.5 Y 2 0
- Page 33 and 34: 3.7. GRADIENT 243 From (1749) andD.
- Page 35 and 36: 3.7. GRADIENT 245 This equivalence
- Page 37 and 38: 3.7. GRADIENT 247 3.7.1.0.3 Example
- Page 39 and 40: 3.7. GRADIENT 249 f(Y ) [ ∇f(X)
- Page 41 and 42: 3.7. GRADIENT 251 meaning, the grad
- Page 43 and 44: 3.8. MATRIX-VALUED CONVEX FUNCTION
- Page 45 and 46: 3.8. MATRIX-VALUED CONVEX FUNCTION
- Page 47 and 48: 3.8. MATRIX-VALUED CONVEX FUNCTION
- Page 49 and 50: 3.9. QUASICONVEX 259 3.8.3.0.6 Exam
- Page 51 and 52: 3.9. QUASICONVEX 261 3.9.0.0.2 Defi
- Page 53: 3.10. SALIENT PROPERTIES 263 7. - N
3.3. PRACTICAL NORM FUNCTIONS, ABSOLUTE VALUE 223<br />
likely to exist vertex solutions to<br />
minimize<br />
x∈R n ‖Ax‖ 1<br />
subject to x ∈ P<br />
optimized over some bounded polyhedron P . 3.10<br />
<br />
(502)<br />
3.3.1 k smallest entries<br />
Sum <strong>of</strong> the k smallest entries <strong>of</strong> x∈ R n is the optimal objective value from:<br />
for 1≤k ≤n<br />
n∑<br />
n∑<br />
π(x) i = minimize x T y<br />
π(x) i = maximize k t + 1 T z<br />
y∈R n<br />
i=n−k+1 z∈R n , t∈R<br />
subject to 0 ≼ y ≼ 1<br />
≡<br />
subject to x ≽ t1 + z<br />
1 T y = k<br />
z ≼ 0<br />
(503)<br />
i=n−k+1<br />
k∑<br />
i=1<br />
which are dual linear programs, where π(x) 1 = max{x i , i=1... n} where<br />
π is a nonlinear permutation-operator sorting its vector argument into<br />
nonincreasing order. Finding k smallest entries <strong>of</strong> an n-length vector x is<br />
expressible as an infimum <strong>of</strong> n!/(k!(n − k)!) linear <strong>functions</strong> <strong>of</strong> x . The sum<br />
∑ π(x)i is therefore a concave function <strong>of</strong> x ; in fact, monotonic (3.7.1.0.1)<br />
in this instance.<br />
3.3.2 k largest entries<br />
Sum <strong>of</strong> the k largest entries <strong>of</strong> x∈ R n is the optimal objective value from:<br />
[59, exer.5.19]<br />
π(x) i = maximize x T y<br />
y∈R n<br />
subject to 0 ≼ y ≼ 1<br />
1 T y = k<br />
≡<br />
k∑<br />
π(x) i =<br />
i=1<br />
minimize k t + 1 T z<br />
z∈R n , t∈R<br />
subject to x ≼ t1 + z<br />
z ≽ 0<br />
(504)<br />
3.10 Hint: Suppose, for example, P belongs to an orthant and A were orthogonal. Begin<br />
with A=I and apply level sets <strong>of</strong> the objective, as in Figure 65 and Figure 68. [ Or rewrite ] x<br />
the problem as a linear program like (492) and (494) but in a composite variable ← y .<br />
t