Chapter 3 Geometry of convex functions - Meboo Publishing ...

Chapter 3 Geometry of convex functions - Meboo Publishing ... Chapter 3 Geometry of convex functions - Meboo Publishing ...

convexoptimization.com
from convexoptimization.com More from this publisher
10.03.2015 Views

222 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONS Then (498) is equivalent to minimize c c∈R , x∈R n , a∈R 2n subject to x = [I −I ]a a T 1 = c a ≽ 0 Ax = b ≡ minimize a∈R 2n ‖a‖ 1 subject to [A −A ]a = b a ≽ 0 (500) where x ⋆ = [I −I ]a ⋆ . (confer (495)) Significance of this result: (confer p.375) Any vector 1-norm minimization problem may have its variable replaced with a nonnegative variable of twice the length. All other things being equal, nonnegative variables are easier to solve for sparse solutions. (Figure 69, Figure 70, Figure 97) The compressed sensing problem becomes easier to interpret; e.g., for A∈ R m×n minimize ‖x‖ 1 x subject to Ax = b x ≽ 0 ≡ minimize 1 T x x subject to Ax = b x ≽ 0 (501) movement of a hyperplane (Figure 26, Figure 30) over a bounded polyhedron always has a vertex solution [92, p.22]. Or vector b might lie on the relative boundary of a pointed polyhedral cone K = {Ax | x ≽ 0}. In the latter case, we find practical application of the smallest face F containing b from 2.13.4.3 to remove all columns of matrix A not belonging to F ; because those columns correspond to 0-entries in vector x . 3.3.0.0.2 Exercise. Combinatorial optimization. A device commonly employed to relax combinatorial problems is to arrange desirable solutions at vertices of bounded polyhedra; e.g., the permutation matrices of dimension n , which are factorial in number, are the extreme points of a polyhedron in the nonnegative orthant described by an intersection of 2n hyperplanes (2.3.2.0.4). Minimizing a linear objective function over a bounded polyhedron is a convex problem (a linear program) that always has an optimal solution residing at a vertex. What about minimizing other functions? Given some nonsingular matrix A , geometrically describe three circumstances under which there are

3.3. PRACTICAL NORM FUNCTIONS, ABSOLUTE VALUE 223 likely to exist vertex solutions to minimize x∈R n ‖Ax‖ 1 subject to x ∈ P optimized over some bounded polyhedron P . 3.10 (502) 3.3.1 k smallest entries Sum of the k smallest entries of x∈ R n is the optimal objective value from: for 1≤k ≤n n∑ n∑ π(x) i = minimize x T y π(x) i = maximize k t + 1 T z y∈R n i=n−k+1 z∈R n , t∈R subject to 0 ≼ y ≼ 1 ≡ subject to x ≽ t1 + z 1 T y = k z ≼ 0 (503) i=n−k+1 k∑ i=1 which are dual linear programs, where π(x) 1 = max{x i , i=1... n} where π is a nonlinear permutation-operator sorting its vector argument into nonincreasing order. Finding k smallest entries of an n-length vector x is expressible as an infimum of n!/(k!(n − k)!) linear functions of x . The sum ∑ π(x)i is therefore a concave function of x ; in fact, monotonic (3.7.1.0.1) in this instance. 3.3.2 k largest entries Sum of the k largest entries of x∈ R n is the optimal objective value from: [59, exer.5.19] π(x) i = maximize x T y y∈R n subject to 0 ≼ y ≼ 1 1 T y = k ≡ k∑ π(x) i = i=1 minimize k t + 1 T z z∈R n , t∈R subject to x ≼ t1 + z z ≽ 0 (504) 3.10 Hint: Suppose, for example, P belongs to an orthant and A were orthogonal. Begin with A=I and apply level sets of the objective, as in Figure 65 and Figure 68. [ Or rewrite ] x the problem as a linear program like (492) and (494) but in a composite variable ← y . t

3.3. PRACTICAL NORM FUNCTIONS, ABSOLUTE VALUE 223<br />

likely to exist vertex solutions to<br />

minimize<br />

x∈R n ‖Ax‖ 1<br />

subject to x ∈ P<br />

optimized over some bounded polyhedron P . 3.10<br />

<br />

(502)<br />

3.3.1 k smallest entries<br />

Sum <strong>of</strong> the k smallest entries <strong>of</strong> x∈ R n is the optimal objective value from:<br />

for 1≤k ≤n<br />

n∑<br />

n∑<br />

π(x) i = minimize x T y<br />

π(x) i = maximize k t + 1 T z<br />

y∈R n<br />

i=n−k+1 z∈R n , t∈R<br />

subject to 0 ≼ y ≼ 1<br />

≡<br />

subject to x ≽ t1 + z<br />

1 T y = k<br />

z ≼ 0<br />

(503)<br />

i=n−k+1<br />

k∑<br />

i=1<br />

which are dual linear programs, where π(x) 1 = max{x i , i=1... n} where<br />

π is a nonlinear permutation-operator sorting its vector argument into<br />

nonincreasing order. Finding k smallest entries <strong>of</strong> an n-length vector x is<br />

expressible as an infimum <strong>of</strong> n!/(k!(n − k)!) linear <strong>functions</strong> <strong>of</strong> x . The sum<br />

∑ π(x)i is therefore a concave function <strong>of</strong> x ; in fact, monotonic (3.7.1.0.1)<br />

in this instance.<br />

3.3.2 k largest entries<br />

Sum <strong>of</strong> the k largest entries <strong>of</strong> x∈ R n is the optimal objective value from:<br />

[59, exer.5.19]<br />

π(x) i = maximize x T y<br />

y∈R n<br />

subject to 0 ≼ y ≼ 1<br />

1 T y = k<br />

≡<br />

k∑<br />

π(x) i =<br />

i=1<br />

minimize k t + 1 T z<br />

z∈R n , t∈R<br />

subject to x ≼ t1 + z<br />

z ≽ 0<br />

(504)<br />

3.10 Hint: Suppose, for example, P belongs to an orthant and A were orthogonal. Begin<br />

with A=I and apply level sets <strong>of</strong> the objective, as in Figure 65 and Figure 68. [ Or rewrite ] x<br />

the problem as a linear program like (492) and (494) but in a composite variable ← y .<br />

t

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!