v2010.10.26 - Convex Optimization

228 CHAPTER 3. GEOMETRY OF CONVEX FUNCTIONS3.2.0.0.1 Example. Projecting the origin, on an affine subset, in 1-norm.In (1884) we interpret least norm solution to linear system Ax = b asorthogonal projection of the origin 0 on affine subset A = {x∈ R n |Ax=b}where A∈ R m×n is fat full-rank. Suppose, instead of the Euclidean metric,we use taxicab distance to do projection. Then the least 1-norm problem isstated, for b ∈ R(A)minimize ‖x‖ 1x(518)subject to Ax = ba.k.a, compressed sensing problem. Optimal solution can be interpretedas an oblique projection on A simply because the Euclidean metric is notemployed. This problem statement sometimes returns optimal x ⋆ havingminimal cardinality; which can be explained intuitively with reference toFigure 70: [19]Projection of the origin, in 1-norm, on affine subset A is equivalent tomaximization (in this case) of the 1-norm ball B 1 until it kisses A ; rather,a kissing point in A achieves the distance in 1-norm from the origin to A .For the example illustrated (m=1, n=3), it appears that a vertex of theball will be first to touch A . 1-norm ball vertices in R 3 represent nontrivialpoints of minimal cardinality 1, whereas edges represent cardinality 2, whilerelative interiors of facets represent maximal cardinality 3. By reorientingaffine subset A so it were parallel to an edge or facet, it becomes evidentas we expand or contract the ball that a kissing point is not necessarilyunique. 3.9The 1-norm ball in R n has 2 n facets and 2n vertices. 3.10 For n > 0B 1 = {x∈ R n | ‖x‖ 1 ≤ 1} = conv{±e i ∈ R n , i=1... n} (519)is a vertex-description of the unit 1-norm ball. Maximization of the 1-normball until it kisses A is equivalent to minimization of the 1-norm ball until itno longer intersects A . Then projection of the origin on affine subset A isminimizex∈R n ‖x‖ 1subject to Ax = b≡minimize cc∈R , x∈R nsubject to x ∈ cB 1Ax = b(520)3.9 This is unlike the case for the Euclidean ball (1884) where minimum-distanceprojection on a convex set is unique (E.9); all kissable faces of the Euclidean ball aresingle points (vertices).3.10 The ∞-norm ball in R n has 2n facets and 2 n vertices.