v2010.10.26 - Convex Optimization

3.2. PRACTICAL NORM FUNCTIONS, ABSOLUTE VALUE 2253.2 Practical norm functions, absolute valueTo some mathematicians, “all norms on R n are equivalent” [159, p.53];meaning, ratios of different norms are bounded above and below by finitepredeterminable constants. But to statisticians and engineers, all norms arecertainly not created equal; as evidenced by the compressed sensing (sparsity)revolution, begun in 2004, whose focus is predominantly 1-norm.A norm on R n is a convex function f : R n → R satisfying: for x,y ∈ R n ,α∈ R [227, p.59] [159, p.52]1. f(x) ≥ 0 (f(x) = 0 ⇔ x = 0) (nonnegativity)2. f(x + y) ≤ f(x) + f(y) 3.7 (triangle inequality)3. f(αx) = |α|f(x) (nonnegative homogeneity)Convexity follows by properties 2 and 3. Most useful are 1-, 2-, and ∞-norm:‖x‖ 1 = minimize 1 T tt∈R nsubject to −t ≼ x ≼ t(508)where |x| = t ⋆ (entrywise absolute value equals optimal t ). 3.8‖x‖ 2 = minimizet∈Rsubject tot[ tI xx T t]≽S n+1+0(509)where ‖x‖ 2 = ‖x‖ √ x T x = t ⋆ .‖x‖ ∞ = minimize tt∈Rsubject to −t1 ≼ x ≼ t1(510)where max{|x i | , i=1... n} = t ⋆ .‖x‖ 1 = minimizeα∈R n , β∈R n 1 T (α + β)subject to α,β ≽ 0x = α − β(511)3.7 ‖x + y‖ ≤ ‖x‖ + ‖y‖ for any norm, with equality iff x = κy where κ ≥ 0.3.8 Vector 1 may be replaced with any positive [sic] vector to get absolute value,theoretically, although 1 provides the 1-norm.