v2010.10.26 - Convex Optimization

338 CHAPTER 4. SEMIDEFINITE PROGRAMMINGSubstituting 1 − z ← z the direction vector becomesy = 1 − arg maximize z T x ← arg minimize z T xz∈R n z∈R nsubject to 0 ≼ z ≼ 1 subject to 0 ≼ z ≼ 1 (525)z T 1 = kz T 1 = n − k4.5.1.5 optimality conditions for compressed sensingNow we see how global optimality conditions can be stated without referenceto a dual problem: From conditions (469) for optimality of (530), it isnecessary [61,5.5.3] thatx ⋆ ≽ 0 (1)Ax ⋆ = b (2)∇‖x ⋆ ‖ 1 − ∇‖x ⋆ ‖n + A T ν ⋆ ≽ 0k(3)〈∇‖x ⋆ ‖ 1 − ∇‖x ⋆ ‖n + A T ν ⋆ , x ⋆ 〉 = 0k(4l)(764)These conditions must hold at any optimal solution (locally or globally). By(762), the fourth condition is identical toBecause a 1-norm‖x ⋆ ‖ 1 − ‖x ⋆ ‖nk+ ν ⋆T Ax ⋆ = 0 (4l) (765)‖x‖ 1 = ‖x‖nk+ ‖π(|x|) k+1:n ‖ 1 (766)is separable into k largest and n −k smallest absolute entries,‖π(|x|) k+1:n ‖ 1 = 0 ⇔ ‖x‖ 0 ≤ k (4g) (767)is a necessary condition for global optimality. By assumption, matrix Ais fat and b ≠ 0 ⇒ Ax ⋆ ≠ 0. This means ν ⋆ ∈ N(A T ) ⊂ R m , and ν ⋆ = 0when A is full-rank. By definition, ∇‖x‖ 1 ≽ ∇‖x‖nkalways holds. Assumingexistence of a cardinality-k solution, then only three of the four conditionsare necessary and sufficient for global optimality of (530):‖x ⋆ ‖ 1 − ‖x ⋆ ‖nk= 0x ⋆ ≽ 0 (1)Ax ⋆ = b (2)(4g)(768)meaning, global optimality of a feasible solution to (530) is identified by azero objective.