v2010.10.26 - Convex Optimization

4.5. CONSTRAINING CARDINALITY 337iterate: y has nonnegative uniformly distributed random entries in (0, 1]corresponding to the n−k smallest entries of x ⋆ and has 0 elsewhere.When this particular heuristic is successful, cardinality versus iteration ischaracterized by noisy monotonicity. Zero entries behave like memory whilerandomness greatly diminishes likelihood of a stall.4.5.1.4 algebraic derivation of direction vector for convex iterationIn3.2.2.1.2, the compressed sensing problem was precisely represented as anonconvex difference of convex functions bounded below by 0find x ∈ R nsubject to Ax = bx ≽ 0‖x‖ 0 ≤ k≡minimize ‖x‖ 1 − ‖x‖nx∈R nksubject to Ax = bx ≽ 0(530)where convex k-largest norm ‖x‖nkis monotonic on R n + . There we showedhow (530) is equivalently stated in terms of gradientsbecauseminimizex∈R n 〈subject to Ax = bx ≽ 0x , ∇‖x‖ 1 − ∇‖x‖nk〉(761)‖x‖ 1 = x T ∇‖x‖ 1 , ‖x‖nk= x T ∇‖x‖nk, x ≽ 0 (762)The objective function from (761) is a directional derivative (at x in directionx ,D.1.6, conferD.1.4.1.1) of the objective function from (530) while thedirection vector of convex iterationy = ∇‖x‖ 1 − ∇‖x‖nk(763)is an objective gradient where ∇‖x‖ 1 = 1 under nonnegativity and∇‖x‖n = arg maximize z T xk z∈R nsubject to 0 ≼ z ≼ 1z T 1 = k⎫⎪⎬⎪⎭ , x ≽ 0 (533)