v2009.01.01 - Convex Optimization

298 CHAPTER 4. SEMIDEFINITE PROGRAMMING
(143) (467) on page 295. From the numerical data given, cardinality ‖x‖ 0 = 1
is expected. Iteration continues until x T y vanishes (to within some numerical
precision); id est, until desired cardinality is achieved. But this comes not
without a stall.
Stalling, whose occurrence is sensitive to initial conditions of convex
iteration, is a consequence of finding a local minimum of a multimodal
objective 〈x, y〉 when regarded as simultaneously variable in x and y . Stalls
are simply detected as solutions x of undesirable cardinality, sometimes
remedied by reinitializing direction vector y to a random positive state.
Bolstered by success in breaking out of a stall, we then apply convex
iteration to 22,000 randomized problems:
Given random data for m=3, n=6, k=1, in Matlab notation
A=randn(3,6), index=round(5∗rand(1)) +1, b=rand(1)∗A(:,index)
(685)
the sparsest solution x∝e index is a scaled standard basis vector.
Without convex iteration or a nonnegativity constraint x≽0, rate of failure
for this minimum cardinality problem Ax=b by 1-norm minimization of x
is 22%. That failure rate drops to 6% with a nonnegativity constraint. If
we then engage convex iteration, detect stalls, and randomly reinitialize the
direction vector, failure rate drops to 0% but the amount of computation is
approximately doubled.
Stalling is not an inevitable behavior. For some problem types (beyond
mere Ax = b), convex iteration succeeds nearly all the time. Here is a
cardinality problem, with noise, whose statement is just a bit more intricate
but easy to solve in a few convex iterations:
4.5.1.2.2 Example. Signal dropout. [102,6.2]
Signal dropout is an old problem; well studied from both an industrial and
academic perspective. Essentially dropout means momentary loss or gap in
a signal, while passing through some channel, caused by some man-made
or natural phenomenon. The signal lost is assumed completely destroyed
somehow. What remains within the time-gap is system or idle channel noise.
The signal could be voice over Internet protocol (VoIP), for example, audio
data from a compact disc (CD) or video data from a digital video disc (DVD),