v2009.01.01 - Convex Optimization

3.1. CONVEX FUNCTION 203
k/m
1
0.9
1
0.9
0.8
0.8
0.7
signed
0.7
positive
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
0 0.2 0.4 0.6 0.8 1
0
0 0.2 0.4 0.6 0.8 1
m/n
Figure 62: [103,5] [34] Exact recovery transition: Respectively signed
or positive solutions x to Ax=b with sparsity k below thick curve are
recoverable. For Gaussian random matrix A∈ R m×n , thick curve demarcates
phase transition in ability to find sparsest solution x by linear programming.
Then (462) is equivalent to
minimize c
c∈R , x∈R n , a∈R 2n
subject to x = [I −I ]a
a T 1 = c
a ≽ 0
Ax = b
≡
minimize
a∈R 2n ‖a‖ 1
subject to [A −A ]a = b
a ≽ 0
where x ⋆ = [I −I ]a ⋆ . (confer (459)) Significance of this result:
(464)
Any vector 1-norm minimization problem may have its variable
replaced with a nonnegative variable of twice the length.
All other things being equal, nonnegative variables are easier to solve for
sparse solutions. (Figure 62, Figure 80) The compressed sensing problem
becomes easier to interpret; e.g.,
minimize ‖x‖ 1
x
subject to Ax = b
x ≽ 0
≡
minimize 1 T x
x
subject to Ax = b
x ≽ 0
(465)
movement of a hyperplane (Figure 22, Figure 26) over a bounded polyhedron
always has a vertex solution.