sparse image representation via combined transforms - Convex ...

4.11. DISCUSSION 97
ρ(β) =‖β‖ 2 2 , the method (PR) is called ridge regression by Hoerl and Kennard [81, 80].
4.11.2 Non-convex Sparsity Measure
An ideal measure of sparsity is usually nonconvex. For example, in (4.2), the number of
nonzero elements in x is the most intuitive measure of sparsity. The l 0 norm of x, ‖x‖ 0 ,is
equal to the number of nonzero elements, but it is not a convex function. Another choice
of measure of sparsity is the logarithmic function; for x =(x 1 ,... ,x N ) T ∈ R N ,wecan
have ρ(x) = ∑ N
i=1 log |x i|. In sparse image component analysis, another nonconvex sparsity
measure is used: ρ(x) = ∑ N
i=1 log(1 + x2 i ) [53].
Generally speaking, a nonconvex optimization problem is a combinatorial optimization
problem, and hence it is NP hard. Some discussion about how to use reweighting methods
to solve a nonconvex optimization problem is given in the next subsection.
4.11.3 Iterative Algorithm for Non-convex Optimization Problems
Sometimes, a reweighted iterative method canbeusedtofindalocal minimum for a nonconvex
optimization problem. Let’s consider the following problem:
(LO)
minimize
x
N∑
log |x i |, subject to y =Φx;
i=1
and its corresponding version with a Lagrangian multiplier λ, 1
(LO λ )
minimize ‖y − Φx‖ 2
x
2 + λ
N∑
log(|x i | + δ).
i=1
Note that the objective function of (LO) is not convex.
Let’s consider a reweighted iterative algorithm: for δ>0,
(RIA)
x (k+1) = argmin
x
N∑
i=1
|x i |
|x (k) , subject to y =Φx;
i
| + δ
1 More precisely, (LO λ ) is the Lagrangian multiplier version of the following optimization problem:
minimize
x
N∑
log(|x i| + δ), subject to ‖y − Φx‖ ≤ε.
i=1
Note when δ and ε are small, it is close to (LO).