sparse image representation via combined transforms - Convex ...

86 CHAPTER 4. COMBINED IMAGE REPRESENTATION
for example, linear programming. The previous result shows that we can attack a combinatorial
optimization problem by solving a convex optimization problem; if the solution
satisfies certain conditions, then the solution of the convex optimization problem is the same
as the solution of the combinatorial optimization problem. This gives a new possibility to
solve an NP hard problem.
More examples of identical minimum l 1 norm decomposition and minimum l 0 norm
decomposition are given in [43].
An exact minimum l 1 norm problem is
(eP 1 )
minimize
x
‖x‖ 1 , subject to y =Φx.
We consider a problem whose constraint is based on the l 2 norm:
(P 1 ) minimize
x
‖x‖ 1 , subject to ‖y − Φx‖ 2 ≤ ɛ,
where ɛ is a constant. Section 4.4 explains how to solve this problem.
4.4 Lagrange Multipliers
We explain how to solve (P 1 ) based on some insights from the interior point method. One
key idea is to select a barrier function and then minimize the sum of the objective function
and a multiplication of a positive constant and the barrier function. When the solution x
approaches the boundary of the feasible set, the barrier function becomes infinite, thereby
guaranteeing that the solution is always within the feasible set. Note that the subsequent
optimization problem has become a nonconstrained optimization problem. Hence we can
apply some standard methods—for example, the Newton method—to solve it.
A typical interior point method uses a logarithmic barrier function [113]. The algorithm
in [25] is equivalent to using an l 2 penalty function. Since the feasible set in (P 1 ) is the whole
Euclidean space, the demand of restricting the solution in a feasible set is not essential. We
actually solve the following problem
minimize ‖y − Φx‖ 2
x
2 + λρ(x), (4.2)
where λ is a scalar parameter and ρ is a convex separable function: ρ(x) = ∑ N
i=1 ¯ρ(x i), where