sparse image representation via combined transforms - Convex ...

132 CHAPTER 7. FUTURE WORK
of block coordinate relaxation is the following: consider minimizing an objective function
that is a sum of a residual sum of square and a l 1 penalty term on coefficients x,
(B1)
minimize
x
‖y − Φx‖ 2 2 + λ‖x‖ 1 .
Suppose the matrix Φ can be split into several orthogonal and square submatrices, Φ =
[Φ 1 , Φ 2 ,... ,Φ k ]; and x can be split into some subvectors accordingly, x =(x T 1 ,xT 2 ,... ,xT k )T .
Problem (B1) is equivalent to
(B2)
minimize
x 1 ,x 2 ,... ,x k
k∑
k∑
‖y − Φ i x i ‖ 2 2 + λ ‖x i ‖ 1 .
i=1
i=1
When x 1 ,... ,x i−1 ,x i+1 ,... ,x k are fixed but not x i , the minimizer ˜x i of (B2) for x i is
˜x i = η λ/2
⎛
⎜
⎝Φ T i
⎛
⎜
⎝y −
⎞⎞
k∑
⎟⎟
Φ l x l ⎠⎠ ,
l=1
l≠i
where η λ/2 is a soft-thresholding operator (for x ∈ R):
{
sign(x)(|x|−λ/2), if |x| ≥λ/2,
η λ/2 (x) =
0, otherwise.
We may iteratively solve problem (B2) and at each iteration, we rotate the soft-thresholding
scheme through all subsystems. (Each subsystem is associated with a pair (Φ i ,x i ).) This
method is called block coordinate relaxation (BCR) in [124, 125]. Bruce, Sardy and Tseng
[124] report that in their experiments, BCR is faster than the interior point method proposed
by Chen, Donoho and Saunders [27]. They also note that if some subsystems are not
orthogonal, then BCR does not apply.
Motivated by BCR, we may split our original optimization problem into several subproblems.
(Recall that our matrix Φ is also a combination of submatrices associated with
some image transforms.) We can develop another iterative method. In each iteration, we
solve each subproblem one by one by assuming that the coefficients associated with other
subsystems are fixed. If a subproblem corresponds to an orthogonal matrix, then the solution
is simply a result of soft-thresholding of the analysis transform of the residual image. If
a subproblem does not correspond to an orthogonal matrix, moreover, if it corresponds to a