sparse image representation via combined transforms - Convex ...

4.7. NEWTON DIRECTION 93
4.7 Newton Direction
For fixed γ, we use Newton’s method for convex optimization to solve problem (4.2). Starting
from an initial guess x (0) , at every step, Newton’s then generates a new vector that is
closer to the true solution:
x (i+1) = x (i) + β i n(x (i) ), i =0, 1,... , (4.11)
where β i is a damping parameter (β i is chosen by line search to make sure that the value of
the objective function is reduced), n(x (i) ) is the Newton direction, a function of the current
guess x (i) . Let f(x) =‖y − Φx‖ 2 2 + λρ(x) denote the objective function in problem (4.2).
The gradient and Hessian of f(x) are defined in (4.8) and (4.7). The Newton direction at
x (i) satisfies
[ ]
H(x (i) ) · n(x (i) )=−g(x (i) ). (4.12)
This is a system of linear equations. We choose iterative methods to solve it, as discussed
in the next section and also the next chapter.
4.8 Comparison with Existing Algorithms
We compare our method with the method proposed by Chen, Donoho and Saunders (CDS)
[27]. The basic conclusion is that the two methods are very similar, but our method is
simpler in derivation, requires fewer variables and is potentially more efficient in numerical
computing. We start by reviewing the CDS approach, describe the difference between our
approach and theirs and then discuss benefits of these changes.
Chen, Donoho and Saunders proposed a primal-dual log-barrier perturbed LP algorithm.
Basically, they solve [27, equation (6.3), page 56]
where
minimize
x o c T x o + 1 2 ‖γxo ‖ 2 + 1 2 ‖p‖2 subject to Ax o + δp = y, x o ≥ 0,
• γ and δ are normally small (e.g., 10 −4 ) regularization parameters;
• c = λ1, where λ is the penalization parameter as defined in (4.2) and 1 is an all-one