sparse image representation via combined transforms - Convex ...

110 CHAPTER 5. ITERATIVE METHODS
where
⎛
D(x k )= √ λ/2 ⎜
⎝
¯ρ ′′ (x k 1 ) ⎞
. .. ⎟
⎠
¯ρ ′′ (x k N )
1/2
,
and
⎛
g(x k )= λ ¯ρ ′ (x k 1 )
⎞
⎜
2 ⎝ .
⎟
⎠ .
¯ρ ′ (x k N )
Note that this is the [N] case. Recall ¯ρ ′ and ¯ρ ′′ are defined in (4.4) and (4.5).
We can transfer problem (LS) into a damped LS problem by solving for a shifted variable
¯d: δ ¯d = D(x k )d + D −1 (x k )g(x k ). The problem (LS) becomes
(dLS) :
minimize
¯d
[ ΦD −1 (x k )δ
∥ δI
] [
y − Φ(xk − D
¯d −2 (x k )g(x k ))
−
0
]∥ ∥∥∥∥
2
.
This is the [R] case. This method is also called diagonal preconditioning. Potentially, it
will turn the original LS problem into one with clustered singular values, so that LSQR
may take fewer iterations. LSQR also works with shorter vectors (as it handles the δI term
implicitly).
5.2.2 Why LSQR?
Since the Hessian in (5.1) is at least semidefinite, we prefer a conjugate-gradients type
method. LSQR is analytically equivalent to the standard method of conjugate gradients,
but possesses more favorable numerical properties. Particularly when the Hessian is illconditioned—which
is very likely to happen when the iteration gets closer to the convergence
point—LSQR is more stable than the standard CG. But LSQR costs slightly more storage
and work per iteration.