sparse image representation via combined transforms - Convex ...

5.2. LSQR 111
5.2.3 Algorithm LSQR
A formal description of LSQR is given in [117, page 50]. We list it here for the convenience
of readers.
Algorithm LSQR:to minimize ‖b − Ax‖ 2 .
1. Initialize.
β 1 u 1 = b, α 1 v 1 = A T u 1 ,w 1 = v 1 ,x 0 =0, ¯φ 1 = β 1 , ¯ρ 1 = α 1 .
2. For i =1, 2, 3,...
(a) Continue the bidiagonalization.
i. β i+1 u i+1 = Av i − α i u i
ii. α i+1 v i+1 = A T u i+1 − β i+1 v i .
(b) Construct and apply next orthogonal transformation.
i. ρ i =(¯ρ 2 i + βi+1 2 )1/2
ii. c i =¯ρ i /ρ i
iii. s i = β i+1 /ρ i
iv. θ i+1 = s i α i+1
v. ¯ρ i+1 = −c i α i+1
vi. φ i = c i ¯φi
vii. ¯φi+1 = s i ¯φi .
(c) Update x, w.
i. x i = x i−1 +(φ i /ρ i )w i
ii. w i+1 = v i+1 − (θ i+1 /ρ i )w i .
(d) Test for convergence. Exit if some stopping criteria have been met.
5.2.4 Discussion
There are two possible dangers in the previous approaches (LS) and (dLS). They are both
caused by the existence of large entries in the vector D −1 (x k )g(x k ). The first danger
occurs in (LS), when D −1 (x k )g(x k ) is large, the right-hand side is large even though the
elements of d will be small as Newton’s method converges. Converting (5.1) to (LS) is a