sparse image representation via combined transforms - Convex ...

106 CHAPTER 5. ITERATIVE METHODS
methods are called Krylov subspace approximations [78, Page 49].
[C2] At step k, the CG error vector and the MINRES residual vector can be written as
e k = p C k (A)e 0,
r k = p M k (A)r 0,
where p C k and pM k
are two polynomials with degree no higher than k that take value
1 at the origin. Moreover,
‖e k ‖ A = min‖p k (A)e 0 ‖ A ,
p k
‖r k ‖ = min‖p k (A)r 0 ‖,
p k
where p k is a kth-degree polynomial with value 1 at the origin.
[C3] Suppose for a Hermitian and positive semidefinite matrix A, A = UΛU T is its eigendecomposition,
where U is an orthogonal matrix and Λ is a diagonal matrix. Suppose
Λ=diag{λ 1 ,... ,λ N }. We have sharp bounds for the norms of the error and the
residual in CG and MINRES:
‖e k ‖ A /‖e 0 ‖ A ≤ min max k(λ i )|,
p k i=1,2,... ,N
for CG; (5.3)
‖r k ‖/‖r 0 ‖≤min max k(λ i )|,
p k i=1,2,... ,N
for MINRES. (5.4)
In (5.3) and (5.4), if the eigenvalues are tightly clustered around a single point (away
from the origin), then the right-hand sides are more likely to be minimized; hence, iterative
methods tends to converge quickly. On the other hand, if the eigenvalues are widely spread,
especially if they lie on the both sides of the origin, then the values on the right-hand sides
of both inequalities are difficult to be minimized; hence, iterative methods may converge
slowly.
Note that the above results are based on exact arithmetic. In finite-precision computation,
these error bounds are in general not true, because the round-off errors that are due
to finite precision may destroy assumed properties in the methods (e.g., orthogonality). For
further discussion, we refer to Chapter 4 of [78] and the references therein.