sparse image representation via combined transforms - Convex ...

5.1. OVERVIEW 107
5.1.4 Preconditioner
Before we move into detailed discussion, we would like to point out that the content of this
section applies to both CG and MINRES.
When the original matrix A does not have a good eigenvalue distribution, a preconditioner
may help. In our case, we only need to consider the problem as in (5.2) for Hermitian
matrices. We consider solving the preconditioned system
L −1 AL −H y = L −1 b, x = L −H y, (5.5)
where L is a preconditioner. When the matrix-vector multiplications associated with L −H
and L −1 have fast algorithms and L −1 AL −H has a good eigenvalue distribution, the system
in (5.2) can be solved in fewer iterations.
Before giving a detailed discussion about preconditioners, we need to restate our setting.
Here A is the Hessian at step k: A = H(x k ). To simplify the discussion, we assume that Φ
is a concatenation of several orthogonal matrices:
Φ=[T 1 ,T 2 ,... ,T m ],
where T 1 ,T 2 ,... ,T m are n × n orthogonal square matrices. The number of columns of Φ is
equal to N = mn 2 . Hence from (4.8),
⎡
⎤
I + D 1 T1 T T 2 ··· T1 T T m
1
2 A = T2 T T 1 I + D 2
⎢
. , (5.6)
⎣ .
.. . ⎥
⎦
TmT T 1 ··· I + D m
where
⎛
D i = 1 2 λ ⎜
⎝
¯ρ ′′ (x 1+(i−1)n2
k
)
. ..
¯ρ ′′ (x n2 +(i−1)n 2
k
)
⎞
⎟
⎠ ,
i =1, 2,... ,m,
are diagonal matrices. To simplify (5.6), we consider A left and right multiplied by a block