sparse image representation via combined transforms - Convex ...

118 CHAPTER 5. ITERATIVE METHODS
eigendecomposition of the inverse of à may not satisfy this criterion. To see this, let’s look
at a 2 × 2 block matrix. Suppose
(
S1 + I I
)
à =
I
S 2 + I
can be factorized as
à =
(
λ11 λ 12
λ 21 λ 22
)(
D1
D 2
)(
λ
T
11 λ T 21
λ T 12 λ T 22
)
,
where matrix
( )
λ11 λ 12
λ 21 λ 22
is an orthogonal matrix and D 1 and D 2 are diagonal matrices. We have
( )(
à −1 λ11 λ 12 D
−1
1
=
λ 22
λ 21
D −1
2
)(
λ
T
11 λ T 21
λ T 12 λ T 22
)
.
At the same time,
( ) (
(S1 + I) −1 I (S1 + I) −1
=
(S 2 + I − (S 1 + I) −1 ) −1 I
)
à −1 (
I
(S 1 + I) −1 I
)
.
Hence
(S 2 + I − (S 1 + I) −1 ) −1 = λ 21 D −1
1 λT 21 + λ 22 D −1
2 λT 22.
So if there were fast algorithms to multiply with matrices λ 21 , λ 22 and their transposes,
then there would be a fast algorithm to multiply with matrix (S 2 + I − (S 1 + I) −1 ) −1 .
But we know that in general, there is no fast algorithm to multiply with this matrix (see
also the previous subsection).
So in general, such a eigendecomposition will not give a
block preconditioner that has fast algorithms to multiply with its block elements. Hence
we proved that at least in the 2 × 2 case, an eigendecomposition of inverse approach is not
favored.