sparse image representation via combined transforms - Convex ...

Chapter 5
Iterative Methods
This chapter discusses the algorithms we use to solve for the Newton direction (see (4.12))
in our sparse representation problem. We choose an iterative method because there is a fast
algorithm to implement the matrix-vector multiplication. Since our matrix is Hermitian
(moreover symmetric), we basically choose between CG and MINRES. We choose LSQR,
which is a variation of the CG, because our system is at least positive semidefinite and
LSQR is robust against rounding error caused by finite-precision arithmetic.
In Section 5.1, we start with an overview of the iterative methods. Section 5.2 explains
why we favor LSQR and how to apply it to our problem. Section 5.3 gives some details on
MINRES. Section 5.4 contains some discussion.
5.1 Overview
5.1.1 Our Minimization Problem
We wish to solve the following problem:
minimize f(x) =‖y − Φx‖ 2
x
2 + λρ(x),
where y is the vectorized analyzed image, Φ is a (flat) matrix with each column a vectorized
basis function of a certain image analysis transform (e.g., a vectorized basis function for
the 2-D DCT or the 2-D wavelet transform), x is the coefficient vector, and ρ is a separable
convex function, ρ(x) = ∑ N
i=1 ¯ρ(x i), where ¯ρ is a convex 1-D function.
We apply a damped Newton method. To find the Newton direction, the following system
103