sparse image representation via combined transforms - Convex ...

32 CHAPTER 3. IMAGE TRANSFORMS AND IMAGE FEATURES
the set of eigenvalues of this system. Note: when the signal is infinitely long and the
impulse response is relatively short, we can drop the cyclic constraint. Since an LTI
system is a widely assumed model in signal processing, the Fourier transform plays
an important role in analyzing this type of system.
2. A covariance matrix of a stationary Gaussian (i.e., Normal) time series is Toeplitz
and symmetric. If we can assume that the time series does not have long-range dependency
2 , which implies that off-diagonal elements diminish when they are far from
the diagonal, then the covariance matrix is almost circulant. Hence, the Fourier series
again become the eigenvectors of this matrix. The DFT matrix DFT N nearly diagonalizes
the covariance matrix, which implies that after the discrete Fourier transform
of the time series, the coefficients are almost independently distributed. This idea can
be depicted as
DFT N ·{the covariance matrix}·DFT T N ≈{a diagonal matrix}.
Generally speaking, it is easier to process independent coefficients than a correlated
time series. The eigenvalue amplitudes determine the variance of the associated coefficients.
Usually, the eigenvalue amplitudes decay fast: say, exponentially. The
coefficients corresponding to the eigenvalues having large amplitudes are important
coefficients in analysis. The remaining coefficients could almost be considered constant.
Some research in this direction can be found in [58], [59] and [101].
From the above two examples, we see that FT plays an important role in both DSP and
time series analysis.
Fast Fourier Transform
To illustrate the importance of the fast Fourier transform in contemporary computational
science, we find that the following sentences (the first paragraph of the Preface in [136]) say
exactly what we would like to say.
The fast Fourier transform (FFT) is one of the truly great computational developments
of this century. It has changed the face of science and engineering so
2 No long-range dependency means that if two locations are far away in the series, then the corresponding
two random variables are nearly independent.