sparse image representation via combined transforms - Convex ...

3.1. DCT AND HOMOGENEOUS COMPONENTS 37
of length N, N ∈ N:
X = {X 0 ,X 1 ,... ,X N−1 },
Y = {Y 0 ,Y 1 ,... ,Y N−1 }.
Let ̂X and Ŷ denote the discrete Fourier transform (as defined in (3.1)) of X and Y
respectively. Let X ∗ Y denote the convolution of these two sequences:
X ∗ Y [k] =
k∑
X[i]Y [k − i]+
i=0
N−1
∑
i=k+1
X[i]Y [N + k − i], k =0, 1, 2,... ,N − 1.
Let ̂X ∗ Y denote the discrete Fourier transform of the sequence X ∗ Y . We have
̂X ∗ Y [k] =
̂X[k]Ŷ [k], k =0, 1, 2,... ,N − 1. (3.10)
In other words, the discrete Fourier transform of a convolution of any two sequences, is equal
to the elementwise multiplication of the discrete Fourier transforms of these two sequences.
We skip the proof.
A crude implementation of the convolution would take N multiplications and N − 1
additions to calculate each element in ̂X ∗ Y , and hence N(2N − 1) operations to get the
sequence ̂X ∗ Y . This is an order O(N 2 ) algorithm.
From (3.10), we can first calculate the DFT of the sequences X and Y . From the result
in (3.9), this is an O(N log 2 N) algorithm. Then we multiply the two DFT sequences,
using N operations. Then we can calculate the inverse DFT of the sequence { ̂X[k]Ŷ [k] :
k =0, 1,... ,N − 1}: another O(N log 2 N) algorithm. Letting C ′ (N) denote the overall
complexity of the method, we have
C ′ (N) = 2C(N)+N + C(N)
= 3C(N)+N
by (3.9)
≍ N log 2 N.
Hence we can do fast convolution with complexity O(N log 2 N), which is lower than order
O(N 2 ).