Thèse de doctorat: Algorithmes de classification répartis sur le cloud

tel-00744768, version 1 - 23 Oct 2012
6.4. VQ PARALLELIZATION SCHEME 131
towards critical points are guaranteed.
The results are gathered with different values of τ (τ ∈ {1, 10, 100}) in the
charts displayed in Figure 6.6. For each chart, we plot the performance curves
when the distributed architecture has a different number of computing instances:
M ∈ {1, 2, 10}. The curve of reference is the sequential execution of the VQ,
that is M = 1. We can notice that for every value of τ ∈ {1, 10, 100}, multiple
resources do not bring speedup for convergence. Even if more data are processed,
there is no increase in the convergence speed. To conclude, no gain in terms of
wall time can be brought using this parallel scheme. In the next subsection we
investigate the cause of these non-satisfactory results and propose a solution to
overcome this problem.
6.4.2 Towards a better parallelization scheme
Let us start the investigation of the parallel scheme given by iterations (6.6)
by rewriting the sequential VQ iterations (6.2) using the notation introduced in
Section 6.1. For t ≥ τ it holds
t
w(t + 1) = w(t − τ + 1) − εt ′ +1H z{t ′ +1 mod n}, w(t ′ ) . (6.7)
t ′ =t−τ+1
For iterations (6.6), consider a time slot t ≥ 0 where an averaging phase occurs,
that is, t mod τ = 0 and t > 0. Then, for all i ∈ {1, . . . , M},
w i (t + 1) = w i t
(t − τ + 1) − εt ′
M 1
+1 H
M
z j
t ′ +1 , wj (t ′ )
. (6.8)
t ′ =t−τ+1
To the empirical distortion defined by equation (6.5) is associated its theoretical
counterpart the distortion function C (see for example [98] or [31]). In the
first situation of iterations (6.7), for a sample z and t ′ > 0, H(z, w(t ′ )) is an
observation and an estimator of the gradient ∇C(w(t ′ )) (see e.g. [95]). In the
second situation of iterations (6.8), let us assume that the multiple versions are
close to each other. This means that w j (t ′ ) ≈ w i (t ′ ), for all (i, j) ∈ {1, . . . , M} 2 .
Thus, the average
1
M
M
H
j=1
j=1
z j
{t ′ +1 mod n} , wj (t ′
)
can also be viewed as an estimation of the gradient ∇C(w i (t ′ )), where i ∈
{1, . . . , M}.