MACHINE LEARNING TECHNIQUES - LASA

50
where M is the number of datapoints, i.e. x={x 1 , x 2 ,..., x M }, ( , )
measure, e.g. the Euclidean distance.
d x y is a distance
Update Step (M-step): Each cluster’s parameters µ
k, αk and σ are adjusted to
k
match the data points for which the cluster is responsible:
σ
2
k
=
µ =
k
∑
i
∑
i
rx
k
i
r
k
i
∑r
i
N⋅∑
i
∑ ri
i
k
∑∑
α =
( x − µ ) 2
k
i i k
The α represents a measure of the likelihood that the Gaussian k (or cluster k)
k
generated the whole dataset.
k
i
k
i
r
k
i
r
k
i
(3.7)
(3.8)
(3.9)
This fits a mixture of spherical Gaussians. “Spherical” means that the variance of the Gaussians
is the same in all directions. In other words, in the case of multidimensional dataset, the
covariance matrix of the Gaussian is diagonal and isotropic (i.e. all the elements on the diagonal
are equal and all elements on the off-diagonal are zero). This algorithm is still not good at
modeling datasets spread along two elongated clusters, as shown in Figure 3-9. If we wish to
model the clusters by axis-aligned Gaussians, we replace the assignment rule given by Equations
(3.6) and (3.8) with the following:
r
k
i
=
i=
1
k
( 2πσ
) i
k '
( 2πσi
)
⎛⎛ N
⎜⎜ d k
⎜⎜−∑
⎜⎜ i=
1 2
⎝⎝
( µ , x )
k
( σi
)
2 ⎞⎞
i ⎟⎟
2 ⎟⎟
⎟⎟
⎠⎠
(3.10)
⎛⎛ N
2 ⎞⎞
⎜⎜ d( µ k'
, xi)
⎟⎟
⎜⎜−∑
k
2 ⎟⎟
1
⎜⎜ i=
1 2( σi
) ⎟⎟
⎝⎝
⎠⎠
∑αk
e
N
∏
k '
α
k
N
∏
i=
1
k
j
1
µ =
∑
i
∑
i
rx
k
i
r
e
k
i
i,
j
(3.11)
© A.G.Billard 2004 – Last Update March 2011