MACHINE LEARNING TECHNIQUES - LASA

51
k
( σ
j )
2
=
k
∑
i
α =
r
k
( x ) 2
,
− µ
k
i i j j
N⋅
k
i
∑
∑ r
i
∑∑
i
k
i
r
r
k
i
k
i
(3.12)
(3.13)
Soft K-means and the mixture of Gaussians are two examples of maximum likelihood algorithm.
Figure 3-13: Examples of clustering with Mixtures of Gaussians using spherical Gaussians (left) and nonspherical
Gaussians (i.e. with full covariance matrix) (right). Notice how the clusters become elongated
along the direction of the clusters (the grey circles represent the first and second variances of the
distributions).
[DEMOS\CLUSTERING\GMM-ELONGATED.ML]
A fatal flaw of maximum likelihood (KABOOM!):
The major drawback of maximizing the likelihood (people tend to actually minimize the –log of the
likelihood) is that there is no actual maximum. Indeed, the pdf is a positive but unbounded
function. As a result, one may observe the so-called KABOOM effect.
When the mean of one of the Gaussian is located on a particular data point, and, if the variance is
2
2
already very smallσ