MACHINE LEARNING TECHNIQUES - LASA

47
Cases where K-means might be viewed as failing:
Unbalanced clusters:
The K-means algorithm takes into account only the distance between the means and data points;
it has no representation of the weight and breadth of each cluster.. Consequently, data points
belonging to unbalanced clusters (clusters with unbalanced number of points, spread with a
smaller breadth) will be incorrectly assigned.
Elongated clusters:
The K-means algorithm has no way to represent the shape of a cluster. Hence, a simple measure
of distance would be unable to separate two elongated clusters as shown in Figure 3-9.
Figure 3-9: Typical examples for which K-means is ill-suited: unbalanced clusters (left) and elongated
clusters (right). In both cases the algorithm fails to separate the two clusters.
[DEMOS\CLUSTERING\KMEANS-UNBALANCED.ML] [DEMOS\CLUSTERING\KMEANS-ELONGATED.ML]
3.1.3 Soft K-means
`Soft K-means algorithm' was offered as an alternative to K-means in order to “soften” the
assignment of variables. Each data point x is given a soft `degree of assignment' to each of the
means. The degree to which
i
i
x is assigned to cluster k is the responsibility
point i and is a real value comprised between 0 and 1.
r
k '
( −β ⋅d( µ k,
xi)
)
= e
∑ ( −β ⋅ ( µ k'
, i)
)
e
(3.5)
k
i d x
i
r
k
of cluster k for
β is the stiffness and determines the binding between clusters. The larger β , the bigger the
1
distance across two clusters. A measure of the disparity across clusters is given by σ ≡ ,
β
the radius of the circle that surrounds the means of the K clusters.
∑
The sum of the K responsibilities for the i th point is 1, i.e. r k = 1
k
i
© A.G.Billard 2004 – Last Update March 2011