MACHINE LEARNING TECHNIQUES - LASA

42
Figure 3-3: Example of distance measurement in hierarchical clustering methods
It is clear that the number and type of clusters will strongly depend on the choice of the distance
metric and on the method used to merge the clusters. A typical measure of distance between two
N-dimensional data points xy , takes the general form:
p
d ( x, y)
= x −y
N
∑ i i
p
(3.1)
i=
1
The 1-norm distance, i.e. p=1, sometimes referred to as the Manhattan distance, because it is the
distance a car would drive in a city laid out in square blocks (if there are no one-way streets). The
2-norm distance is the classical Euclidean distance.
Figure 3-4 shows examples of data sets in which such nearest neighbor technique would fail.
Failure to converge to a correct solution might occur, for instance, when the data points within a
dataset are further apart than the two clusters. An even worse situation occurs when the clusters
contain one another, as shown in Figure 3-4 right. In other words such a simple clustering
technique works well only when the clusters are linearly separable. A solution to such a situation
is to change coordinate system, e.g. using polar coordinates. However, determining the
appropriate coordinate system remains a challenge in itself.
Figure 3-4: Example of pairs of clusters, easy to see but awkward to extract for clustering algorithms
© A.G.Billard 2004 – Last Update March 2011