MACHINE LEARNING TECHNIQUES - LASA

88
5.6 Kernel K-Means
Kernel K-Means is one attempt at using the kernel trick to improve the properties of one of the
simplest clustering techiques to date, the so-called K-means clustering technique, see Section
3.1.2.
i
K-means builds partition the data into a finite set of K clusters C , i = 1... K,
(here, do not confuse
the scalar K with the Gram matrix seen previously). K-means relies on a measure of distance
across datapoint, usually the Euclidean distance. It proceeds iteratively by updating, at each
iteration, the centers
datapoints { i} i 1
function:
µ 1
,...., µ of the clusters until no update is required. Given a set of M
K
M
X x =
= , the K-means processus consists in minimizing the following objective
J
( )
1
K
2
j
j i
x C
K
= x −
i i
=
i= 1
j i
x ∈C mi
∑∑ (5.25)
∈
µ ,...., µ µ with µ
Where m is the number of datapoints in cluster C
i .
i
Since each cluster relies on a common distance measure, each cluster is separated from the
other by a linear hyperplane, as illustrated below:
∑
x
j
∗µ1
∗µ2
*µ3
To counter this disadvantage, kernel k-means first maps the datapoints onto a higher-dimensional
feature space through a non-linear mapφ . It then proceeds as classical K-means and search for
hyperplanes in the feature space. To do this, kernel K-means exploits once more the kernel trick
k x, x' = φ x φ x'
as the dot product in feature space. Using the
and sets the kernel ( ) ( ) ( )
observation that kernel k-means objective function can be expanded into a sum of inner product
across datapoints, yields:
© A.G.Billard 2004 – Last Update March 2011