MACHINE LEARNING TECHNIQUES - LASA

97
where
N is the number of support vectors (to recall, the support vectors are all the points
s
which the corresponding Lagrange multiplier α
i
> 0
x
i
for
; these points lie exactly on the margin).
Thus the only diference from the optimal hyperplane case is that the α
i
now have an upper
C
bound with .
M
Once again, we must use the KKT conditions to solve the primal minimization of
complementary conditions to find b .
L and the KKT
P
5.7.3 Non-Linear Support Vector Machines
Let us now consider an extension of the linear type of classifier we considered before to tackle
non-linear classification problem, such as the one highlighted in Figure 5-8.
Figure 5-8: Degree 3 polynomial kernel. The background colour shows the shape of the decision surface.
Figure 5-9: Classification using a polynomial kernel with different degrees (SVM). The data is not linearly
separable (left). By increasing the degree of the polynomial, the separation plane becomes non-linear and is
able to correctly separate the data. [DEMOS\CLASSIFICATION\SVM-POLY.ML]
Following the same rational as presented earlier on, let us first map the data onto an Euclidean
space H , using a mapping φ :
φ : X a H
x a
φ
( x)
Assume that the training problem is in the form of dot products x x ( x )
i j j
, i T x
(5.54)
= ⋅ . In this case,
the training algorithm in the mapped space would only depend on the data through dot products
i
j
in H , i.e. on functions of the form ( x ), ( x )
φ φ . If we can define a "kernel function" k such
© A.G.Billard 2004 – Last Update March 2011