MACHINE LEARNING TECHNIQUES - LASA

98
i
that ( , ) ( ),
( )
j
i
j
k x x φ x φ x
= , then training depends only on knowing k and would not
require to know φ .
The optimization problem consists then to maximize the following quantity:
i j
i
( αi −αi )( αi −αi) k( x , x ) −ε ( αi − αi ) + y ( αi −αi
)
M M M
1
− ʹ′ ʹ′ ʹ′
2
l
subject to
1( αi
αʹ′
i=
i )
∑ ∑ ∑ (5.55)
i, j= 1 i= 1 i=
1
∑ − = 0 and α , α [ 0, A
i i ]
ʹ′ ∈ . ε ∈R. In this case, the class label is
computed as follows:
( )
M
⎛⎛
⎞⎞
j
y = sgn ⎜⎜ αik x, x + b⎟⎟
⎝⎝ i
⎠⎠
∑ (5.56)
Each expansion corresponds to a separating hyperplane in a feature space. In this sense, the
can be considered a dual representation of the hyperplane's normal vector. A test point is
classified by comparing it to all the training points with non-zero weight.
α
i
5.7.4 n-SVM
Chosing the right parameter C may be difficult in practice. n-SVM is an alternative that optimizes
for the best tradeoff between model complexity (the largest margin) and penalty on the error
automatically. To this end, it introduces two other parameters n and r. ν ≥ 0 is an open parameter
while ρ will be optimized for. The objective function becomes:
M
⎛⎛ 2 1 ⎞⎞
min { ⎜⎜ w − νρ + ∑ξi
,
w, ξ
M
⎟⎟
⎝⎝
i=
1 ⎠⎠
i
subject to y ,
i
( wx b)
and ξ ≥0, ρ ≥0.
i
+ ≥ ρ−ξ
i
(5.57)
To understand the role of ρ , observe first that when the points are well classified, the margin has
now been changed to 2 ρ / w . The larger ρ the larger the margin. Since points within the margin
may be misclassified, one can compute the margin error, i.e. the number of points that are within
the margin while misclassified. ν varies the effect of this increase in the margin error while
optimizing for a large value of ρ . One can show that:
• ν is an upper bound on the fraction of margin error (i.e. the number of datapoints
misclassified in the margin)
• ν is a lower bound on the number of support vectors
© A.G.Billard 2004 – Last Update March 2011