MACHINE LEARNING TECHNIQUES - LASA
MACHINE LEARNING TECHNIQUES - LASA
MACHINE LEARNING TECHNIQUES - LASA
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
91<br />
A simple pattern recognition algorithm<br />
We now have all the tools to derive a simple classification algorithm.<br />
It is now possible, using the canonical dot product as a similarity measure in our dot product<br />
space H to design a simple pattern recognition algorithm that will separate our input space into<br />
two classes. The algorithm proceeds as follows:<br />
1. Assign each point randomly to one of the two classes<br />
2. Compute the means of the two classes in the feature space:<br />
c<br />
c<br />
+<br />
−<br />
1<br />
=<br />
m<br />
1<br />
=<br />
m<br />
∑<br />
+ iy | i =+ 1<br />
∑<br />
− iy | i =−1<br />
Where m +<br />
and m −<br />
are the number of examples with positive and negative labels,<br />
respectively. We assume that both classes are non-empty.<br />
3. Reassign each point to the class whose mean is closest.<br />
x<br />
i<br />
i<br />
x<br />
(5.29)<br />
This geometric construction can be formulated in terms of the dot product<br />
between c +<br />
and c −<br />
lies the point c ( c c ) /2<br />
+ −<br />
xx , ' . Halfway<br />
= + . We compute the class to which the point x<br />
belongs by checking whether the vector connecting x− c encloses an angle smaller than π /2<br />
with the vector w= c+ − c connecting the class means. This leads to:<br />
−<br />
( )<br />
y = sgn x−c,<br />
w<br />
( x− ( c+ + c− ) c+ + c−<br />
)<br />
= sgn / 2,<br />
( xc+ − xc−<br />
+ b)<br />
= sgn , , .<br />
(5.30)<br />
1 2 2<br />
with offset b = ( c− − c+<br />
)<br />
: .<br />
2<br />
The decision boundary indicated by the dotted line is a hyperplane, orthogonal to w, see Figure<br />
5-5.<br />
Figure 5-5: A simple geometrical classification algorithm. The decision boundary indicated by the dotted<br />
line is a hyperplane orthogonal to w. (From Scholkopf & Smola 2002)<br />
© A.G.Billard 2004 – Last Update March 2011