SVM Example

Figure 1: Sample data points in R 2 . Blue diamonds are positive examples andred squares are negative examples.We would like to discover a simple SVM that accurately discriminates thetwo classes. Since the data is linearly separable, we can use a linear SVM (thatis, one whose mapping function Φ() is the identity function). By inspection, itshould be obvious that there are three support vectors (see Figure 2):{s 1 =( 10), s 2 =( 31), s 3 =( 3−1In what follows we will use vectors augmented with a 1 as a bias input, andfor clarity we will differentiate these with an over-tilde. So, if s 1 = (10), then˜s 1 = (101). Figure 3 shows the SVM architecture, and our task is to find valuesfor the α i such that)}α 1 Φ(s 1 ) · Φ(s 1 ) + α 2 Φ(s 2 ) · Φ(s 1 ) + α 3 Φ(s 3 ) · Φ(s 1 ) = −1α 1 Φ(s 1 ) · Φ(s 2 ) + α 2 Φ(s 2 ) · Φ(s 2 ) + α 3 Φ(s 3 ) · Φ(s 2 ) = +1α 1 Φ(s 1 ) · Φ(s 3 ) + α 2 Φ(s 2 ) · Φ(s 3 ) + α 3 Φ(s 3 ) · Φ(s 3 ) = +1Since for now we have let Φ() = I, this reduces toα 1 ˜s 1 · ˜s 1 + α 2 ˜s 2 · ˜s 1 + α 3 ˜s 3 · ˜s 1 = −1α 1 ˜s 1 · ˜s 2 + α 2 ˜s 2 · ˜s 2 + α 3 ˜s 3 · ˜s 2 = +1α 1 ˜s 1 · ˜s 3 + α 2 ˜s 2 · ˜s 3 + α 3 ˜s 3 · ˜s 3 = +1Now, computing the dot products results in2