Journal of Computers - Academy Publisher

JOURNAL OF COMPUTERS, VOL. 6, NO. 9, SEPTEMBER 2011 1963
machine with Gaussian noises on triangular fuzzy number
space to forecast fuzzy nonlinear system [11].
The rough set theory [12] is a powerful preprocessing
tool to find out knowledge from an amount of uncertain
and incomplete data and is applied to the support vector
machines to reduce the features of data to process and
eliminate redundancy. At the same time, it also improves
performance of the classical support vector machines. To
deal with the overfitting problem of the traditional
support vector machine, Zhang and Wang proposed a
rough margin based support vector machine [13]. In this
paper, we propose a double margin based fuzzy support
vector machine by combination of rough theory and
fuzzy support vector machine, namely a double margin
(rough margin) based on fuzzy support vector machine
(RFSVM). The proposed method not only inherits the
characteristic of the FSVM method, but also considers the
effects of decision hyperplane depending on the position
of training samples in the rough margin. So presented
method further reduce overfitting due to noises or outliers.
This paper is organized as follows. In section 2, a brief
review of support vector machine is described. In Section
3, we describe the proposed RFSVM in detail which
contain both binary classification and multiple
classification RFSVM. In the following section, we
evaluate our method on benchmark data sets and compare
it with the existing support vector machine. Some
conclusions are given in the final section.
II. SUPPORT VECTOR MACHINES ALGORITHM
In this section, we briefly describe the support vector
machines in binary classification problems.
Given a dataset of labeled training points (x1, y1), (x2,
y2),…, (xl, yl), where N
( xi, yi) ⊆ R × { + 1, − 1} , i=1, 2…l.
Supposed training data are linearly separable. That is to
say, there is some hyperplane which correctly separates
the positive examples and negative examples. The point x
lying on the hyperplane satisfies +b = 0, where w
is normal to the hyperplane. In this case, support vector
machine algorithm finds the optimal separating
hyperplane with the maximal margin. When the training
data are linearly non-separable or approximately
separable, it is needed to introduce the trade-off
parameter. When the training data is not linearly
separable, support vector machine learning algorithm
introduces kernel strategy that maps the input data to a
higher-dimension feature space z by using a nonlinearly
mapping function ϕ () x and then the data in feature space
z is indeed linearly or approximately separable. All
training data satisfy the following decision function
⎧+
1, if yi
=+ 1
f( xi) = sign( < w, x >+ b)
= ⎨
. (1)
⎩ − 1, if yi
= -1
All training points satisfy the following inequalities:
⎧<
wx , i > + b≥ + 1, ifyi = + 1 . (2)
⎨
⎩ < wx , i > + b≤ − 1, ifyi = -1
In fact, it can be written as yi( < w, xi > + b)
≥ 1,
i=1,2,…,l. above inequalities. It is seen that finding the
hyperplane is equivalent to obtain the maximizing margin
© 2011 ACADEMY PUBLISHER
2
by minimizing || w || subject to constraints (2). So the
primal optimal problem is given as
1 2
min || w ||
wb , 2
st .. y( < w, x >+ b)
≥1.
(3)
i i
i = 1, 2,..., l
To solve optimal problem, we introduce Lagrange
multiplier to transform the primal problem (3) into its
dual problem that becomes the following quadratic
programming (QP) problem:
l l l
1
min ∑∑αiα jyy i j( xi⋅xj) −∑αi
α 2 i= 1 j= 1 i=
1 . (4)
l
∑
s. t. α y = 0, 0 ≤ α , i= 1,2,..., l.
i=
1
i i i
In classifier, the solution in feature space using a
linearly mapping function ϕ ( x)
only replaces the dot
product x ⋅ x j by inner product vectors ϕ( x) ⋅ ϕ(
x j ) . The
mapping function ϕ( x)
and ϕ ( xi
) satisfy
< ϕ( x), ϕ(
xj) >= K( x, xi)
, where K( x, xi) is called kernel
function. In real world application, we would never need
to explicitly know what ϕ is. A decision function with
SVM is obtained by computing dot products of a given
test point x, or more specifically by computing following
sign:
Ns
*
f( x) = α y ( s ⋅ x) + b
∑
i=
1
i i i
Ns
*
∑α
i
i=
1
iϕ i ϕ
Ns
*
= ∑α
i
i=
1
i ( i,
) +
. (5)
= y ( s ) ⋅ ( x) + b
yK s x b
Where the coefficient α is positive, i
i
s is support vector
and Ns is the number of support vectors.
In most cases, as the learning of a suitable hyperplane
is too restrictive to be of practical use and causes a large
overlap of classes, there is nonexistent some separable
hyperplane. To deal with linearly non-separable data, it
often allows that some points are misclassified, and
introduces nonnegative slack variables ξ > 0 measuring
the number of misclassifications and a punishment
parameter C which is a cost trade-off between
maximizing the margin and minimizing the classification
error of training data. The sum of the slacks ∑ ξ is an
i
upper bound on the number of training errors. And, the
original constraints (2) are relaxed to
yi( < w, xi>+ b) ≥1 − ξi,
i = 1,2,..., l.
(6)
Thus, constructing optimal hyperplane is equivalent to
solve the following optimization problem:
l
1 2
min || w|| + C∑ξi
wb , , ξ 2
i=
1
st .. yi( < w, ϕ( xi) >+ b)
≥1−ξ (7)
i
ξ ≥ 0, i = 1, 2,..., l.
i