Journal of Computers - Academy Publisher

1936 JOURNAL OF COMPUTERS, VOL. 6, NO. 9, SEPTEMBER 2011
feasibility of this approach is examined on the testing
function and nonlinear under-actuated systems.
This paper is organized as follows. The LS-SVM
regression algorithm is briefly reviewed in Section 2.
Parameters optimization algorithm based on the CAS
algorithm is addressed in Section 3. The results of testing
and simulation are presented to demonstrate the
effectiveness of the proposed method in Section 4. The
application of LS-SVM based on the CAS Algorithm is
given in Section 5. Finally, the paper is concluded in
Section 6.
II. LS_SVM REGRESSION
The LS-SVM, evolved from the SVM, changes the
inequality constraint of a SVM into an equality constraint
and forces the sum of squared error (SSE) loss function to
become an experience loss function of the training set.
Then the problem has become one of solved linear
programming problems. This can be specifically
described as follows [4]:
Given the following training sample set (D):
{ ( , y ) k = 1,
2,
L,
N}
D = x k k
where N is the total number of training data pairs,
k
n
R ∈ x is the regression vector and ∈ R is the
n
output. According to SVM theory, the input space R is
mapped into a feature space, and then the linear equation
in the feature space can be defined as:
T
f ( x) = w ϕ ( x)
+ b
(1)
h
where the nonlinear mapping ϕ : R → R maps the
input data into a so-called high dimensional feature space
(which can be infinite dimension). The regularized cost
function of the LS-SVM is given as:
where,
1 T 1
min J ( w , e)
= w w + γ
2 2
T
n
y k
N
2
∑ ek
k = 1
s. t.
yk
= w ϕ ( xk
) + b + ek
, k = 1,
2,
L,
N (2)
h n
w ∈ R is the weight vector, ek ∈ R is slack
variable, b ∈ R is a bias term and γ ∈ R is regularization
item. The Lagrangian corresponding to Eq. (2) can be
defined as follows:
L ( w,b,e; α)
=
N
−∑
k = 1
k
T { w ( x ) + b + e y }
J ( w,e) α ϕ
− (3)
where α k ∈ R(
k = 1,
2,
L,
N ) are the Lagrange multipliers.
The KKT conditions can be expressed by
N
∑
k = 1
© 2011 ACADEMY PUBLISHER
k
w = α ϕ(x
)
(4)
k
k
k
n
k
T
α = γe
(5)
N
∑
k = 1
k
k
α = 0
(6)
k
w ϕ ( x ) b + e − y = 0
(7)
k
+ k k
After elimination of w and e k , the solution of the
optimization problem can be obtained by solving the
following set of linear equations
⎡b⎤
⎡0
⎢ ⎥ = ⎢v
⎣α⎦
⎢⎣
T
1 N
T N
, N ] ∈ R
with y = [ y , L,
y ] ∈ R ,
v −1
T ⎤ ⎡0⎤
−1 ⎥ ⎢ ⎥
Ω + γ I ⎥⎦
⎣ y⎦
r¡
N
= [ 1,
L,
1
T
]
N
∈ R
α = [ α1 , L α and Ω is an N × N kernel matrix.
By using the kernel trick [2], one obtains
T
Ω = ϕ ( x ) ϕ(
x ) = ( x , x ) , ∀k,
l = 1,
2,
L,
N.
kl
k
l
K k l
And the resulting LS-SVM regression model becomes
N
∑
k = 1
(8)
f ( x) = α K(
x,
x ) + b
(9)
where α k , b are the solution to Eq. (8).
k
Note that the dot product ϕ ⋅) ϕ(
⋅)
in the feature space
( T
is replaced by a prechosen kernel function K( ⋅,
⋅)
due to
the employment of the kernel trick. Thus, there is no need
to construct the feature vector w or to know the nonlinear
mapping ϕ(⋅) explicitly. Given a training set, the training
of an LS-SVM is equal to solving a set of linear equations
as Eq. (8). This greatly simplifies the regression problem.
The chosen kernel function must satisfy the Mercer’s
condition [2]. Possible kernel functions are, e.g.:
Linear kernel
K( x , x ) = x ⋅ x .
k
Polynomial kernel
k
l
K ( x , x ) = ( x ⋅ x + 1)
.
l
Gaussian RBF kernel
k
k
l
K( x , x ) = exp( − x − x / 2 ) .
k
l
l
k
m
k
2 2
l σ
where d denotes the polynomial degree, σ is the kernel
(bandwidth) parameter.
It is well known that LS-SVM generalization
performance depends on a good setting of regularization
parameter and the kernel parameter. In order to achieve
the better generalization performance, it is necessary to
select and optimize these parameters.
III. PARAMETERS OPTIMIZATION OF LS_SVM BASED ON
CAS ALGORITHM