MACHINE LEARNING TECHNIQUES - LASA

137
Figure 6-11: The weight vector moves toward the input vector
The outstar rule, where the decay term is gated by the output term
y
i
.
dw
dt
ij
{ }
= α x − w y
(6.28)
j ij i
In the discrete case, we have:
( )
Δ w = α ⋅x ⋅ y − w
(6.29)
ij i i ij
( )
y = 1 ⇒ Δ w = α ⋅ y −w
i ij j ij
( ) (1 α) ( 1)
⇒ w t = − ⋅w t− + α⋅y
ij ij j
(6.30)
In this case, the weight vector moves toward the output vector.
6.6.3 Principal Components
Let us consider again the activation function of the Perceptron and rewrite as follows:
where
i
r r
y = w x = w ⋅x
∑
i ij j
i
w r is the weight vector along i and x r the input vector. We have:
whereθ is the angle across the two vector.
r r r r
w ⋅ x=
w x cos θ
i
i
This quantity is maximal when the angle θ is zero, i.e. when the two vectors are aligned. Thus, if
the weight converge towards the principal components of the input space (i.e. w r is the 1 st
1
eigenvector, w r the second, etc), then the first output vector y
2
1
will transmit maximally the input
information along the direction with largest variance. In other words, if we define a learning rule,
such that it projects the weights along the principal components of the input space, such a
network would in effect produce a PCA analysis.
i
( )
© A.G.Billard 2004 – Last Update March 2011