MACHINE LEARNING TECHNIQUES - LASA

131
y= ∑ w x ) of
Because of the linearity of the activation function of the Perceptron neuron (i.e.
j j
j
the Adaline network, we have:
∂E
∂y
∂y
∂w
p
p
p
j
= x
p
j
p p
( z y )
=− −
(6.14)
(6.15)
( )
p p p
Δ
pwj
= γ z −y ⋅ x
(6.16)
This has proven to be a most powerful rule and is at the core of almost all current supervised
learning methods for ANN. But, it should be emphasized that nothing we have written guarantees
that the method will cause the weight to converge. It can be proved that the method will give an
optimal (in a least square error sense) approximation of the function being modeled. However,
the method does not ensure convergence to a global optimum.
6.4.2 The Backpropagation Network
An example of multi-layered Perceptron, or feed-forward neural network, is shown in Figure 6-6
Activity in the network is propagated forwards via a first set of weights from the input layer to the
hidden layer, and, then, via a second set of weights from hidden layer to output layer. The error is
calculated by Equation (6.12), similarly to what was done for the Adaline network. Now two sets
of weights must be calculated.
Here, however, one does not have access to the desired output of the hidden units. This is
referred to, as the Credit assignment problem – in that we must assign how much effect each
weight in the first layer of weights has on the final output of the network. In order to compute the
weight change, we need to propagate backwards, to backpropagate, the error across the two
layers. The algorithm is quite general and applies to any number of hidden layers.
An example of multilayered perceptron is shown in
Figure 6-6: Multi-layered feed-forward NN
© A.G.Billard 2004 – Last Update March 2011