MACHINE LEARNING TECHNIQUES - LASA

126
6.3.1 Learning rule for the Perceptron
Let
{ 1
,..., }
( )
( p) ( p
p
)
= be the set of input patterns and
X x x n
set of values for the output.
{ }
( p) ( 1) ( P)
= ,..., the associated
yˆ yˆ yˆ
Learning Algorithm:
1. Initialize the weights to small random values w ( 0 ) ( 0 i n)
and x
0
= 1.
2. Present the next pattern
n
⎛⎛
yt = f⎜⎜∑
wi
t⋅xi
⎝⎝ i=
0
3. Calculate ( ) ( )
{ 1
,..., }
( )
( p) ( p
p
)
X = x x n
( p) 4. Adjust the weights following ( ) ( 1) η ˆ ( )
( p)
⎞⎞
⎟⎟
⎠⎠
i
≤ ≤ with
0
( p
( )
)
( )
i i i
w fixed (the bias)
w t = w t− + ⋅ y − y t ⋅ x t (6.2)
The learning rate η modulates the speed at which the weight will vary.
5. Move on to the next pattern and go back to 2.
The process stops if all patterns have been learned. However, it is not clear that all patterns can
be learned. The Perceptron acts as a classifier that separates data into two classes. Note that the
dataset must be linearly separable for the learning to converge, see illustration in Figure 6-3.
Figure 6-3: Learning in one neuron.
Exercise: Which of the AND, OR and XOR gates satisfy the condition of linear separability?
© A.G.Billard 2004 – Last Update March 2011