MACHINE LEARNING TECHNIQUES - LASA

149
6. Decrease the size of the neighborhood and the learning rate
( −r
( )
2
ki , t )
2
2σ
h t = e where s 2 is the
Example of neighborhood function could be ( )
width parameter that can gradually be decreased over time.
7. Unless the net is adequately trained (for example η has become very small),
go back to step 2.
ki ,
6.8.1.2 Normalized weight vectors
The weights in Kohonen net are often normalized to unit length. This allows a somewhat cheaper
computation when determining the winning unit. In effect, the network tries to adjust the direction
of the weight vectors to line up with that of the input. The basic idea is that the distance between
the two vectors is minimal, when the two vectors are aligned. So, minimizing the Euclidean
distance is equivalent to maximizing the vector dot product
r r i .
x⋅
w
6.8.1.3 Caveats for the Kohonen Net
The use of Kohonen nets is not as straightforward as it might appear. Although the net does
much of the work in sorting out the organization implicit in the input space, problems can arise.
The first difficulty is in choosing the initial weights. These are often set at random, but if the
distribution of the weights actually selected does not match well with the organization of the input
space, the net will converge very slowly. One way round to this is to set the initial weights
deliberately to reflect the structure of the input space if that is known. For example, thee unit
weight vectors could be smoothly and uniformly distributed across the input space.
A second problem is in deciding how to vary the neighborhood and learning rate to achieve the
best organization. The answer really depends on the application. Note, though, that the standard
Kohonen training algorithm assumes that the training and performance are separate: first you
train the network and, when that is complete, then you use it to classify inputs. For some
applications (for instance, in robot control) it is more appropriate that the net continue learning
and that it generates classifications even though partially trained.
A third problem concerns the presentation of inputs. The training procedure above presupposes
that inputs are presented fairly and at random – in other words – there is no systematic bias in
favor of one part of the input space against others. Without this condition, the net may not
produce a topographic organization, but rather a single clump of units all organized around the
over represented part of the input space. This problem too arises in robotic applications, where
the world is experienced sequentially and consecutive inputs are not randomly chosen from the
input space. Batching up inputs and presenting them randomly from those batches can solve the
problem.
© A.G.Billard 2004 – Last Update March 2011