MACHINE LEARNING TECHNIQUES - LASA

148
6.8.1.1 Algorithm for Kohonen's Self Organizing Map
Assume output nodes are connected in an array (usually 1 or 2 dimensional). Assume that the
network is fully connected - all nodes in input layer are connected to all nodes in output layer.
Use the competitive learning algorithm as follows:
1. Initialize the weights – the initial weights may be random values or values
assigned to cover the space of inputs.
2. Present an input x r to the net
3. For each output node j , calculate the distance between the input x r and its
weight vector
j
w r , e.g. using the Euclidean distance
n
r r j r r j j
, = − = ∑ i
−
i
i=
1
( )
d x w x w x w
(6.57)
4. Determine the "winning" output node i , for which d is minimal.
r r r r
i
k
w −x ≤ w −x ∀k
(6.58)
Note: the above equation is equivalent to w i x >= w k x only if the weights are
normalized.
5. Update the weights of the winner node and of nodes in a neighborhood using
the rule:
Δ r = ⋅ ⋅ r −
r
( )
i
( ) η ( ) ( ) ( )
k
w t t hki
,
t x w t
where η ( t) ∈ [0,1] is a learning rate or gain and h ( t)
ki ,
r
ki ,
1
( t)
(6.59)
= is the
neighborhood function. It is equal to 1 when i = k and falls off with the distance
r between output nodes i and k. Thus, the closer are the units to the winner,
ki
the larger the update. Note that both the learning rate and the neighborhood vary
with time. It is here that the topological information is supplied. Nearby units
receive similar updates and thus end up responding to nearby input patterns. The
i
above rule drags the weight vector w and the weights of nearby units towards
the input x.
Note that if the output nodes are 1, the Kohonen rule is equivalent to the Outstar
rule, see Section 6.6.2.
Δ w = α⋅x ⋅y −γ
⋅ w
(6.60)
ij i j ij
© A.G.Billard 2004 – Last Update March 2011