MACHINE LEARNING TECHNIQUES - LASA

154
If
( ) = ( − )
⎛⎛ ⎞⎞
xi ( t+ 1 ) = θ ( ai ( t)
) ai ( t) =⎜⎜∑
wjixj
( t)
⎟⎟
⎝⎝ j ⎠⎠
x t+ 1 = x t ∀ j ≠i
j
( ) ( )
j
x t x t 1 for all units j, then the network has reached a stable
j
j
state, otherwise we keep changing the state of the units until it converges.
Note that the properties of the Hopfield network may be sensitive to the choice of synchronous
versus asynchronous activation.
Figure 6-14: Hopfield Network trained on 4 patterns (top row). (middle row) increasing amounts of noise are
added to one of the patterns. (bottom row) Hopfield network after convergence from the noisy state. Notice
how the network is able to retrieve the original pattern even when most of the image is noise. The rightmost
image is pure random noise, the network converges to a spurious state due to a local minimum in the
network energy. [DEMOS\HOPFIELD\HOPFIELD.EXE]
6.9.4 Capacity of the static Hopfield Network
One can show that the maximal number of patterns N that can be stored in a network of size K is:
Nmax 0.138⋅
K
; (6.70)
The capacity of the Hopfield network decreases importantly in the face of correlated patterns.
One way of suppressing the effect of correlated patterns is to modify the learning rule in order to
decorrelate the patterns. The learning rule becomes:
1 N N
ji i j
K
µν
µ = 1ν=
1
µ −1
ν
= ∑∑ ( )
(6.71)
w x C x
© A.G.Billard 2004 – Last Update March 2011