MACHINE LEARNING TECHNIQUES - LASA

152
correlated), spurious minima can also appear. This means that some patterns are associated with
patterns that are not among the pattern vectors.
Hopfield networks are sometimes called associative networks since they associate a class
pattern to each input pattern. The importance of the different Hopfield networks in practical
application is limited due to theoretical limitations of the network structure (capacity sensitive to
correlated data) but, in situations where data can be decorrelated, they may form interesting
models. Hopfield networks are typically used for classification problems with binary pattern
vectors.
6.9.1 Hopfield Network Structure
xi
w
ij
= w
ji
x
j
The Hopfield network is composed of K neurons and K2 connection weights w , i, j 1,.., K
ij
= . It
is fully connected through symmetric, bi-directional weights, i.e. wij = wji
. It has no selfconnections,
i.e. w
ii
= 0
∀ i .
In the Hopfield network, learning takes one time step and retrieval takes several time steps!
6.9.2 Learning Phase
The learning rule is intended to make a set of desired patterns
1
stable states of the Hopfield network’s activity rule.
Initialization:
0 = 0 = 0 i,
j
At time t=0, set all the weights to w ( ) w ( )
Update:
ij
ji
∀ .
r
x n = { x n ,..., x n }, n=
1,..., N
The network weights are updated only once to represent the correlations across all bits from all
patterns, following:
w
= η∑ x x
(6.65)
n n
ji i j
n
x ∈−
n
j
{ 1,1}
K
© A.G.Billard 2004 – Last Update March 2011