MACHINE LEARNING TECHNIQUES - LASA

155
C
µν
−1
where ( )
is the µν th element of the inverse matrix
overlap of each pattern on each bit and is given by:
( C)
µν
N
i=
1
µ ν
i i
1
C − . C measures the degree of
= ∑ x x
(6.72)
The price paid for the improvement on the network capacity is that the learning rule becomes
non-local, which makes the model less attractive from a biological point of view.
6.9.5 Convergence of the Static Hopfield Network
One can show that the Hopfield Network with an asynchronous update rule is bound to converge.
The idea is that the dynamics of the system can be described by an Energy function:
1
E =− ∑ Wijss
(6.73)
i j
2
i,
j
It remains to show that E is a Lyapunov function; i.e., it is bound to converge to a stable fixed
*
point x . A Lyapunov function E is, by definition, a continuous differentiable, real value function
with the following two properties:
* *
( ) ( )
Positive Definite: E x > 0 ∀x≠ x and E x = 0
(6.74)
d *
All trajectory flow downhill: E ( x ) < 0 ∀ x ≠ x
(6.75)
dx
Hint: Once the network has converged, all units have reached a stable state, i.e.
i
( ) ( 1 )
s t = s t+ ∀ i.
i
6.10 Continuous Time Hopfield Network and Leaky-Integrator Neurons
As mentioned previously, the convergence and capacity of the Hopfield network can be majorly
affected when changing from asynchronous update to synchronous update. It turns out that, once
we move to a continuous-time version of the Hopfield network, this issue melts away.
Let us assume that each neuron's activity
x
j
is a continuous function of time xj
( )
t . The neuron's
response to the activation of other neurons in the network is then given by a first-order differential
equation:
© A.G.Billard 2004 – Last Update March 2011