MACHINE LEARNING TECHNIQUES - LASA

156
dmi
τ
i
=− mi + ∑ wij ⋅xj
t
dt
j
1
x = +
i −D⋅ ( mi
+ b)
1 e
( )
(6.76)
where m is the membrane potential, x the firing rate, τ the time constant, b the bias and
i i i
D a scalar that defines the slope of the sigmoid function.
Neurons following the activation given in (6.76) are called leaky-integrator neurons.
⎛⎛
f ⎜⎜∑
wij
⋅ xj
t
⎝⎝ j
( )
⎞⎞
⎟⎟
⎠⎠
corresponds to the integrative term. As an effect of this term, the activity of the
neuron increases over time due to the external activation of the other neurons to reach a plateau.
The speed of convergence depends onτ , see
Figure 6-15: Typical activity pattern of a leaky-integrator neuron in response to an incoming input (synaptic
∑
activity) S( t) = w ⋅x ( t)
. Unpon receiving a steady input, the membrane potential m increases or
ij j
j
decreases until it reaches the input value.
For a given steady or null external activation, the activity of the neuron decays or leaks
exponentially according to the time constant τ following:
d x
i t
dt
1
τ
( ) x ( t )
=− (6.77)
i
It is important to notice that this term corresponds to adding a self-connection to each neuron,
and, as such provides a memory of the neuron’s activity over time.
Similarly to the static case, one can show that the network is bound to converge under some
conditions. But, in most cases, the complexity of the dynamics that results from combining
several neurons whose dynamics follows a first order differential equation, as given in (6.77),
becomes quickly intractable analytically.
© A.G.Billard 2004 – Last Update March 2011