MACHINE LEARNING TECHNIQUES - LASA

120
6 Artificial Neural Networks
Artificial neural network (ANN) has become one of those buzzwords that is either popular or
unpopular, but leaves no one indifferent. Unfortunately, for most people, the word relates only to
one type of neural network, namely feed-forward neural networks. ANNs are, however, far more
than this. We will see, in this chapter and the next one, different ANN architectures that greatly
differ from one another, both in terms of computation they can perform as well as in their
algorithms. We will see how the architecture and the learning rule determine the type of
computation an ANN can perform.
6.1 Applications of ANN
ANNs can perform diverse types of computation. Here is a non-exhaustive list:
Type of Computation
Pattern recognition
Associative memory
PCA
ICA
Dimensionality Reduction
Learning Time series
ANNs
Feed-forward ANN and Perceptron
Hebbian Network, Willshaw Net, Hopfield Net
Hebbian Network
Anti-hebbian learning
SOM, Kohonen
TDNN, RNN
ANNs are powerful tools that can deal with large, multidimensional, non-linear, datasets. They
have found application in numerous domains, as statistical tools for data mining in finance and
particle physics or as optimization tools in industrial control systems and robotics.
There are two modes of functioning for an ANN:
1. Activation transfer mode when activation is transmitted throughout the network.
2. Learning mode when the network organizes itself, usually on the basis of the most
recent activation transfer.
6.2 Biological motivation
ANNs were developed with the goal to mimic the highly parallel and distributed computation
performed in the human brain.
6.2.1 The Brain as an Information Processing System
The human brain contains about 10 billion nerve cells, or neurons. On average, each neuron is
connected to other neurons through about 10 000 synapses. (The actual figures vary greatly,
depending on the local neuroanatomy.) The brain's network of neurons forms a massively parallel
information processing system. This contrasts with conventional computers, in which a single
processor executes a single series of instructions.
Against this, consider the time taken for each elementary operation: neurons typically operate at
a maximum rate of about 100 Hz, while a conventional CPU carries out several hundred million
machine level operations per second. Despite of being built with very slow hardware, the brain
has quite remarkable capabilities:
© A.G.Billard 2004 – Last Update March 2011