Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Self-organizing incremental neural network and its
application
F. Shen 1 O. Hasegawa 2
1 National Key Laboratory for Novel Software Technology, Nanjing University
2 Imaging Science and Engineering Lab, Tokyo Institute of Technology
June 12, 2009
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Contents of this tutorial
1 What is SOINN
2 Why SOINN
3 Detail algorithm of SOINN
4 SOINN for machine learning
5 SOINN for associative memory
6 References
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

1 What is SOINN
2 Why SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
3 Detail algorithm of SOINN
4 SOINN for machine learning
5 SOINN for associative memory
6 References
What is SOINN
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

What is SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
What is SOINN
SOINN: Self-organizing incremental neural network
Represent the topological structure of the input data
Realize online incremental learning
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

What is SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
What is SOINN
SOINN: Self-organizing incremental neural network
Represent the topological structure of the input data
Realize online incremental learning
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

What is SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
What is SOINN
SOINN: Self-organizing incremental neural network
Represent the topological structure of the input data
Realize online incremental learning
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

What is SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
What is SOINN
SOINN: Self-organizing incremental neural network
Represent the topological structure of the input data
Realize online incremental learning
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

1 What is SOINN
2 Why SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
3 Detail algorithm of SOINN
4 SOINN for machine learning
5 SOINN for associative memory
6 References
Background
Characteristics of SOINN
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
Characteristics of SOINN
Background: Networks for topology representation
SOM(Self-Organizing Map): predefine structure and size of
the network
NG(Neural Gas): predefine the network size
GNG(Growing Neural Gas): predefine the network size;
constant learning rate leads to non-stationary result.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
Characteristics of SOINN
Background: Networks for topology representation
SOM(Self-Organizing Map): predefine structure and size of
the network
NG(Neural Gas): predefine the network size
GNG(Growing Neural Gas): predefine the network size;
constant learning rate leads to non-stationary result.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
Characteristics of SOINN
Background: Networks for topology representation
SOM(Self-Organizing Map): predefine structure and size of
the network
NG(Neural Gas): predefine the network size
GNG(Growing Neural Gas): predefine the network size;
constant learning rate leads to non-stationary result.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
Characteristics of SOINN
Background: Networks for topology representation
SOM(Self-Organizing Map): predefine structure and size of
the network
NG(Neural Gas): predefine the network size
GNG(Growing Neural Gas): predefine the network size;
constant learning rate leads to non-stationary result.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
Characteristics of SOINN
Background: Networks for incremental learning
Incremental learning: Learning new knowledge without destroy
of old learned knowledge (Stability-Plasticity Dilemma)
ART(Adaptive Resonance Theory): Need a user defined
threshold.
Multilayer Perceptrons: To learn new knowledge will destroy
old knowledge
Sub-network methods: Need plenty of storage
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
Characteristics of SOINN
Background: Networks for incremental learning
Incremental learning: Learning new knowledge without destroy
of old learned knowledge (Stability-Plasticity Dilemma)
ART(Adaptive Resonance Theory): Need a user defined
threshold.
Multilayer Perceptrons: To learn new knowledge will destroy
old knowledge
Sub-network methods: Need plenty of storage
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
Characteristics of SOINN
Background: Networks for incremental learning
Incremental learning: Learning new knowledge without destroy
of old learned knowledge (Stability-Plasticity Dilemma)
ART(Adaptive Resonance Theory): Need a user defined
threshold.
Multilayer Perceptrons: To learn new knowledge will destroy
old knowledge
Sub-network methods: Need plenty of storage
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
Characteristics of SOINN
Background: Networks for incremental learning
Incremental learning: Learning new knowledge without destroy
of old learned knowledge (Stability-Plasticity Dilemma)
ART(Adaptive Resonance Theory): Need a user defined
threshold.
Multilayer Perceptrons: To learn new knowledge will destroy
old knowledge
Sub-network methods: Need plenty of storage
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
Characteristics of SOINN
Background: Networks for incremental learning
Incremental learning: Learning new knowledge without destroy
of old learned knowledge (Stability-Plasticity Dilemma)
ART(Adaptive Resonance Theory): Need a user defined
threshold.
Multilayer Perceptrons: To learn new knowledge will destroy
old knowledge
Sub-network methods: Need plenty of storage
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Characteristics of SOINN
Background
Characteristics of SOINN
Neurons are self-organized with no predefined network
structure and size
Adaptively find suitable number of neurons for the network
Realize online incremental learning without any priori
condition
Find typical prototypes for large-scale data set.
Robust to noise
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Characteristics of SOINN
Background
Characteristics of SOINN
Neurons are self-organized with no predefined network
structure and size
Adaptively find suitable number of neurons for the network
Realize online incremental learning without any priori
condition
Find typical prototypes for large-scale data set.
Robust to noise
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Characteristics of SOINN
Background
Characteristics of SOINN
Neurons are self-organized with no predefined network
structure and size
Adaptively find suitable number of neurons for the network
Realize online incremental learning without any priori
condition
Find typical prototypes for large-scale data set.
Robust to noise
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Characteristics of SOINN
Background
Characteristics of SOINN
Neurons are self-organized with no predefined network
structure and size
Adaptively find suitable number of neurons for the network
Realize online incremental learning without any priori
condition
Find typical prototypes for large-scale data set.
Robust to noise
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Characteristics of SOINN
Background
Characteristics of SOINN
Neurons are self-organized with no predefined network
structure and size
Adaptively find suitable number of neurons for the network
Realize online incremental learning without any priori
condition
Find typical prototypes for large-scale data set.
Robust to noise
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Characteristics of SOINN
Background
Characteristics of SOINN
Neurons are self-organized with no predefined network
structure and size
Adaptively find suitable number of neurons for the network
Realize online incremental learning without any priori
condition
Find typical prototypes for large-scale data set.
Robust to noise
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

1 What is SOINN
2 Why SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
3 Detail algorithm of SOINN
4 SOINN for machine learning
5 SOINN for associative memory
6 References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Structure: Two-layer competitive network
Two-layer competitive
network
First layer: Competitive
for input data
Second layer: Competitive
for output of first-layer
Output topology structure
and weight vector of
second layer
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Structure: Two-layer competitive network
Two-layer competitive
network
First layer: Competitive
for input data
Second layer: Competitive
for output of first-layer
Output topology structure
and weight vector of
second layer
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Structure: Two-layer competitive network
Two-layer competitive
network
First layer: Competitive
for input data
Second layer: Competitive
for output of first-layer
Output topology structure
and weight vector of
second layer
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Structure: Two-layer competitive network
Two-layer competitive
network
First layer: Competitive
for input data
Second layer: Competitive
for output of first-layer
Output topology structure
and weight vector of
second layer
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Structure: Two-layer competitive network
Two-layer competitive
network
First layer: Competitive
for input data
Second layer: Competitive
for output of first-layer
Output topology structure
and weight vector of
second layer
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Training flowchart of SOINN
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Adaptively updated
threshold
Between-class
insertion
Update weight of
nodes
Within-class
insertion
Remove noise nodes
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Training flowchart of SOINN
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Adaptively updated
threshold
Between-class
insertion
Update weight of
nodes
Within-class
insertion
Remove noise nodes
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Training flowchart of SOINN
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Adaptively updated
threshold
Between-class
insertion
Update weight of
nodes
Within-class
insertion
Remove noise nodes
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Training flowchart of SOINN
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Adaptively updated
threshold
Between-class
insertion
Update weight of
nodes
Within-class
insertion
Remove noise nodes
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Training flowchart of SOINN
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Adaptively updated
threshold
Between-class
insertion
Update weight of
nodes
Within-class
insertion
Remove noise nodes
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Training flowchart of SOINN
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Adaptively updated
threshold
Between-class
insertion
Update weight of
nodes
Within-class
insertion
Remove noise nodes
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Training flowchart of SOINN
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Adaptively updated
threshold
Between-class
insertion
Update weight of
nodes
Within-class
insertion
Remove noise nodes
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
First layer: adaptively updating threshold Ti
Basic idea: within-class distance ≤ T ≤ between-class distance
1 Initialize: Ti = +∞ when node i is a new node.
2 When i is winner or second winner, update Ti by
If i has neighbors, Ti is updated as the maximum distance
between i and all of its neighbors.
Ti = max ||Wi − Wc|| (1)
c∈Ni
If i has no neighbors, Ti is updated as the minimum distance
of i and all other nodes in network A.
Ti = min
c∈A\{i} ||Wi − Wc|| (2)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
First layer: adaptively updating threshold Ti
Basic idea: within-class distance ≤ T ≤ between-class distance
1 Initialize: Ti = +∞ when node i is a new node.
2 When i is winner or second winner, update Ti by
If i has neighbors, Ti is updated as the maximum distance
between i and all of its neighbors.
Ti = max ||Wi − Wc|| (1)
c∈Ni
If i has no neighbors, Ti is updated as the minimum distance
of i and all other nodes in network A.
Ti = min
c∈A\{i} ||Wi − Wc|| (2)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
First layer: adaptively updating threshold Ti
Basic idea: within-class distance ≤ T ≤ between-class distance
1 Initialize: Ti = +∞ when node i is a new node.
2 When i is winner or second winner, update Ti by
If i has neighbors, Ti is updated as the maximum distance
between i and all of its neighbors.
Ti = max ||Wi − Wc|| (1)
c∈Ni
If i has no neighbors, Ti is updated as the minimum distance
of i and all other nodes in network A.
Ti = min
c∈A\{i} ||Wi − Wc|| (2)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
First layer: adaptively updating threshold Ti
Basic idea: within-class distance ≤ T ≤ between-class distance
1 Initialize: Ti = +∞ when node i is a new node.
2 When i is winner or second winner, update Ti by
If i has neighbors, Ti is updated as the maximum distance
between i and all of its neighbors.
Ti = max ||Wi − Wc|| (1)
c∈Ni
If i has no neighbors, Ti is updated as the minimum distance
of i and all other nodes in network A.
Ti = min
c∈A\{i} ||Wi − Wc|| (2)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
First layer: adaptively updating threshold Ti
Basic idea: within-class distance ≤ T ≤ between-class distance
1 Initialize: Ti = +∞ when node i is a new node.
2 When i is winner or second winner, update Ti by
If i has neighbors, Ti is updated as the maximum distance
between i and all of its neighbors.
Ti = max ||Wi − Wc|| (1)
c∈Ni
If i has no neighbors, Ti is updated as the minimum distance
of i and all other nodes in network A.
Ti = min
c∈A\{i} ||Wi − Wc|| (2)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
First layer: adaptively updating threshold Ti
Basic idea: within-class distance ≤ T ≤ between-class distance
1 Initialize: Ti = +∞ when node i is a new node.
2 When i is winner or second winner, update Ti by
If i has neighbors, Ti is updated as the maximum distance
between i and all of its neighbors.
Ti = max ||Wi − Wc|| (1)
c∈Ni
If i has no neighbors, Ti is updated as the minimum distance
of i and all other nodes in network A.
Ti = min
c∈A\{i} ||Wi − Wc|| (2)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Second layer: constant threshold Tc
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Basic idea 1: within-class distance ≤ T ≤ between-class
distance
Basic idea 2: we already have some knowledge of input data
from results of first-layer.
Within-class distance:
dw = 1
||Wi − Wj|| (3)
NC
(i,j)∈C
Between-class distance of two class Ci and Cj:
db(Ci,Cj) = min ||Wi − Wj|| (4)
i∈Ci,j∈Cj
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Second layer: constant threshold Tc
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Basic idea 1: within-class distance ≤ T ≤ between-class
distance
Basic idea 2: we already have some knowledge of input data
from results of first-layer.
Within-class distance:
dw = 1
||Wi − Wj|| (3)
NC
(i,j)∈C
Between-class distance of two class Ci and Cj:
db(Ci,Cj) = min ||Wi − Wj|| (4)
i∈Ci,j∈Cj
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Second layer: constant threshold Tc
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Basic idea 1: within-class distance ≤ T ≤ between-class
distance
Basic idea 2: we already have some knowledge of input data
from results of first-layer.
Within-class distance:
dw = 1
||Wi − Wj|| (3)
NC
(i,j)∈C
Between-class distance of two class Ci and Cj:
db(Ci,Cj) = min ||Wi − Wj|| (4)
i∈Ci,j∈Cj
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Second layer: constant threshold Tc
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Basic idea 1: within-class distance ≤ T ≤ between-class
distance
Basic idea 2: we already have some knowledge of input data
from results of first-layer.
Within-class distance:
dw = 1
||Wi − Wj|| (3)
NC
(i,j)∈C
Between-class distance of two class Ci and Cj:
db(Ci,Cj) = min ||Wi − Wj|| (4)
i∈Ci,j∈Cj
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Second layer: constant threshold Tc
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Basic idea 1: within-class distance ≤ T ≤ between-class
distance
Basic idea 2: we already have some knowledge of input data
from results of first-layer.
Within-class distance:
dw = 1
||Wi − Wj|| (3)
NC
(i,j)∈C
Between-class distance of two class Ci and Cj:
db(Ci,Cj) = min ||Wi − Wj|| (4)
i∈Ci,j∈Cj
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Second layer: constant threshold Tc (continue)
1 Set Tc as the minimum between-cluster distance.
Tc = db(Ci1 ,Cj1 ) = min
k,l=1,...,Q,k=l db(Ck,Cl) (5)
2 Set Tc as the minimum between-class distance.
Tc = db(Ci1 ,Cj1 ) = min
k,l=1,...,Q,k=l db(Ck,Cl) (6)
3 If Tc is less than within-class distance dw, set Tc as the next
minimum between-cluster distance.
Tc = db(Ci2 ,Cj2 ) = min db(Ck,Cl) (7)
k,l=1,...,Q,k=l,k=i1,l=j1
4 Go to step 2 to update Tc until Tc is greater than dw.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Second layer: constant threshold Tc (continue)
1 Set Tc as the minimum between-cluster distance.
Tc = db(Ci1 ,Cj1 ) = min
k,l=1,...,Q,k=l db(Ck,Cl) (5)
2 Set Tc as the minimum between-class distance.
Tc = db(Ci1 ,Cj1 ) = min
k,l=1,...,Q,k=l db(Ck,Cl) (6)
3 If Tc is less than within-class distance dw, set Tc as the next
minimum between-cluster distance.
Tc = db(Ci2 ,Cj2 ) = min db(Ck,Cl) (7)
k,l=1,...,Q,k=l,k=i1,l=j1
4 Go to step 2 to update Tc until Tc is greater than dw.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Second layer: constant threshold Tc (continue)
1 Set Tc as the minimum between-cluster distance.
Tc = db(Ci1 ,Cj1 ) = min
k,l=1,...,Q,k=l db(Ck,Cl) (5)
2 Set Tc as the minimum between-class distance.
Tc = db(Ci1 ,Cj1 ) = min
k,l=1,...,Q,k=l db(Ck,Cl) (6)
3 If Tc is less than within-class distance dw, set Tc as the next
minimum between-cluster distance.
Tc = db(Ci2 ,Cj2 ) = min db(Ck,Cl) (7)
k,l=1,...,Q,k=l,k=i1,l=j1
4 Go to step 2 to update Tc until Tc is greater than dw.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Second layer: constant threshold Tc (continue)
1 Set Tc as the minimum between-cluster distance.
Tc = db(Ci1 ,Cj1 ) = min
k,l=1,...,Q,k=l db(Ck,Cl) (5)
2 Set Tc as the minimum between-class distance.
Tc = db(Ci1 ,Cj1 ) = min
k,l=1,...,Q,k=l db(Ck,Cl) (6)
3 If Tc is less than within-class distance dw, set Tc as the next
minimum between-cluster distance.
Tc = db(Ci2 ,Cj2 ) = min db(Ck,Cl) (7)
k,l=1,...,Q,k=l,k=i1,l=j1
4 Go to step 2 to update Tc until Tc is greater than dw.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Second layer: constant threshold Tc (continue)
1 Set Tc as the minimum between-cluster distance.
Tc = db(Ci1 ,Cj1 ) = min
k,l=1,...,Q,k=l db(Ck,Cl) (5)
2 Set Tc as the minimum between-class distance.
Tc = db(Ci1 ,Cj1 ) = min
k,l=1,...,Q,k=l db(Ck,Cl) (6)
3 If Tc is less than within-class distance dw, set Tc as the next
minimum between-cluster distance.
Tc = db(Ci2 ,Cj2 ) = min db(Ck,Cl) (7)
k,l=1,...,Q,k=l,k=i1,l=j1
4 Go to step 2 to update Tc until Tc is greater than dw.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Updating learning rate ǫ1(t) and ǫ2(t)
Update of weight vector
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
∆Ws1 = ǫ1(t)(ξ − Ws1 ) (8)
∆Wi = ǫ2(t)(ξ − Wi) (∀i ∈ Ns1 ) (9)
After the size of network becomes stable, fine tune the network
stochastic approximation: a number of adaptation steps with
a
strength ǫ(t) decaying slowly but not too slowly, i.e.,
∞
t=1 ǫ(t) = ∞, and ∞ t=1 ǫ2 (t) < ∞.
The harmonic series satisfies the conditions.
ǫ1(t) = 1
t , ǫ2(t) = 1
100t
(10)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Updating learning rate ǫ1(t) and ǫ2(t)
Update of weight vector
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
∆Ws1 = ǫ1(t)(ξ − Ws1 ) (8)
∆Wi = ǫ2(t)(ξ − Wi) (∀i ∈ Ns1 ) (9)
After the size of network becomes stable, fine tune the network
stochastic approximation: a number of adaptation steps with
a
strength ǫ(t) decaying slowly but not too slowly, i.e.,
∞
t=1 ǫ(t) = ∞, and ∞ t=1 ǫ2 (t) < ∞.
The harmonic series satisfies the conditions.
ǫ1(t) = 1
t , ǫ2(t) = 1
100t
(10)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Updating learning rate ǫ1(t) and ǫ2(t)
Update of weight vector
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
∆Ws1 = ǫ1(t)(ξ − Ws1 ) (8)
∆Wi = ǫ2(t)(ξ − Wi) (∀i ∈ Ns1 ) (9)
After the size of network becomes stable, fine tune the network
stochastic approximation: a number of adaptation steps with
a
strength ǫ(t) decaying slowly but not too slowly, i.e.,
∞
t=1 ǫ(t) = ∞, and ∞ t=1 ǫ2 (t) < ∞.
The harmonic series satisfies the conditions.
ǫ1(t) = 1
t , ǫ2(t) = 1
100t
(10)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Updating learning rate ǫ1(t) and ǫ2(t)
Update of weight vector
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
∆Ws1 = ǫ1(t)(ξ − Ws1 ) (8)
∆Wi = ǫ2(t)(ξ − Wi) (∀i ∈ Ns1 ) (9)
After the size of network becomes stable, fine tune the network
stochastic approximation: a number of adaptation steps with
a
strength ǫ(t) decaying slowly but not too slowly, i.e.,
∞
t=1 ǫ(t) = ∞, and ∞ t=1 ǫ2 (t) < ∞.
The harmonic series satisfies the conditions.
ǫ1(t) = 1
t , ǫ2(t) = 1
100t
(10)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Single-layer SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
For topology
representation,
first-layer is enough
Within-class
insertion slightly
happened in
first-layer
Using subclass and
density to judge if
connection is
needed.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Single-layer SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
For topology
representation,
first-layer is enough
Within-class
insertion slightly
happened in
first-layer
Using subclass and
density to judge if
connection is
needed.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Single-layer SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
For topology
representation,
first-layer is enough
Within-class
insertion slightly
happened in
first-layer
Using subclass and
density to judge if
connection is
needed.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Single-layer SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
For topology
representation,
first-layer is enough
Within-class
insertion slightly
happened in
first-layer
Using subclass and
density to judge if
connection is
needed.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Single-layer SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
For topology
representation,
first-layer is enough
Within-class
insertion slightly
happened in
first-layer
Using subclass and
density to judge if
connection is
needed.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Single-layer SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
For topology
representation,
first-layer is enough
Within-class
insertion slightly
happened in
first-layer
Using subclass and
density to judge if
connection is
needed.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation
Stationary and non-stationary
Stationary: all training data obey same distribution
Non-stationary: next training sample maybe obey different
distribution from previous one.
Original data Stationary Non-stationary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation
Stationary and non-stationary
Stationary: all training data obey same distribution
Non-stationary: next training sample maybe obey different
distribution from previous one.
Original data Stationary Non-stationary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation
Stationary and non-stationary
Stationary: all training data obey same distribution
Non-stationary: next training sample maybe obey different
distribution from previous one.
Original data Stationary Non-stationary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation
Stationary and non-stationary
Stationary: all training data obey same distribution
Non-stationary: next training sample maybe obey different
distribution from previous one.
Original data Stationary Non-stationary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation
Stationary and non-stationary
Stationary: all training data obey same distribution
Non-stationary: next training sample maybe obey different
distribution from previous one.
Original data Stationary Non-stationary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation
Stationary and non-stationary
Stationary: all training data obey same distribution
Non-stationary: next training sample maybe obey different
distribution from previous one.
Original data Stationary Non-stationary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation
Stationary and non-stationary
Stationary: all training data obey same distribution
Non-stationary: next training sample maybe obey different
distribution from previous one.
Original data Stationary Non-stationary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation (continue)
Original data Two-layer SOINN Single-layer SOINN
Conclusion of experiments: SOINN is able to
Represent topology structure of input data.
Realize incremental learning.
Automatically learn number of nodes, de-noise, etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation (continue)
Original data Two-layer SOINN Single-layer SOINN
Conclusion of experiments: SOINN is able to
Represent topology structure of input data.
Realize incremental learning.
Automatically learn number of nodes, de-noise, etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation (continue)
Original data Two-layer SOINN Single-layer SOINN
Conclusion of experiments: SOINN is able to
Represent topology structure of input data.
Realize incremental learning.
Automatically learn number of nodes, de-noise, etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation (continue)
Original data Two-layer SOINN Single-layer SOINN
Conclusion of experiments: SOINN is able to
Represent topology structure of input data.
Realize incremental learning.
Automatically learn number of nodes, de-noise, etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation (continue)
Original data Two-layer SOINN Single-layer SOINN
Conclusion of experiments: SOINN is able to
Represent topology structure of input data.
Realize incremental learning.
Automatically learn number of nodes, de-noise, etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation (continue)
Original data Two-layer SOINN Single-layer SOINN
Conclusion of experiments: SOINN is able to
Represent topology structure of input data.
Realize incremental learning.
Automatically learn number of nodes, de-noise, etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation (continue)
Original data Two-layer SOINN Single-layer SOINN
Conclusion of experiments: SOINN is able to
Represent topology structure of input data.
Realize incremental learning.
Automatically learn number of nodes, de-noise, etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN
Training process of SOINN
Similarity threshold for judging input data
Learning rate
Simple version of SOINN
Simulation results
Artificial data set: topology representation (continue)
Original data Two-layer SOINN Single-layer SOINN
Conclusion of experiments: SOINN is able to
Represent topology structure of input data.
Realize incremental learning.
Automatically learn number of nodes, de-noise, etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

1 What is SOINN
2 Why SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
3 Detail algorithm of SOINN
4 SOINN for machine learning
5 SOINN for associative memory
6 References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Some objectives of unsupervised learning
Automatically learn number of classes of input data
Clustering with no priori knowledge
Topology representation
Realize real-time incremental learning
Separate classes with low density overlapped area
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Some objectives of unsupervised learning
Automatically learn number of classes of input data
Clustering with no priori knowledge
Topology representation
Realize real-time incremental learning
Separate classes with low density overlapped area
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Some objectives of unsupervised learning
Automatically learn number of classes of input data
Clustering with no priori knowledge
Topology representation
Realize real-time incremental learning
Separate classes with low density overlapped area
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Some objectives of unsupervised learning
Automatically learn number of classes of input data
Clustering with no priori knowledge
Topology representation
Realize real-time incremental learning
Separate classes with low density overlapped area
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Some objectives of unsupervised learning
Automatically learn number of classes of input data
Clustering with no priori knowledge
Topology representation
Realize real-time incremental learning
Separate classes with low density overlapped area
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Some objectives of unsupervised learning
Automatically learn number of classes of input data
Clustering with no priori knowledge
Topology representation
Realize real-time incremental learning
Separate classes with low density overlapped area
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN for unsupervised learning: If two nodes connected
with one path, the nodes belong to one class
1 Do SOINN for input data, output topology representation of
nodes
2 Initialize all nodes as unclassified.
3 Randomly choose one unclassified node i from node set A.
Mark node i as classified and label it as class Ci.
4 Search A to find all unclassified nodes that are connected to
node i with a “path.” Mark these nodes as classified and label
them as the same class as node i.
5 Go to Step3 to continue the classification process until all
nodes are classified.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN for unsupervised learning: If two nodes connected
with one path, the nodes belong to one class
1 Do SOINN for input data, output topology representation of
nodes
2 Initialize all nodes as unclassified.
3 Randomly choose one unclassified node i from node set A.
Mark node i as classified and label it as class Ci.
4 Search A to find all unclassified nodes that are connected to
node i with a “path.” Mark these nodes as classified and label
them as the same class as node i.
5 Go to Step3 to continue the classification process until all
nodes are classified.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN for unsupervised learning: If two nodes connected
with one path, the nodes belong to one class
1 Do SOINN for input data, output topology representation of
nodes
2 Initialize all nodes as unclassified.
3 Randomly choose one unclassified node i from node set A.
Mark node i as classified and label it as class Ci.
4 Search A to find all unclassified nodes that are connected to
node i with a “path.” Mark these nodes as classified and label
them as the same class as node i.
5 Go to Step3 to continue the classification process until all
nodes are classified.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN for unsupervised learning: If two nodes connected
with one path, the nodes belong to one class
1 Do SOINN for input data, output topology representation of
nodes
2 Initialize all nodes as unclassified.
3 Randomly choose one unclassified node i from node set A.
Mark node i as classified and label it as class Ci.
4 Search A to find all unclassified nodes that are connected to
node i with a “path.” Mark these nodes as classified and label
them as the same class as node i.
5 Go to Step3 to continue the classification process until all
nodes are classified.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN for unsupervised learning: If two nodes connected
with one path, the nodes belong to one class
1 Do SOINN for input data, output topology representation of
nodes
2 Initialize all nodes as unclassified.
3 Randomly choose one unclassified node i from node set A.
Mark node i as classified and label it as class Ci.
4 Search A to find all unclassified nodes that are connected to
node i with a “path.” Mark these nodes as classified and label
them as the same class as node i.
5 Go to Step3 to continue the classification process until all
nodes are classified.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN for unsupervised learning: If two nodes connected
with one path, the nodes belong to one class
1 Do SOINN for input data, output topology representation of
nodes
2 Initialize all nodes as unclassified.
3 Randomly choose one unclassified node i from node set A.
Mark node i as classified and label it as class Ci.
4 Search A to find all unclassified nodes that are connected to
node i with a “path.” Mark these nodes as classified and label
them as the same class as node i.
5 Go to Step3 to continue the classification process until all
nodes are classified.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Artificial data set: 5 classes with 10% noise
Original data Clustering result
Conclusion of experiments
Automatically reports number of classes.
Perfectly clustering data with different shape and distribution.
Find typical prototypes; incremental learning; de-noise; etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Artificial data set: 5 classes with 10% noise
Original data Clustering result
Conclusion of experiments
Automatically reports number of classes.
Perfectly clustering data with different shape and distribution.
Find typical prototypes; incremental learning; de-noise; etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Artificial data set: 5 classes with 10% noise
Original data Clustering result
Conclusion of experiments
Automatically reports number of classes.
Perfectly clustering data with different shape and distribution.
Find typical prototypes; incremental learning; de-noise; etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Artificial data set: 5 classes with 10% noise
Original data Clustering result
Conclusion of experiments
Automatically reports number of classes.
Perfectly clustering data with different shape and distribution.
Find typical prototypes; incremental learning; de-noise; etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Artificial data set: 5 classes with 10% noise
Original data Clustering result
Conclusion of experiments
Automatically reports number of classes.
Perfectly clustering data with different shape and distribution.
Find typical prototypes; incremental learning; de-noise; etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Artificial data set: 5 classes with 10% noise
Original data Clustering result
Conclusion of experiments
Automatically reports number of classes.
Perfectly clustering data with different shape and distribution.
Find typical prototypes; incremental learning; de-noise; etc.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Face recognition: AT&T face data set
Experiment results
Automatically reports there are 10 classes.
Prototypes of every classes are reported.
With such prototypes, recognition ratio (1-NN rule) is 90%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Face recognition: AT&T face data set
Experiment results
Automatically reports there are 10 classes.
Prototypes of every classes are reported.
With such prototypes, recognition ratio (1-NN rule) is 90%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Face recognition: AT&T face data set
Experiment results
Automatically reports there are 10 classes.
Prototypes of every classes are reported.
With such prototypes, recognition ratio (1-NN rule) is 90%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Face recognition: AT&T face data set
Experiment results
Automatically reports there are 10 classes.
Prototypes of every classes are reported.
With such prototypes, recognition ratio (1-NN rule) is 90%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Face recognition: AT&T face data set
Experiment results
Automatically reports there are 10 classes.
Prototypes of every classes are reported.
With such prototypes, recognition ratio (1-NN rule) is 90%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Prototype-based classifier: based on 1-NN or k-NN rule
Nearest Neighbor Classifier (NNC): all training data as
prototypes
Nearest Mean Classifier (NMC): mean of each class as
prototypes
k-means classifier (KMC), Learning Vector Quantization
(LVQ), and others: predefine number of prototypes for every
class.
Main difficulty
1 How to find enough prototypes without overfitting
2 How to realize Incremental learning
Incremental of new data inside one class (non-stationary or
concept drift);
Incremental of new classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Prototype-based classifier: based on 1-NN or k-NN rule
Nearest Neighbor Classifier (NNC): all training data as
prototypes
Nearest Mean Classifier (NMC): mean of each class as
prototypes
k-means classifier (KMC), Learning Vector Quantization
(LVQ), and others: predefine number of prototypes for every
class.
Main difficulty
1 How to find enough prototypes without overfitting
2 How to realize Incremental learning
Incremental of new data inside one class (non-stationary or
concept drift);
Incremental of new classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Prototype-based classifier: based on 1-NN or k-NN rule
Nearest Neighbor Classifier (NNC): all training data as
prototypes
Nearest Mean Classifier (NMC): mean of each class as
prototypes
k-means classifier (KMC), Learning Vector Quantization
(LVQ), and others: predefine number of prototypes for every
class.
Main difficulty
1 How to find enough prototypes without overfitting
2 How to realize Incremental learning
Incremental of new data inside one class (non-stationary or
concept drift);
Incremental of new classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Prototype-based classifier: based on 1-NN or k-NN rule
Nearest Neighbor Classifier (NNC): all training data as
prototypes
Nearest Mean Classifier (NMC): mean of each class as
prototypes
k-means classifier (KMC), Learning Vector Quantization
(LVQ), and others: predefine number of prototypes for every
class.
Main difficulty
1 How to find enough prototypes without overfitting
2 How to realize Incremental learning
Incremental of new data inside one class (non-stationary or
concept drift);
Incremental of new classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Prototype-based classifier: based on 1-NN or k-NN rule
Nearest Neighbor Classifier (NNC): all training data as
prototypes
Nearest Mean Classifier (NMC): mean of each class as
prototypes
k-means classifier (KMC), Learning Vector Quantization
(LVQ), and others: predefine number of prototypes for every
class.
Main difficulty
1 How to find enough prototypes without overfitting
2 How to realize Incremental learning
Incremental of new data inside one class (non-stationary or
concept drift);
Incremental of new classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Prototype-based classifier: based on 1-NN or k-NN rule
Nearest Neighbor Classifier (NNC): all training data as
prototypes
Nearest Mean Classifier (NMC): mean of each class as
prototypes
k-means classifier (KMC), Learning Vector Quantization
(LVQ), and others: predefine number of prototypes for every
class.
Main difficulty
1 How to find enough prototypes without overfitting
2 How to realize Incremental learning
Incremental of new data inside one class (non-stationary or
concept drift);
Incremental of new classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Prototype-based classifier: based on 1-NN or k-NN rule
Nearest Neighbor Classifier (NNC): all training data as
prototypes
Nearest Mean Classifier (NMC): mean of each class as
prototypes
k-means classifier (KMC), Learning Vector Quantization
(LVQ), and others: predefine number of prototypes for every
class.
Main difficulty
1 How to find enough prototypes without overfitting
2 How to realize Incremental learning
Incremental of new data inside one class (non-stationary or
concept drift);
Incremental of new classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Prototype-based classifier: based on 1-NN or k-NN rule
Nearest Neighbor Classifier (NNC): all training data as
prototypes
Nearest Mean Classifier (NMC): mean of each class as
prototypes
k-means classifier (KMC), Learning Vector Quantization
(LVQ), and others: predefine number of prototypes for every
class.
Main difficulty
1 How to find enough prototypes without overfitting
2 How to realize Incremental learning
Incremental of new data inside one class (non-stationary or
concept drift);
Incremental of new classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Prototype-based classifier: based on 1-NN or k-NN rule
Nearest Neighbor Classifier (NNC): all training data as
prototypes
Nearest Mean Classifier (NMC): mean of each class as
prototypes
k-means classifier (KMC), Learning Vector Quantization
(LVQ), and others: predefine number of prototypes for every
class.
Main difficulty
1 How to find enough prototypes without overfitting
2 How to realize Incremental learning
Incremental of new data inside one class (non-stationary or
concept drift);
Incremental of new classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Prototype-based classifier: based on 1-NN or k-NN rule
Nearest Neighbor Classifier (NNC): all training data as
prototypes
Nearest Mean Classifier (NMC): mean of each class as
prototypes
k-means classifier (KMC), Learning Vector Quantization
(LVQ), and others: predefine number of prototypes for every
class.
Main difficulty
1 How to find enough prototypes without overfitting
2 How to realize Incremental learning
Incremental of new data inside one class (non-stationary or
concept drift);
Incremental of new classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN for supervised learning: Targets
Automatically learn the number of prototypes needed to
represent every class
Only the prototypes used to determine the decision boundary
will be remained
Realize both types of incremental learning
Robust to noise
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN for supervised learning: Targets
Automatically learn the number of prototypes needed to
represent every class
Only the prototypes used to determine the decision boundary
will be remained
Realize both types of incremental learning
Robust to noise
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN for supervised learning: Targets
Automatically learn the number of prototypes needed to
represent every class
Only the prototypes used to determine the decision boundary
will be remained
Realize both types of incremental learning
Robust to noise
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN for supervised learning: Targets
Automatically learn the number of prototypes needed to
represent every class
Only the prototypes used to determine the decision boundary
will be remained
Realize both types of incremental learning
Robust to noise
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN for supervised learning: Targets
Automatically learn the number of prototypes needed to
represent every class
Only the prototypes used to determine the decision boundary
will be remained
Realize both types of incremental learning
Robust to noise
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Adjusted SOINN Classifier (ASC)
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN learns k for
k-means.
Noise-reduction removes
noisy prototypes
Center-cleaning removes
prototypes unuseful for
decision
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Adjusted SOINN Classifier (ASC)
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN learns k for
k-means.
Noise-reduction removes
noisy prototypes
Center-cleaning removes
prototypes unuseful for
decision
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Adjusted SOINN Classifier (ASC)
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN learns k for
k-means.
Noise-reduction removes
noisy prototypes
Center-cleaning removes
prototypes unuseful for
decision
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Adjusted SOINN Classifier (ASC)
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN learns k for
k-means.
Noise-reduction removes
noisy prototypes
Center-cleaning removes
prototypes unuseful for
decision
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
ASC: noise-reduction & center-cleaning
Noise-reduction
If the label of a node differs from the label of majority voting of its
k-neighbors, it is considered an outlier.
Center-cleaning
If a prototype of class i has never been the nearest prototype of
other classes, remove the prototype.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
ASC: noise-reduction & center-cleaning
Noise-reduction
If the label of a node differs from the label of majority voting of its
k-neighbors, it is considered an outlier.
Center-cleaning
If a prototype of class i has never been the nearest prototype of
other classes, remove the prototype.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
ASC: noise-reduction & center-cleaning
Noise-reduction
If the label of a node differs from the label of majority voting of its
k-neighbors, it is considered an outlier.
Center-cleaning
If a prototype of class i has never been the nearest prototype of
other classes, remove the prototype.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
ASC: noise-reduction & center-cleaning
Noise-reduction
If the label of a node differs from the label of majority voting of its
k-neighbors, it is considered an outlier.
Center-cleaning
If a prototype of class i has never been the nearest prototype of
other classes, remove the prototype.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
ASC: noise-reduction & center-cleaning
Noise-reduction
If the label of a node differs from the label of majority voting of its
k-neighbors, it is considered an outlier.
Center-cleaning
If a prototype of class i has never been the nearest prototype of
other classes, remove the prototype.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (I)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 6; Recognition ratio = 100%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (I)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 6; Recognition ratio = 100%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (I)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 6; Recognition ratio = 100%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (I)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 6; Recognition ratio = 100%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (I)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 6; Recognition ratio = 100%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (II)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 86; Recognition ratio = 98%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (II)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 86; Recognition ratio = 98%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (II)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 86; Recognition ratio = 98%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (II)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 86; Recognition ratio = 98%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (II)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 86; Recognition ratio = 98%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (III)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 87; Recognition ratio = 97.8%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (III)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 87; Recognition ratio = 97.8%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (III)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 87; Recognition ratio = 97.8%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (III)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 87; Recognition ratio = 97.8%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: artificial data (III)
Original data SOINN results ASC results
Test results of ASC
No. of prototypes = 87; Recognition ratio = 97.8%.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Experiment results: optdigits
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
ASC with different parameter sets (ad,λ), displayed with average
of 10 times training and standard deviation
Parameter set of {ad, λ}
(50, 50) (25, 25) (10, 10)
recognition ratio (%) 97.7 ± 0.2 97.4 ± 0.2 97.0 ± 0.2
No. of prototypes 377 ± 12 258 ± 7 112 ± 7
Compression ratio (%) 9.9 ± 0.3 6.8 ± 0.2 2.9 ± 0.2
Compare with SVM and 1-NN
LibSVM: 1197 support vectors; Recognition ratio = 96.6%.
1-NN: best classifier (98%). All 3823 samples as prototypes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Experiment results: optdigits
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
ASC with different parameter sets (ad,λ), displayed with average
of 10 times training and standard deviation
Parameter set of {ad, λ}
(50, 50) (25, 25) (10, 10)
recognition ratio (%) 97.7 ± 0.2 97.4 ± 0.2 97.0 ± 0.2
No. of prototypes 377 ± 12 258 ± 7 112 ± 7
Compression ratio (%) 9.9 ± 0.3 6.8 ± 0.2 2.9 ± 0.2
Compare with SVM and 1-NN
LibSVM: 1197 support vectors; Recognition ratio = 96.6%.
1-NN: best classifier (98%). All 3823 samples as prototypes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Experiment results: optdigits
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
ASC with different parameter sets (ad,λ), displayed with average
of 10 times training and standard deviation
Parameter set of {ad, λ}
(50, 50) (25, 25) (10, 10)
recognition ratio (%) 97.7 ± 0.2 97.4 ± 0.2 97.0 ± 0.2
No. of prototypes 377 ± 12 258 ± 7 112 ± 7
Compression ratio (%) 9.9 ± 0.3 6.8 ± 0.2 2.9 ± 0.2
Compare with SVM and 1-NN
LibSVM: 1197 support vectors; Recognition ratio = 96.6%.
1-NN: best classifier (98%). All 3823 samples as prototypes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Experiment results: optdigits
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
ASC with different parameter sets (ad,λ), displayed with average
of 10 times training and standard deviation
Parameter set of {ad, λ}
(50, 50) (25, 25) (10, 10)
recognition ratio (%) 97.7 ± 0.2 97.4 ± 0.2 97.0 ± 0.2
No. of prototypes 377 ± 12 258 ± 7 112 ± 7
Compression ratio (%) 9.9 ± 0.3 6.8 ± 0.2 2.9 ± 0.2
Compare with SVM and 1-NN
LibSVM: 1197 support vectors; Recognition ratio = 96.6%.
1-NN: best classifier (98%). All 3823 samples as prototypes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: UCI repository data sets
Comparison results of ASC and other classifiers: recognition ratio
Data set ASC (ad, λ) NSC (σ 2 max) KMC (M) NNC (k) LVQ (M)
Iris 97.4 ± 0.86 96.3 ± 0.4 96.2 ± 0.8 96.7 ± 0.6 96.1 ± 0.6
Breast cancer 97.4 ± 0.38 97.2 ± 0.2 95.9 ± 0.3 97.0 ± 0.2 96.3 ± 0.4
Ionosphere 90.4 ± 0.64 91.9 ± 0.8 87.4 ± 0.6 86.1 ± 0.7 86.4 ± 0.8
Glass 73.5 ± 1.6 70.2 ± 1.5 68.8 ± 1.1 72.3 ± 1.2 68.3 ± 2.0
Liver disorders 62.6 ± 0.83 62.9 ± 2.3 59.3 ± 2.3 67.3 ± 1.6 66.3 ± 1.9
Pima Indians 72.0 ± 0.63 68.6 ± 1.6 68.7 ± 0.9 74.7 ± 0.7 73.5 ± 0.9
Wine 82.6 ± 1.55 75.3 ± 1.7 71.9 ± 1.9 73.9 ± 1.9 72.3 ± 1.5
Average 82.3 ± 0.93 80.4 ± 1.2 78.3 ± 1.1 81.1 ± 0.99 79.9 ± 1.2
In average, ASC has best recognition performance.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: UCI repository data sets
Comparison results of ASC and other classifiers: recognition ratio
Data set ASC (ad, λ) NSC (σ 2 max) KMC (M) NNC (k) LVQ (M)
Iris 97.4 ± 0.86 96.3 ± 0.4 96.2 ± 0.8 96.7 ± 0.6 96.1 ± 0.6
Breast cancer 97.4 ± 0.38 97.2 ± 0.2 95.9 ± 0.3 97.0 ± 0.2 96.3 ± 0.4
Ionosphere 90.4 ± 0.64 91.9 ± 0.8 87.4 ± 0.6 86.1 ± 0.7 86.4 ± 0.8
Glass 73.5 ± 1.6 70.2 ± 1.5 68.8 ± 1.1 72.3 ± 1.2 68.3 ± 2.0
Liver disorders 62.6 ± 0.83 62.9 ± 2.3 59.3 ± 2.3 67.3 ± 1.6 66.3 ± 1.9
Pima Indians 72.0 ± 0.63 68.6 ± 1.6 68.7 ± 0.9 74.7 ± 0.7 73.5 ± 0.9
Wine 82.6 ± 1.55 75.3 ± 1.7 71.9 ± 1.9 73.9 ± 1.9 72.3 ± 1.5
Average 82.3 ± 0.93 80.4 ± 1.2 78.3 ± 1.1 81.1 ± 0.99 79.9 ± 1.2
In average, ASC has best recognition performance.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: UCI repository data sets (continue)
Comparison results of ASC and other classifiers: compression ratio
Data set ASC (a ∗ d, λ ∗ ) NSC (σ 2 ∗ ∗ ∗ ∗
max ) KMC (M ) NNC (k ) LVQ (M )
Iris 5.2 (6, 6) 7.3 (0.25) 8.0 (4) 100 (14) 15 (22)
Breast cancer 1.4 (8,8) 1.8 (35.0) 0.29 (1) 100 (5) 5.9 (40)
Ionosphere 3.4 (15, 15) 31 (1.25) 4.0 (7) 100 (2) 6.8 (24)
Glass 13.7 (15, 15) 97 (0.005) 17 (6) 100 (1) 45 (97)
Liver disorders 4.6 (6, 6) 4.9 (600) 11 (19) 100 (14) 8.4 (29)
Pima Indians 0.6 (6, 6) 1.7 (2600) 1.0 (4) 100 (17) 3.4 (26)
Wine 3.2 (6, 6) 96 (4.0) 29 (17) 100 (1) 32 (57)
Average 4.6 34.2 10.0 100 16.6
In average, ASC has best compression ratio.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment results: UCI repository data sets (continue)
Comparison results of ASC and other classifiers: compression ratio
Data set ASC (a ∗ d, λ ∗ ) NSC (σ 2 ∗ ∗ ∗ ∗
max ) KMC (M ) NNC (k ) LVQ (M )
Iris 5.2 (6, 6) 7.3 (0.25) 8.0 (4) 100 (14) 15 (22)
Breast cancer 1.4 (8,8) 1.8 (35.0) 0.29 (1) 100 (5) 5.9 (40)
Ionosphere 3.4 (15, 15) 31 (1.25) 4.0 (7) 100 (2) 6.8 (24)
Glass 13.7 (15, 15) 97 (0.005) 17 (6) 100 (1) 45 (97)
Liver disorders 4.6 (6, 6) 4.9 (600) 11 (19) 100 (14) 8.4 (29)
Pima Indians 0.6 (6, 6) 1.7 (2600) 1.0 (4) 100 (17) 3.4 (26)
Wine 3.2 (6, 6) 96 (4.0) 29 (17) 100 (1) 32 (57)
Average 4.6 34.2 10.0 100 16.6
In average, ASC has best compression ratio.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Requirement of Semi-supervised learning
Labeled instances are difficult, expensive, or time consuming
to obtain.
How can a system use large amount of unlabeled data with
limited labeled data to built good classifiers?
New data are continually added to an already huge database
How can a system learn new knowledge without forgetting
previous learned knowledge?
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Requirement of Semi-supervised learning
Labeled instances are difficult, expensive, or time consuming
to obtain.
How can a system use large amount of unlabeled data with
limited labeled data to built good classifiers?
New data are continually added to an already huge database
How can a system learn new knowledge without forgetting
previous learned knowledge?
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Requirement of Semi-supervised learning
Labeled instances are difficult, expensive, or time consuming
to obtain.
How can a system use large amount of unlabeled data with
limited labeled data to built good classifiers?
New data are continually added to an already huge database
How can a system learn new knowledge without forgetting
previous learned knowledge?
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Requirement of Semi-supervised learning
Labeled instances are difficult, expensive, or time consuming
to obtain.
How can a system use large amount of unlabeled data with
limited labeled data to built good classifiers?
New data are continually added to an already huge database
How can a system learn new knowledge without forgetting
previous learned knowledge?
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Requirement of Semi-supervised learning
Labeled instances are difficult, expensive, or time consuming
to obtain.
How can a system use large amount of unlabeled data with
limited labeled data to built good classifiers?
New data are continually added to an already huge database
How can a system learn new knowledge without forgetting
previous learned knowledge?
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN used for Semi-supervised learning
1 SOINN:represent topology,
incremental learning;
2 Labeled data: label nodes
(winner);
3 Division of a cluster
Condition of division
Rc−1 ≤ Rc&Rc > Rc+1 (11)
Rc =
dis(wa, wc) (12)
a∈Nc
c-1: former node
c+1: unlabeled neighbors.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN used for Semi-supervised learning
1 SOINN:represent topology,
incremental learning;
2 Labeled data: label nodes
(winner);
3 Division of a cluster
Condition of division
Rc−1 ≤ Rc&Rc > Rc+1 (11)
Rc =
dis(wa, wc) (12)
a∈Nc
c-1: former node
c+1: unlabeled neighbors.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN used for Semi-supervised learning
1 SOINN:represent topology,
incremental learning;
2 Labeled data: label nodes
(winner);
3 Division of a cluster
Condition of division
Rc−1 ≤ Rc&Rc > Rc+1 (11)
Rc =
dis(wa, wc) (12)
a∈Nc
c-1: former node
c+1: unlabeled neighbors.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN used for Semi-supervised learning
1 SOINN:represent topology,
incremental learning;
2 Labeled data: label nodes
(winner);
3 Division of a cluster
Condition of division
Rc−1 ≤ Rc&Rc > Rc+1 (11)
Rc =
dis(wa, wc) (12)
a∈Nc
c-1: former node
c+1: unlabeled neighbors.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
SOINN used for Semi-supervised learning
1 SOINN:represent topology,
incremental learning;
2 Labeled data: label nodes
(winner);
3 Division of a cluster
Condition of division
Rc−1 ≤ Rc&Rc > Rc+1 (11)
Rc =
dis(wa, wc) (12)
a∈Nc
c-1: former node
c+1: unlabeled neighbors.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Experiment: original data
5%, 15%, or 40% overlap
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
training samples 500, validation samples 5,000, and test
samples 5,000
labeled samples: 10% and 20%
light blue: unlabeled data; others: labeled data
- - - - - ideal decision boundary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Experiment: original data
5%, 15%, or 40% overlap
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
training samples 500, validation samples 5,000, and test
samples 5,000
labeled samples: 10% and 20%
light blue: unlabeled data; others: labeled data
- - - - - ideal decision boundary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Experiment: original data
5%, 15%, or 40% overlap
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
training samples 500, validation samples 5,000, and test
samples 5,000
labeled samples: 10% and 20%
light blue: unlabeled data; others: labeled data
- - - - - ideal decision boundary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Experiment: original data
5%, 15%, or 40% overlap
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
training samples 500, validation samples 5,000, and test
samples 5,000
labeled samples: 10% and 20%
light blue: unlabeled data; others: labeled data
- - - - - ideal decision boundary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Experiment: original data
5%, 15%, or 40% overlap
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
training samples 500, validation samples 5,000, and test
samples 5,000
labeled samples: 10% and 20%
light blue: unlabeled data; others: labeled data
- - - - - ideal decision boundary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Experiment: original data
5%, 15%, or 40% overlap
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
training samples 500, validation samples 5,000, and test
samples 5,000
labeled samples: 10% and 20%
light blue: unlabeled data; others: labeled data
- - - - - ideal decision boundary
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Experiment results
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Separate classes
with few labeled
samples.
For UCI data sets,
work better than
other typical
methods.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Experiment results
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Separate classes
with few labeled
samples.
For UCI data sets,
work better than
other typical
methods.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Experiment results
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Separate classes
with few labeled
samples.
For UCI data sets,
work better than
other typical
methods.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
SOINN used for active learning
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Targets: Actively ask for label of some samples to label all
classes
Idea:
1 Use SOINN to learn the topology structure of input data.
2 Actively label the vertex nodes of every class
3 Use vertex nodes to label all nodes.
4 Actively label the nodes lie in the overlapped area.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
SOINN used for active learning
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Targets: Actively ask for label of some samples to label all
classes
Idea:
1 Use SOINN to learn the topology structure of input data.
2 Actively label the vertex nodes of every class
3 Use vertex nodes to label all nodes.
4 Actively label the nodes lie in the overlapped area.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
SOINN used for active learning
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Targets: Actively ask for label of some samples to label all
classes
Idea:
1 Use SOINN to learn the topology structure of input data.
2 Actively label the vertex nodes of every class
3 Use vertex nodes to label all nodes.
4 Actively label the nodes lie in the overlapped area.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
SOINN used for active learning
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Targets: Actively ask for label of some samples to label all
classes
Idea:
1 Use SOINN to learn the topology structure of input data.
2 Actively label the vertex nodes of every class
3 Use vertex nodes to label all nodes.
4 Actively label the nodes lie in the overlapped area.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
SOINN used for active learning
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Targets: Actively ask for label of some samples to label all
classes
Idea:
1 Use SOINN to learn the topology structure of input data.
2 Actively label the vertex nodes of every class
3 Use vertex nodes to label all nodes.
4 Actively label the nodes lie in the overlapped area.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
SOINN used for active learning
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Targets: Actively ask for label of some samples to label all
classes
Idea:
1 Use SOINN to learn the topology structure of input data.
2 Actively label the vertex nodes of every class
3 Use vertex nodes to label all nodes.
4 Actively label the nodes lie in the overlapped area.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
SOINN used for active learning
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Targets: Actively ask for label of some samples to label all
classes
Idea:
1 Use SOINN to learn the topology structure of input data.
2 Actively label the vertex nodes of every class
3 Use vertex nodes to label all nodes.
4 Actively label the nodes lie in the overlapped area.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment: artificial data set under stationary
environment
Original data: Four classes in all, with 10% noise.
Results: under stationary environment; 10 teacher vectors.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment: artificial data set under stationary
environment
Original data: Four classes in all, with 10% noise.
Results: under stationary environment; 10 teacher vectors.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment: artificial data set under stationary
environment
Original data: Four classes in all, with 10% noise.
Results: under stationary environment; 10 teacher vectors.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment: artificial data set under non-stationary
environment
16 teacher vectors are asked.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Unsupervised learning
Supervised learning
Semi-supervised learning
Active learning
Experiment: artificial data set under non-stationary
environment
16 teacher vectors are asked.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

1 What is SOINN
2 Why SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
3 Detail algorithm of SOINN
4 SOINN for machine learning
5 SOINN for associative memory
6 References
Background
SOINN-AM
Experiments
General Associative Memory
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
Background: typical associative memory systems
Distributed Learning Associative Memory:
Hopfield Network: most famous network, for auto-associative
memory
Bidirectional Associative Memory (BAM), for
hetero-associative memory
Competitive Learning Associative Memory
KFMAM: Kohonon feature map associative memory.
Difficulties
Forget previously learned knowledge when learning new
knowledge incrementally.
Storage limitation.
Memory real-valued data.
Many-to-Many associate.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
Background: typical associative memory systems
Distributed Learning Associative Memory:
Hopfield Network: most famous network, for auto-associative
memory
Bidirectional Associative Memory (BAM), for
hetero-associative memory
Competitive Learning Associative Memory
KFMAM: Kohonon feature map associative memory.
Difficulties
Forget previously learned knowledge when learning new
knowledge incrementally.
Storage limitation.
Memory real-valued data.
Many-to-Many associate.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
Background: typical associative memory systems
Distributed Learning Associative Memory:
Hopfield Network: most famous network, for auto-associative
memory
Bidirectional Associative Memory (BAM), for
hetero-associative memory
Competitive Learning Associative Memory
KFMAM: Kohonon feature map associative memory.
Difficulties
Forget previously learned knowledge when learning new
knowledge incrementally.
Storage limitation.
Memory real-valued data.
Many-to-Many associate.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
Background: typical associative memory systems
Distributed Learning Associative Memory:
Hopfield Network: most famous network, for auto-associative
memory
Bidirectional Associative Memory (BAM), for
hetero-associative memory
Competitive Learning Associative Memory
KFMAM: Kohonon feature map associative memory.
Difficulties
Forget previously learned knowledge when learning new
knowledge incrementally.
Storage limitation.
Memory real-valued data.
Many-to-Many associate.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
Background: typical associative memory systems
Distributed Learning Associative Memory:
Hopfield Network: most famous network, for auto-associative
memory
Bidirectional Associative Memory (BAM), for
hetero-associative memory
Competitive Learning Associative Memory
KFMAM: Kohonon feature map associative memory.
Difficulties
Forget previously learned knowledge when learning new
knowledge incrementally.
Storage limitation.
Memory real-valued data.
Many-to-Many associate.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
Background: typical associative memory systems
Distributed Learning Associative Memory:
Hopfield Network: most famous network, for auto-associative
memory
Bidirectional Associative Memory (BAM), for
hetero-associative memory
Competitive Learning Associative Memory
KFMAM: Kohonon feature map associative memory.
Difficulties
Forget previously learned knowledge when learning new
knowledge incrementally.
Storage limitation.
Memory real-valued data.
Many-to-Many associate.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
Background: typical associative memory systems
Distributed Learning Associative Memory:
Hopfield Network: most famous network, for auto-associative
memory
Bidirectional Associative Memory (BAM), for
hetero-associative memory
Competitive Learning Associative Memory
KFMAM: Kohonon feature map associative memory.
Difficulties
Forget previously learned knowledge when learning new
knowledge incrementally.
Storage limitation.
Memory real-valued data.
Many-to-Many associate.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
Background: typical associative memory systems
Distributed Learning Associative Memory:
Hopfield Network: most famous network, for auto-associative
memory
Bidirectional Associative Memory (BAM), for
hetero-associative memory
Competitive Learning Associative Memory
KFMAM: Kohonon feature map associative memory.
Difficulties
Forget previously learned knowledge when learning new
knowledge incrementally.
Storage limitation.
Memory real-valued data.
Many-to-Many associate.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
Background: typical associative memory systems
Distributed Learning Associative Memory:
Hopfield Network: most famous network, for auto-associative
memory
Bidirectional Associative Memory (BAM), for
hetero-associative memory
Competitive Learning Associative Memory
KFMAM: Kohonon feature map associative memory.
Difficulties
Forget previously learned knowledge when learning new
knowledge incrementally.
Storage limitation.
Memory real-valued data.
Many-to-Many associate.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
Background: typical associative memory systems
Distributed Learning Associative Memory:
Hopfield Network: most famous network, for auto-associative
memory
Bidirectional Associative Memory (BAM), for
hetero-associative memory
Competitive Learning Associative Memory
KFMAM: Kohonon feature map associative memory.
Difficulties
Forget previously learned knowledge when learning new
knowledge incrementally.
Storage limitation.
Memory real-valued data.
Many-to-Many associate.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
Background: typical associative memory systems
Distributed Learning Associative Memory:
Hopfield Network: most famous network, for auto-associative
memory
Bidirectional Associative Memory (BAM), for
hetero-associative memory
Competitive Learning Associative Memory
KFMAM: Kohonon feature map associative memory.
Difficulties
Forget previously learned knowledge when learning new
knowledge incrementally.
Storage limitation.
Memory real-valued data.
Many-to-Many associate.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Objectives of SOINN-AM
Incremental learning of memory pairs.
Robust for noise data.
Dealing with real-valued data.
Many-to-many association.
Background
SOINN-AM
Experiments
General Associative Memory
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Objectives of SOINN-AM
Incremental learning of memory pairs.
Robust for noise data.
Dealing with real-valued data.
Many-to-many association.
Background
SOINN-AM
Experiments
General Associative Memory
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Objectives of SOINN-AM
Incremental learning of memory pairs.
Robust for noise data.
Dealing with real-valued data.
Many-to-many association.
Background
SOINN-AM
Experiments
General Associative Memory
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Objectives of SOINN-AM
Incremental learning of memory pairs.
Robust for noise data.
Dealing with real-valued data.
Many-to-many association.
Background
SOINN-AM
Experiments
General Associative Memory
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Objectives of SOINN-AM
Incremental learning of memory pairs.
Robust for noise data.
Dealing with real-valued data.
Many-to-many association.
Background
SOINN-AM
Experiments
General Associative Memory
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Objectives of SOINN-AM
Incremental learning of memory pairs.
Robust for noise data.
Dealing with real-valued data.
Many-to-many association.
Background
SOINN-AM
Experiments
General Associative Memory
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture of SOINN-AM
Background
SOINN-AM
Experiments
General Associative Memory
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Algorithms of SOINN-AM
Basic idea of memory phase
Background
SOINN-AM
Experiments
General Associative Memory
1 Combine key vector and associate vector as input data.
2 Use SOINN to learn such input data.
Basic idea of recall phase
1 Using key part of nodes to find winner node for key vector,
the distance is d.
2 If d ≤ ǫ, output the associative part of winner as the recall
results.
3 If d > ǫ, report unknown for key vector.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Algorithms of SOINN-AM
Basic idea of memory phase
Background
SOINN-AM
Experiments
General Associative Memory
1 Combine key vector and associate vector as input data.
2 Use SOINN to learn such input data.
Basic idea of recall phase
1 Using key part of nodes to find winner node for key vector,
the distance is d.
2 If d ≤ ǫ, output the associative part of winner as the recall
results.
3 If d > ǫ, report unknown for key vector.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Algorithms of SOINN-AM
Basic idea of memory phase
Background
SOINN-AM
Experiments
General Associative Memory
1 Combine key vector and associate vector as input data.
2 Use SOINN to learn such input data.
Basic idea of recall phase
1 Using key part of nodes to find winner node for key vector,
the distance is d.
2 If d ≤ ǫ, output the associative part of winner as the recall
results.
3 If d > ǫ, report unknown for key vector.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Algorithms of SOINN-AM
Basic idea of memory phase
Background
SOINN-AM
Experiments
General Associative Memory
1 Combine key vector and associate vector as input data.
2 Use SOINN to learn such input data.
Basic idea of recall phase
1 Using key part of nodes to find winner node for key vector,
the distance is d.
2 If d ≤ ǫ, output the associative part of winner as the recall
results.
3 If d > ǫ, report unknown for key vector.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Algorithms of SOINN-AM
Basic idea of memory phase
Background
SOINN-AM
Experiments
General Associative Memory
1 Combine key vector and associate vector as input data.
2 Use SOINN to learn such input data.
Basic idea of recall phase
1 Using key part of nodes to find winner node for key vector,
the distance is d.
2 If d ≤ ǫ, output the associative part of winner as the recall
results.
3 If d > ǫ, report unknown for key vector.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Algorithms of SOINN-AM
Basic idea of memory phase
Background
SOINN-AM
Experiments
General Associative Memory
1 Combine key vector and associate vector as input data.
2 Use SOINN to learn such input data.
Basic idea of recall phase
1 Using key part of nodes to find winner node for key vector,
the distance is d.
2 If d ≤ ǫ, output the associative part of winner as the recall
results.
3 If d > ǫ, report unknown for key vector.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Algorithms of SOINN-AM
Basic idea of memory phase
Background
SOINN-AM
Experiments
General Associative Memory
1 Combine key vector and associate vector as input data.
2 Use SOINN to learn such input data.
Basic idea of recall phase
1 Using key part of nodes to find winner node for key vector,
the distance is d.
2 If d ≤ ǫ, output the associative part of winner as the recall
results.
3 If d > ǫ, report unknown for key vector.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Algorithms of SOINN-AM
Basic idea of memory phase
Background
SOINN-AM
Experiments
General Associative Memory
1 Combine key vector and associate vector as input data.
2 Use SOINN to learn such input data.
Basic idea of recall phase
1 Using key part of nodes to find winner node for key vector,
the distance is d.
2 If d ≤ ǫ, output the associative part of winner as the recall
results.
3 If d > ǫ, report unknown for key vector.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Original data
Binary data
Real-valued data
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
Comparison with typical AM systems
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Robustness of noise
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Background
SOINN-AM
Experiments
General Associative Memory
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Many-to-Many associate testing
Background
SOINN-AM
Experiments
General Associative Memory
SOINN-AM recalls all patterns perfectly.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture and basic idea of GAM
Background
SOINN-AM
Experiments
General Associative Memory
Input layer: key vector
and associate vector.
Memory layer: Memory
patterns with classes.
Associate layer: Build
association between
classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture and basic idea of GAM
Background
SOINN-AM
Experiments
General Associative Memory
Input layer: key vector
and associate vector.
Memory layer: Memory
patterns with classes.
Associate layer: Build
association between
classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture and basic idea of GAM
Background
SOINN-AM
Experiments
General Associative Memory
Input layer: key vector
and associate vector.
Memory layer: Memory
patterns with classes.
Associate layer: Build
association between
classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
Architecture and basic idea of GAM
Background
SOINN-AM
Experiments
General Associative Memory
Input layer: key vector
and associate vector.
Memory layer: Memory
patterns with classes.
Associate layer: Build
association between
classes.
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

1 What is SOINN
2 Why SOINN
Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
3 Detail algorithm of SOINN
4 SOINN for machine learning
5 SOINN for associative memory
6 References
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
References about SOINN
SOINN for unsupervised learning:
Furao Shen and Osamu Hasegawa, ”An Incremental Network for On-line
Unsupervised Classification and Topology Learning”, Neural Networks,
Vol.19, No.1, pp.90-106, (2005)
Furao Shen, Tomotaka Ogura and Osamu Hasegawa, ”An enhanced
self-organizing incremental neural network for online unsupervised
learning”, Neural Networks, Vol.20, No.8, pp.893-903, (2007)
SOINN for Supervised learning:
Furao Shen and Osamu Hasegawa, ”A Fast Nearest Neighbor Classifier
Based on Self-organizing Incremental Neural Network”, Neural Networks,
Vol.21, No.10, pp1537-1547, (2008)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
References about SOINN
SOINN for unsupervised learning:
Furao Shen and Osamu Hasegawa, ”An Incremental Network for On-line
Unsupervised Classification and Topology Learning”, Neural Networks,
Vol.19, No.1, pp.90-106, (2005)
Furao Shen, Tomotaka Ogura and Osamu Hasegawa, ”An enhanced
self-organizing incremental neural network for online unsupervised
learning”, Neural Networks, Vol.20, No.8, pp.893-903, (2007)
SOINN for Supervised learning:
Furao Shen and Osamu Hasegawa, ”A Fast Nearest Neighbor Classifier
Based on Self-organizing Incremental Neural Network”, Neural Networks,
Vol.21, No.10, pp1537-1547, (2008)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
References about SOINN
SOINN for unsupervised learning:
Furao Shen and Osamu Hasegawa, ”An Incremental Network for On-line
Unsupervised Classification and Topology Learning”, Neural Networks,
Vol.19, No.1, pp.90-106, (2005)
Furao Shen, Tomotaka Ogura and Osamu Hasegawa, ”An enhanced
self-organizing incremental neural network for online unsupervised
learning”, Neural Networks, Vol.20, No.8, pp.893-903, (2007)
SOINN for Supervised learning:
Furao Shen and Osamu Hasegawa, ”A Fast Nearest Neighbor Classifier
Based on Self-organizing Incremental Neural Network”, Neural Networks,
Vol.21, No.10, pp1537-1547, (2008)
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
References about SOINN
SOINN for Semi-supervised and active learning
Youki Kamiya, Toshiaki Ishii, Furao Shen and Osamu Hasegawa: ”An
Online Semi-Supervised Clustering Algorithm Based on a Self-organizing
Incremental Neural Network,” IJCNN 2007, Orlando, FL, USA, August
2007
Furao Shen, Keisuke Sakurai, Youki Kamiya and Osamu Hasegawa: ”An
Online Semi-supervised Active Learning Algorithm with Self-organizing
Incremental Neural Network,” IJCNN 2007, Orlando, FL, USA, August
2007
SOINN for Associative Memory:
Sudo Akihito; Sato Akihiro; Hasegawa Osamu, ”Associative Memory for
Online Learning in Noisy Environments Using Self-organizing Incremental
Neural Network”, IEEE Transactions on Neural Networks, (2009) in press
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
References about SOINN
SOINN for Semi-supervised and active learning
Youki Kamiya, Toshiaki Ishii, Furao Shen and Osamu Hasegawa: ”An
Online Semi-Supervised Clustering Algorithm Based on a Self-organizing
Incremental Neural Network,” IJCNN 2007, Orlando, FL, USA, August
2007
Furao Shen, Keisuke Sakurai, Youki Kamiya and Osamu Hasegawa: ”An
Online Semi-supervised Active Learning Algorithm with Self-organizing
Incremental Neural Network,” IJCNN 2007, Orlando, FL, USA, August
2007
SOINN for Associative Memory:
Sudo Akihito; Sato Akihiro; Hasegawa Osamu, ”Associative Memory for
Online Learning in Noisy Environments Using Self-organizing Incremental
Neural Network”, IEEE Transactions on Neural Networks, (2009) in press
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
References about SOINN
SOINN for Semi-supervised and active learning
Youki Kamiya, Toshiaki Ishii, Furao Shen and Osamu Hasegawa: ”An
Online Semi-Supervised Clustering Algorithm Based on a Self-organizing
Incremental Neural Network,” IJCNN 2007, Orlando, FL, USA, August
2007
Furao Shen, Keisuke Sakurai, Youki Kamiya and Osamu Hasegawa: ”An
Online Semi-supervised Active Learning Algorithm with Self-organizing
Incremental Neural Network,” IJCNN 2007, Orlando, FL, USA, August
2007
SOINN for Associative Memory:
Sudo Akihito; Sato Akihiro; Hasegawa Osamu, ”Associative Memory for
Online Learning in Noisy Environments Using Self-organizing Incremental
Neural Network”, IEEE Transactions on Neural Networks, (2009) in press
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application

Contents
What is SOINN
Why SOINN
Detail algorithm of SOINN
SOINN for machine learning
SOINN for associative memory
References
References about SOINN
Download papers and program of SOINN
http://www.isl.titech.ac.jp/˜ hasegawalab/soinn.html
F. Shen, O. Hasegawa Self-organizing incremental neural network and its application