Recursive Generalized Neural Networks (RGNN) for the Modeling of ...

Recursive Generalized Neural
Networks (RGNN) for the Modeling
of a Load Sensing Pump
Travis Wiens
Nutaksas Research
Auckland
New Zealand
travis@nutaksas.com
Rich Burton, Greg
Schoenau, Doug Bitner
University of Saskatchewan
Saskatoon
Canada

Greetings from Canada…
2

and New Zealand.
3

White Box Models
● Linearized models for small fluctuations
only
● Requires understanding
● Parameter determination: expensive and
difficult
● Aids understanding
4

● Krus, P. 1988.
● Erkkila, M. 1999.
White Box Models
● Kim, S.D. and Cho, H.S. 1988.
● Wu, D., et al. 2002.
5

Neural Network Black Box
Models
● Difficulty in selecting form:
● Number of layers
● Number of neurons
● Activation function
● Difficulty in training:
● Approximate derivatives
● Training time
● Does not aid understanding
6

Neural Network Black Box
Models
● McNamara, J., et al, 1997.
● Xu,X.P., et al, 1994.
● Watton, J. and Xue, Y., 1997.
● Y. Chen, et al, 1997.
● Xu, X.P., et al, 1994.
● Lamontagne, D., et al, 2003.
● Li, L., et al, 2006 and 2007.
7

Artificial Neural Network
● Large number of simple units
● Complex ensemble behavior
● Universal approximator
8

u 1
u 2
u N
Single Neuron
w 1
w 2
w N
+
tanh y
y=tanh(u 1*w 1+u 2*w 2+...+u N*w N)
9

Neural Network Black Box
Models
● Difficulty in selecting form:
● Number of layers
● Number of neurons
● Activation function
● Solution: Generalized Neural Network
10

Recurrent Generalized Neural
● Werbos, 1990
Network
● Maximum number of weights without
recurrence
● All other connections with delay (differs
from Werbos)
● No layers
● One structure parameter: number of
neurons
11

Recurrent Generalized Neural
Network
12

Neural Network Black Box
Models
● Difficulty in training:
● Approximate derivatives
● Training time
● Solution: Complex Method
13

Complex Method of
Optimization
● Box, M.J., 1965; Andersson, J., 2001; Xu,
X.P., 1994
● No derivatives required
● No function knowledge required (linear or
non-linear)
● Allows bounded parameters
14

Complex Method of
Optimization
1) Generate Population (random or
selected parameters)
2) Evaluate “Fitness” of each individual:
error in training data
3) Reflect worst individual through centroid
of others (distance plus 30%), checking
bounds
4) If new individual is still worst, move it
toward best individual
15

Problem
● Model a load sensing pump
● Experimental data:
● Source pressure, Ps
● Control piston pressure, Pc
● Pump flow, Q
● 12000 data points at 200 Hz
● Problem: model Q (Pc, Ps)
20

Load Sensing Pump
21

RGNN/Complex Parameters
● 30 Neurons
● 810 Weights
● Activation functions: tanh for hidden,
linear for output
● Bounds: forward weights: +/- 5, feedback
weights +/- 1
● Fitness: negative RMS error, number of
finite values for unstable parameters
22

● Insert Flow Graphs
Results
23

Results
24

Results
25

• Li, 2007
– Error on Training Data:
2.56e­005 m 3 /s
– Error on Verification
Data: 2.29e­005 m 3 /s
– Training Time: 10­12
hours
– Required considerable
“tweaking”
Comparison
• RGNN
– Error on Training Data:
1.18e­005 m 3 /s
– Error on Verification
Data: 1.52e­005 m 3 /s
– Training Time: 2.5 hours
– Minimal “tweaking”
26

● Initialization Effects
Results
27

● Few Parameters
● Little setup needed
● Faster Training
Conclusion
● Initialization Important
28

● Travis Wiens:
More Information
● travis@nutaksas.com
● Code/data:
● Nutaksas Blog:
http://blog.nutaksas.com
● Matlab Central File Exchange:
http://www.mathworks.com/matlabcentral/
29