State Algorithm - Ra.informatik.tu-darmstadt.de

ACRI 2006 Perpignan 

Optimal 6-State Algorithms for the Behavior of 

Several Moving Creatures 

Darmstadt University of Technology 

Computer Architecture Group 

Mathias Halbach, Rolf Hoffmann, Lars Both 

• Goal of the Project 

• Creature's Exploration Problem 

• CA Model 

– Cell Type 

– Actions of the Creature 

– Rule 

– Conflicts, 1-Phase Arbitration 

– Different Behaviors of a Creature 

– Number and Classes of State Algorithms 

• Procedure to find the best algorithms 

• Performance of the best algorithms 

• Several Creatures 

– Evaluation 

• Conclusion 

1

Goal of the Project in general 



• Challenge 

– Automatically detecting optimal rules for given CA problems, in 

particular problems with moving creatures/agents/robots 

– Why: State Explosion, Very time consuming 

• Solution 

– Find efficient algorithms for 

• selection of relevant rules 

• simulation 

• evaluation 

– Hardware Support 

• Applications 

– Artificial Worlds 

– Models of Real Worlds 

– Computational Tasks 

2

Creature's Exploration Problem 



• The Problem 

– Given is a 2D-CA with obstacles and 

moving creatures. 

– Goal: 

Find an optimal local rule for the 

creatures to visit a maximum 

number of empty cells with a 

minimum number of time steps for a 

given set of initial configurations. 

• Applications 

– Mowing a lawn in shortest time 

– Vacuum cleaning a room by a robot 

– Exploring an unknown environment 

3

The Actions of the Creatures 



L 

R 

Lm 

Rm 

(turn Left) 

(turn Right) 

(turn Left and move) 

move forward and 

simultaneously turn left 

(turn Right and move) 

move forward and 

simultaneously turn right 

4

The Rule for the Creatures 

FrontCell 

Legend 



(a1) if (obstacle or creature) then 

(a2) if (collision) then 

turn (L/R) 

turn (L/R) 

obstacle or 

creature 

irrelevant 

5 

(b) if not((a1) or (a2)) then turn and move(Lm/Rm) 

creature 

in one 

out of two 

directions

CA: Cell Type 



Type 

CREATURE 

EMPTY 

Direction r 

toNorth, toEast, 

toSouth, toWest 

(irrelevant) 

control state s 

0, 1, ... (N-1) 

OBSTACLE 

(irrelevant) 

6

CA: Rule 



Rule for My.Type = CREATURE 

(a1) {CASE free} 

if (My.FrontCell.Type = EMPTY) and (My.FrontCell.SignalsFree) then 

• My.Type := EMPTY // delete, because of moving 

else 

(b) {CASE not free} 

• My.Direction:= TurnRight/Left (My.Direction) // only turn R, L 

Rule for My.Type = EMPTY 

(a2) {CASE free} 

n = Number of creatures with direction to MY 

if (n=1) then 

• My.Type := CREATURE // create (move by copy) 

• My.Direction := TurnRight/Left (My.NeighborCreature.Direction) // new direction 

Coupled Action 

In case a1 and a2 where a creature can move, it changes its own type 

to EMPTY (case a1) and at the same time the empty cell changes its 

type to CREATURE (case a2). 

7 

In case b where the cell cannot move, it will turn only right or left.

Conflict Resolution 

cell in conflict 

(front cell) 



• More than one creature wants to move to the same cell 

• Result of conflict resolution in general 

– either no creature can move, or 

– one creature is selected to move 

• Implementations 

– 2-phase arbitration 

(1) check for conflict and select one creature 

(2) the selected creature will move 

– 1-phase arbitration (our solution, see next slide) 

8

1-Phase Arbitration 



request i 

grant i 

front 

cell 

feed-back 

logic 

• front cell contains a feed-back logic for the arbitration 

• evaluates the number Q of requests 

• if (Q>1) then grant = 0 else grant = 1 

• grant is computed within the current clock cycle 

9

Different Behaviors of a Creature 

• Modeled by a changdable Control Machine and a fixed Output-Machine 

• Control Machine implements State Algorithm 



grant signal from neighbor in front (front cell) 

m 

0 0 

0 1 

0 2 

0 3 

0 4 

0 5 

1 0 

1 1 

1 2 

1 3 

1 4 

1 5 

1 L 

2 L 

0 L 

4 R 

5 R 

3 R 

3 Lm 

1 Rm 

5 Lm 

0 Rm 

4 Lm 

2 Rm 

s' 

d 

s 

v 

r 

s control state 

r direction 

d action 

v(r,d) new direction 

m creature can move 

L/R turn left/R if (m=1) 

Lm/Rm turn left/R and move if (m=0) 

if d=1 then 

r:=r+1 (turn right) 

else 

r:=r-1 (turn left) 

10 

MEALY- 

Control-Machine 

MOORE- 

Output-Machine

Representation of a State Algorithm 

• state/output-table state/output-graph 

m 

6-state algorithm 



11 

0 0 

0 1 

0 2 

0 3 

0 4 

0 5 

1 0 

1 1 

1 2 

1 3 

1 4 

1 5 

1 L 

2 L 

0 L 

4 R 

5 R 

3 R 

3 Lm 

1 Rm 

5 Lm 

0 Rm 

4 Lm 

2 Rm 

s 

action 

d 

• string repesentation 

0 2 

blocked, m=0 

move, m=1 

1L2L0L4R5R3R-3Lm1Rm5Lm0Rm4Lm2Rm 

= 1L2L0L4R5R3R-3L1R5L0R4L2R simplified 

L 

1 

3 5 

4 

Rm 

Rm Lm Rm Lm 

R 

R 

Lm 

R 

L

Number of State Algorithms 



• M = Number of all control algorithms (Mealy state 

machines), represented by a state table 

– #s: number of states 

– #x: number of input values (blocked or free e.g. 2) 

– #y: number of actions (turn left or right e.g. 2) 

• M = (#s #y) (#s #x) (2#s) (2#s) 

12

Classes of State Algorithms 



• P (without prefix) 

– Each state reachable from initial state 0 is able to return to the initial state. 

• N (normalized) 

– Equivalent algorithms which only differ in their state encodings are represented 

in a unique form. Only the representatives are in this class. 

• R (irreducible) 

– Algorithms in this class are true n-state algorithms which cannot be reduced to 

algorithms with less than n states. 

• V (all reachable) 

– All states can be reached from the initial state 0 for each algorithm in this class. 

• Relevant Algorithms 

– Q = P ∩ N ∩ V ∩ R 

13



14 

N 

3 states, 2 inputs, 2 outputs 

R 

12 36 99 49 

1124 3228 8124 1348 

1124 3228 8124 1348 

1260 2148 10653 4751 

P 

all algorithms M = 46 656 

V

Relevant algorithms for #x = #y = 2 

states 

All algorithms 

Relevant 

#s 

M 

Algorithms 



2 

3 

4 

5 

6 

4 4 = 256 

6 6 = 46 656 

8 8 = 16 777 216 

10 10 = 10 * 10 9 

12 12 = 8.92 * 10 12 

108 

[42.2 %] 

8 124 

[17.4 %] 

798 384 

[4.8 %] 

98 869 740 

[1 %] 

14 762 149 668 

[0.2 %] 

15

Procedure to find the best algorithms 

generate next algorithm (state table) 



∀ algorithms 

discard irrelevant algorithms 

∀ initial configurations: simulate and evaluate behavior 

discard alg. with poor performance 

store set of candidates with good performance 

detailed evalutation and statistics 

identify best algorithms 

test for robustness with further initital configurations 

FPGA Hardware 

Software 

16 

testing the performance with more than one creature

5 Initial Configurations: The First Test Set 

1 2 3 4 5 



17

Further 21 initial configurations for robustness test 



• Robustness : creature behaves also well under other 

conditions (initial cofigurations) 

Samples 

Config. 16 

Config. 15 

18

Performance of the best algorithms 

successful 

(100 % crossed) 

unsuccessful 



AL 

GO 

RIT 

HM 

A6 

B6 

C6 

D6 

E6 

F6 

1 

2 

3 

4 

5 

26 Configuration (Test set) 

6 7 8 9 1 1 1 1 1 1 1 1 1 1 2 2 

0 1 2 3 4 5 6 7 8 9 0 1 

2 

2 

2 

3 

2 

4 

2 

5 

2 

6 

Success 

23 

24 

23 

22 

24 

24 

Crossing 

Perform. 

99.07 

99.92 

99.29 

98.63 

99.12 

97.94 

Speed 

.136 

.252 

.259 

.251 

.223 

.238 

G6 

24 

99.92 

.260 

x 

H6 

21 

98.91 

.169 

I6 

21 

98.91 

.193 

J6 

16 

78.55 

.299 

K6 

23 

99.87 

.268 

19

The two best 6-State-Algorithms 

Rm 

Lm 



Rm 

1 

L 

L 

R interchanged with L 

1 

R 

R 

0 2 

0 2 

L 

R 

Rm 

Lm 

Lm 

Lm 

Lm 

Rm 

R 

L 

3 5 

3 5 

R 

4 

R 

L 

4 

L 

Rm 

Lm 

Rm 

20 

Algorithm G6 

behaves slighty better for the 26 configs. 

Algorithm B6

Speed of 6-State Algorithms for Config. 15 



J6 

B6 

G6 

D6 

B6: 2599 

G6: 2592 

Speed depends on algorithm 

and configuration 

Alg. 

A6 

B6 

C6 

D6 

E6 

F6 

G6 

1/speed 

5.9 

4.9 

5.9 

3.7 

6.4 

6.4 

4.9 

speed 

.168 

.205 

.170 

.271 

.157 

.156 

.205 

H6 

6.9 

.144 

I6 

6.1 

.162 

J6 

(2.3) 

(.433) 

J6 fast at the beginning 

but is not succesful 

21



22 

Config. 7, Alg. B6 

first cyclic path 

2 cells can not be reached 

second cyclic path

Several Creatures 

Initial configurations used to evaluate the behavior of several creatures 



1 2 4 8 16 

23



24 

Generations and Speed using n Creatures 

Generations to visit all empty cells with n creatures 

33 times faster 

(with 16 creatures)

Generations and Speed using n Creatures 

Definition: 

Speed = empty cells / generations 



Speed per creature = Speed / n 

Four creatures with algorithm A6 work relatively faster 

than one creature with the best algorithm J6 

(0.52 0.53) 

25

Conclusion 



• Creature's Exploration 

Problem was modeled by CA 

• Conflicts are solved using a 

new 1-phase arbitration logic 

• Behavior of a creature is 

modeled by a Mealy-Control- 

Machine and an Output- 

Machine 

• State Algorithms were ordered 

into classes. 

• Relevant algorithms 

– 6 states: 

14 762 149 668 [0,2 %] 

• Procedure to find best alg. 

1. hardware computes 

candidates 

2. software selects best alg. and 

tests for robustness 

3. software tests behavior of 

several creatures 

• Several creatures can perform 

the work more efficiently (in 

terms of speed per creature) 

than only one. 

26

Hardware Platform 



• FPGA Altera Cyclone 

EP1C20F324C7 

• Altera Quartus II Tools 

– VERILOG FPGA-Logic 

• USB Module 

– connected to a PC 

– for control and data 

communication 

• Computation in Hardware 

– algorithm search, simulation and 

evaluation (5 configs in parallel) 

– Hardware/Software Speed-Up: 

100 to 10 000 depending on 

FPGA 

27

State Algorithm - Ra.informatik.tu-darmstadt.de

Create successful ePaper yourself

Delete template?

Save as template?