POLITECNICO DI MILANO - DCSC
POLITECNICO DI MILANO - DCSC
POLITECNICO DI MILANO - DCSC
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>POLITECNICO</strong> <strong>DI</strong> <strong>MILANO</strong><br />
Facoltà di Ingegneria Industriale<br />
Corso di Laurea Specialistica in Ingegneria Spaziale<br />
<strong>DI</strong>STRIBUTED AND ROBUST CONTROL FOR SPACE<br />
MULTI AGENT SYSTEMS<br />
Relatore: Prof. Michèle R. LAVAGNA<br />
Co-relatore: Prof. Paolo MAGGIORE<br />
Aprile 2008<br />
Anno Accademico 2006/07<br />
Tesi di Laurea di:<br />
Andrea SIMONETTO<br />
Matr. 680744
Cite as<br />
@MASTERSTHESIS{Simonetto_ms_08,<br />
AUTHOR = {Simonetto, Andrea},<br />
c○2008 Andrea Simonetto<br />
Submitted March 26th, 2008<br />
TITLE = {{Distributed And Robust Control For Space Multi<br />
Agent Systems}},<br />
}<br />
SCHOOL = {Department of Aerospace Engineering, Politecnico di Milano},<br />
YEAR = {2008},<br />
month = {April},
M.C. Escher (1898 - 1972), Moebius Strip II (Red Ants) 1963<br />
To who will have the patience to understand my words<br />
i
ABSTRACT<br />
Nowadays Multi Agent Systems are increasingly being studied in space applications, mainly<br />
for their reliability, robustness, fault tolerance and cost effectiveness. Although the research<br />
in this area started in Computer Science field at least twenty years ago, the challenges and<br />
open issues are still several, since the application scenario is completely different. This thesis<br />
main aim is, first, to understand the Space Multi Agent System state of the art, starting from<br />
the proposed mission by NASA and ESA, second, to study deeply two applicative scenarios,<br />
namely a formation flying mission and an asteroid exploration one. Different algorithms and<br />
formal techniques have been analyzed in order to built a fully distributed and robust controller<br />
for the system. The overall agent architecture embodies both a high level control and a low<br />
level one; the former uses a suitable extension of a potential field formulation, called Artificial<br />
Physics, which let the basic approach capable of dealing with multiple tasks and skills; on<br />
the other hand, the latter is a trajectory control and a communication network assurance<br />
one. The first is a non linear Lyapunov control, formulated in a H∞ framework, extending<br />
a previous approach to let it robust and reliable to perturbations and model uncertainties.<br />
The second is a dynamic potential field/token-based algorithm, which is a modification of<br />
standard potential field techniques and it increases the overall performances in terms of global<br />
connectivity and efficiency. Several simulation scenarios have been tested, both in formation<br />
flying contests, like Prisma ESA mission, and in asteroid belt environment, showing good<br />
results for reliability, robustness and wide application areas.<br />
Keywords: Multi Agent Systems, Distributed Control, Robust Control, Artificial Physics,<br />
Non Linear Lyapunov Control, Sensor Networks.<br />
iii
ACKNOWLEDGEMENTS<br />
A Master Thesis means a lot of mental effort, both to be able to focus on a particular and<br />
very specific topic for long time, and not to feel either disappointed or upset if something<br />
does not work as it should. Since the latter occurred very often I would like to thank all the<br />
people who have helped me during these long eight months, or maybe more.<br />
First of all, my advisor Michèle, who is a very busy person, but she made me understand<br />
how to work by myself and how to develop a scientific research. She was always kind and<br />
she gave me a lot of advices, supporting me in the hardest moments. Moreover she made the<br />
amazing experience to work abroad possible for me, thus I could not thank her a lot. Then,<br />
of course, the other professors who helped me developing my thesis, Paul and Katia, both<br />
US researchers. Paul was great, he kept pushing me to do better than my best and finally I<br />
managed to do something he liked. He is the person I would like to have as a friend and he<br />
is also a great researcher. Thanks Paul. Katia has a lot of experience, thus her advices were<br />
deep and very helpful for me to understand better how to formulate my problem. I remember<br />
the first meeting with them, they were so suspicious about me, but by the end, they were so<br />
glad to have worked with me. It is something to be proud of. Thanks.<br />
I want to thank my parents, who supported me in this adventure abroad, I imagine they<br />
missed me a lot, or at least I hope, thus thanks dad and mommy, I missed you too. I am<br />
very proud of them, they are very open minded people, I cannot ask for better ones.<br />
Then, of course, my friends. I am very pleased to have a lot of names on this list. And you,<br />
the reader, please don’t miss a name! First my friends from Carnegie Mellon and US: Kevin,<br />
the funniest German I have ever met, I am looking forward for other amazing pool matches;<br />
Giuseppe, very awesome times together, Philadelphia, the swimming pool, the canoeing, I<br />
hope to work with you someday; Joe, Pras, Robin, Sean, Jack I really like those guys. And<br />
then, more: Joe, Becca, Marie, Paul, Eric, Holly, Mike, Garret, Alex Sasha, I had really great<br />
time with you. Thanks for the birthday party and all the attentions you gave me, the moka,<br />
all the ride back to my place, all the fabulous dishes, the skiing stuff, Thanksgiving party<br />
and the superbowl party and even more. Finally, Andrea, a good guy who pretends to get<br />
his PhD next year, isn’t he fun?<br />
I want to mention all my great friends of ASP, another amazing thing I had the pleasure<br />
to do. Thanks Elena, who is very smart and cute, I really like her, even if I cannot remember<br />
v
vi<br />
the exact date of her birthday, sorry girl. Francesca, very long mails, very nice talks, she<br />
is very cool doing very amazing math stuff. I really like her as well. Then in some order,<br />
my forever roomy Lo, Gianma the easy-going guy, Francesco and Franz the normative boys,<br />
Martino Daniele and Marco from Chicago, Davide and Pietro who know how to make a<br />
wash machine work, Alessandro the physician, Stefania and Umberto the mathematicians,<br />
Giacomo, Giorgio, Andrea the open-door man, who met me a lot of time ago, in a certain<br />
competition. I have to tell you the truth, I let you win that time.<br />
How could I forget my friends all over the World? Like Paul in France, Thanos, Fani and<br />
Kostas in Greece, Tamara in Croatia, and even more. Roberta, Goran, Antonio, Ines, Bene,<br />
Nico and other funny guys and ladies. We had great times wherever we were.<br />
What about my Uni friends? Very pleased to have met them all. Mauro now at Toulouse,<br />
very easy man, I remember the days just at the end of July, nobody at school, only me and<br />
you. Who on Earth made us do that? Maffez, very smart man, I really like him, and he is<br />
very impressive in skiing, somehow. Paolo, very very nice, sorry for that girl man. Giorgio<br />
and Ricki and some testosterone moments. Alfu, the most particular one, he really thinks<br />
Armani could have a chance to win something. Sorry, but anyway I like you. Bart, now<br />
in German, who was a true surprise for me, you are so smart! Great. Castel, in ESA, the<br />
most easy-going person I have ever met, always drinking beer, always optimizing something,<br />
amazing. Luca, very deep person, now at Atlanta, the man who helped me with my mind<br />
most in those days at Pitt. I still cannot understand how a tiramisù could cost thirty bucks!<br />
Great NYC. And then all the other: Davide, thanks a lot for the – unfortunately not used<br />
– asteroid model; Fede, thanks for your support, your good words, your friendship; Monica,<br />
in France, very cool to work with you; Dano, very cool discussions, you are so funny; Fabri,<br />
Francesco, Nazi, Michele, Matteo, Marco, Alessandro, Roberto, Gabriele, Marcello, Luca,<br />
Giulio, Elisa, Fabio, Daniele, Luca, Marco, Michele, Alberto, Davide, Dario. I think there is<br />
somebody missing, but I hope they could forgive me.<br />
Then, of course my friend of adventures at university: Guido. I barely understand you,<br />
but I like you after all. I am very pleased to have met you, although I cannot agree on<br />
everything you say, and I have some difficulties to work with you. I am joking, of course.<br />
Thanks for your practical perspective on life and science, it served me a lot to understand<br />
better what was going on. Thanks for all the discussions, the ideas, the funny trips, the nice<br />
girls, thanks man, I hope you could do whatever you want.<br />
I am so pleased to have met you all, friends, you can make it as I had. I remember three<br />
things. The first, an economics book with a long long bibliography, the second, an Iranian<br />
who kept writing third order integrals, finally, some Fusion conference papers I had to read<br />
to understand the research at CMU in October. Well, I have a very long biblio, I could have<br />
had sixth order integrals, but I found a better solution and, by the end, I managed to submit<br />
my own paper to the Fusion conference. Everything is possible guys, just go out and do it.<br />
Finally, my closest friends. Vale, who helped me feeling at home also when I was overseas.<br />
Very funny moments together, the lake over all. Eleonora, from the coolest high school<br />
ever, thanks for the long discussions, your friendship and the basketball matches. You have
absolutely to come to my thesis party. Then, the three people I care most. My best friend,<br />
as a brother for me, Roberto, Bob, who is actually an architect. No one is perfect after all.<br />
Thanks for everything, really everything. Since your English is worse than mine, I put here<br />
just a ⋆, and we will discuss later on what I would have liked to write to thank you. Thanks<br />
thanks thanks a lot, your are a great friend, although you have a lot of defects. Then my<br />
sister, my true sister, Tiziana. I really love here, she is very cute, smart, carina e coccolosa.<br />
Thanks Tita, for everything, for just being you. I hope the thesis is intriguing enough for<br />
you, I really hope, because I do not want write it another time!<br />
Last on this list, You. I don’t know what is going to happen, but I want to thank You<br />
for all your mind, your Love, everything you gave me in such a short period of time. I will<br />
never forget how You made me feel. Thanks.<br />
Thanks to everybody,<br />
vii<br />
andrea =)
viii
CONTENTS<br />
ABSTRACT iii<br />
ACKNOWLEDGEMENTS v<br />
SUMMARY xiii<br />
1 INTRODUCTION 1<br />
1.1 Multi Agent System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1<br />
1.2 Proposed Missions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3<br />
1.2.1 Prisma and Swarm missions . . . . . . . . . . . . . . . . . . . . . . . . 3<br />
1.2.2 Terrestrial Planet Finder Interferometer . . . . . . . . . . . . . . . . . 5<br />
1.2.3 ANTS project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6<br />
1.2.4 APIES feasibility study . . . . . . . . . . . . . . . . . . . . . . . . . . 7<br />
2 FORMULATION OF THE PROBLEM 11<br />
2.1 Typical scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11<br />
2.1.1 Formation flying in Prisma Mission . . . . . . . . . . . . . . . . . . . . 12<br />
2.1.2 Main asteroid belt exploration . . . . . . . . . . . . . . . . . . . . . . 12<br />
2.2 Agent architecture and assumptions . . . . . . . . . . . . . . . . . . . . . . . 15<br />
2.3 Multi agent techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16<br />
2.3.1 Space multi agent system . . . . . . . . . . . . . . . . . . . . . . . . . 18<br />
2.3.2 Formal methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18<br />
2.3.3 Artificial Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20<br />
2.4 Work’s framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21<br />
3 THE DECISIONAL LEVEL 23<br />
3.1 Artificial Physics Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 23<br />
3.1.1 AP state of the art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26<br />
3.2 Extension of AP formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . 27<br />
3.3 Formal methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28<br />
3.4 Information propagation and knowledge bounded agents . . . . . . . . . . . . 28<br />
ix
x CONTENTS<br />
3.4.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28<br />
3.4.2 Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29<br />
3.5 Distributed scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29<br />
4 THE PHYSICAL PART 31<br />
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31<br />
4.2 Possible solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32<br />
4.2.1 Potential field approach . . . . . . . . . . . . . . . . . . . . . . . . . . 32<br />
4.2.2 SDRE approach for non linear systems . . . . . . . . . . . . . . . . . . 32<br />
4.2.3 Non linear Lyapunov control . . . . . . . . . . . . . . . . . . . . . . . 33<br />
4.3 Long period dynamic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36<br />
4.3.1 Robust non linear Lyapunov control . . . . . . . . . . . . . . . . . . . 36<br />
4.3.2 HJI equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37<br />
4.3.3 R as a function of the state . . . . . . . . . . . . . . . . . . . . . . . . 38<br />
4.4 Short period dynamic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39<br />
4.5 Final Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39<br />
5 THE COMMUNICATION PART 41<br />
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41<br />
5.2 Artificial Physics and Token based algorithm . . . . . . . . . . . . . . . . . . 42<br />
5.3 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43<br />
5.4 Positioning Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45<br />
5.4.1 Standard algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46<br />
5.4.2 Dynamic Potential Fields . . . . . . . . . . . . . . . . . . . . . . . . . 46<br />
5.5 Potential Field and Motion Control . . . . . . . . . . . . . . . . . . . . . . . . 49<br />
6 PERTURBATION MODELS 51<br />
6.1 Formation Flying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51<br />
6.1.1 Atmospheric drag effect . . . . . . . . . . . . . . . . . . . . . . . . . . 51<br />
6.1.2 Gravitational effects, J2 and J22 . . . . . . . . . . . . . . . . . . . . . 52<br />
6.2 Asteroid belt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54<br />
7 RESULTS 57<br />
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57<br />
7.1.1 Scalability proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58<br />
7.2 Goal Manager example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58<br />
7.3 Formation flying scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60<br />
7.3.1 Unperturbed and perturbed results . . . . . . . . . . . . . . . . . . . . 60<br />
7.3.2 Montecarlo analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61<br />
7.4 Communication network deployment . . . . . . . . . . . . . . . . . . . . . . . 66<br />
7.4.1 2D Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66<br />
7.4.2 3D Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
CONTENTS xi<br />
7.5 Asteroid belt scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70<br />
7.5.1 The physical part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70<br />
7.5.2 The communication part . . . . . . . . . . . . . . . . . . . . . . . . . . 73<br />
8 FINAL REMARKS 77<br />
8.1 Thesis final remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77<br />
8.2 Future developments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77<br />
8.2.1 Decisional Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77<br />
8.2.2 Physical part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78<br />
8.2.3 Communication part . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78<br />
8.2.4 Missions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78<br />
REFERENCES 79<br />
A SISTEMI MULTI AGENTE PER APPLICAZIONI SPAZIALI i<br />
A.1 Introduzione . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i<br />
A.1.1 Principali Contributi . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii<br />
A.2 Formulazione del Problema . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii<br />
A.3 Controllo di Alto Livello . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii<br />
A.4 Controllo Robusto della Traiettoria . . . . . . . . . . . . . . . . . . . . . . . . iii<br />
A.5 Architettura di Comunicazione . . . . . . . . . . . . . . . . . . . . . . . . . . iii<br />
A.6 Modelli per le Perturbazioni . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv<br />
A.7 Risultati . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv<br />
A.7.1 Controllo di alto livello . . . . . . . . . . . . . . . . . . . . . . . . . . iv<br />
A.7.2 Controllo della traiettoria . . . . . . . . . . . . . . . . . . . . . . . . . iv<br />
A.7.3 Dispiegamento della rete di comunicazione . . . . . . . . . . . . . . . . vii<br />
A.7.4 Cintura degli asteroidi . . . . . . . . . . . . . . . . . . . . . . . . . . . vii<br />
A.8 Sviluppi Futuri . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
xii CONTENTS
SUMMARY<br />
This work deals with multi agent system and, since they are a quite new research topic in<br />
space area, the project is in first instance a collection of ideas, then a detailed analysis of two<br />
typical scenarios: formation flying and asteroid belt exploration missions.<br />
The focus is on understanding how these kind of systems could be applied in space envi-<br />
ronments, in particular how to assure reliability, verifiability and thus robustness. First the<br />
problem is outlined with some proposed mission in this direction, e.g. NASA ANTS mission<br />
to the asteroid belt, second the agent control is shown. Both the high level control, i.e. a<br />
distributed goal manager, and a low level control, an H∞ non linear control for the physical<br />
part, and a dynamical potential field-token based control for the communication architec-<br />
ture, are developed and tested, showing high reliability, wide area of applicability and good<br />
performances.<br />
Chapter division<br />
Chapter 1. Here the framework of the work is presented with some proposed mission in<br />
the multi agent systems contest, both from ESA and NASA.<br />
Chapter 2. The problem is formulated, outlining both the high level control and the low<br />
level control for each agent. The chosen approach for the goal manager is an Artificial<br />
Physics/ Potential Fields approach.<br />
Chapter 3. The high level control is developed extending the Artificial Physics formulation<br />
to more complex scenarios with multiple goals and capabilities.<br />
Chapter 4. The low level control for the physical part is examined and derived. The final<br />
control is a long dynamic - short dynamic control. For the long dynamic one a H∞ non linear<br />
Lyapunov control is used, and its robustness is proved. Then for the short dynamic control<br />
two solutions are proposed.<br />
Chapter 5. The communication network deployment control is developed here, using a<br />
dynamical potential fields approach, in which the communication among the agents is dictated<br />
xiii
xiv CONTENTS<br />
by tokens, which are information packets.<br />
Chapter 6. The perturbation models are briefly discussed here.<br />
Chapter 7. Main results are shown here. First, for goal manager, second for the physical<br />
part, third for the communication network deployment and finally for the whole control.<br />
Main contributions<br />
The original contributions in this thesis are several:<br />
� The Potential Field - Artificial Approach is firstly extended to the multiple goals and<br />
skills case, using matrices algebra (Chapter 3).<br />
� The non linear Lyapunov control is applied to a perturbed system, the spacecraft tra-<br />
jectory control, extending it using an H∞ control. The necessary and sufficient con-<br />
ditions for stability is then derived. This application resulted in a paper submitted to<br />
AIAA/AAS Conference 2008 (Chapter 4).<br />
� The Potential Fields approach is integrated in a token - based algorithm to assure min-<br />
imal communication and high value of connectivity. Then this is tested in environment<br />
with obstacles, which has never been done before. This results in a paper submitted to<br />
IEEE Fusion Conference 2008 (Chapter 5).<br />
Acknowledgments<br />
This Master Thesis has been developed partly in Italy, in Milan, within the Politecnico<br />
di Milano, somehow even in collaboration with Politecnico di Torino, but mostly in US at<br />
Carnegie Mellon University. I would like to thank everyone who helped me in these three<br />
top-level universities and who made this possible.<br />
Milano, March 2008
CHAPTER<br />
1<br />
INTRODUCTION<br />
[...] to boldly go where no one has gone before.<br />
Gene Roddenberry (1921 - 1991)<br />
The chapter is completely dedicated to understand how the new concepts of Multi Agent<br />
Systems and Distributed Systems enter in the space research. First of all, why they are studied<br />
and their advantages are discussed, then an overview of the current proposed missions is<br />
presented.<br />
1.1 Multi Agent System<br />
Space science always leads Man to think to the most challenging ideas to try to catch<br />
the beauty of the cosmos. It is not new that many technological breakthroughs come from<br />
space research. Moreover, the multidisciplinarity, the fusion of different types of knowledge,<br />
the never similar problems are only some of its several amazing pros.<br />
Nowadays, new interpretations of what a space system actually is are appearing in the<br />
scene, in particular the concepts of distributed architecture, formation of satellites and swarm<br />
of agents. The advantages behind approaches of this kind are several, at least many as the<br />
challenges involved in. First of all, the relative simplicity of the single agent – spacecraft,<br />
robot or satellite – in terms of manufacturing, testing, electronics. The smaller the agent is,<br />
the simpler the making and, moreover, the cheaper. Second, having in mind an architecture<br />
of many agents, the reliability is improved and this is a key feature for a space mission to be<br />
founded. The biggest communication satellites have to be tested in many ways to be reliable<br />
enough to fly, moreover they are less fault tolerant as a multi spacecraft system may be. It<br />
is then worth to mention that, in a vision like this, the idea of series productions could be<br />
thought even in space research.<br />
1
2 CHAPTER 1. INTRODUCTION<br />
Distributed architectures could be a reasonable, maybe the only one, option when the<br />
missions to be performed are particularly risky, as the exploration of asteroid belt, or they<br />
involve formation flying concepts, as a large interferometric telescope.<br />
In Table 1.1 the differences between the two types of approaches are shown. As it seems<br />
evident, the distributed system key disadvantages, or challenges to handle, are related to the<br />
design of the system as a whole.<br />
Single agent systems Multi agent systems<br />
Design Complex Simple for the single agent<br />
Software Could have several functions<br />
Communications Important<br />
Basic functions, but complex global<br />
verification<br />
Important for the whole system and<br />
critical between the agents<br />
Manufacturing Complex Simpler and possible in series<br />
Tests Several and Complex<br />
Reliability<br />
Fault tolerance<br />
Cost High<br />
Reasonable for the several test and<br />
internal redundancy<br />
Reasonable for internal redundancy,<br />
but critical<br />
Quite impossible for the system as a<br />
whole but simpler for single agents<br />
High for the intrinsic redundancy<br />
High for the intrinsic redundancy<br />
Lower and dependent on the num-<br />
ber of agents<br />
Table 1.1: Comparison between single agent and multi agent approaches<br />
The fact that the single agent itself could be very basic and simple hides the global complexity<br />
of the complete system. The communication architecture and the control design has to be<br />
faced, in particular the control has to be as simpler as possible, but it has to include some<br />
functions to handle the system as a whole. Moreover, the idea that the total architecture can<br />
not be tested on the ground is quite critical.<br />
The present work deals with the agent control to assure both robust task execution and<br />
communication network building.<br />
Space system design involves many research areas; in fact, the concept of distributed<br />
systems is not new in the literature, in particular in computer science, where problems like<br />
those are called usually M ulti Agent Systems, or MAS, and they have been studied since<br />
1980 [Woo02], [Syc98]. There are several novelties in trying to apply the computer science<br />
methods to space science: first of all the scenario, which is completely different. Typically,<br />
the MAS are robots which have to perform some scheduled tasks, as explore a region, rescue
1.2. PROPOSED MISSIONS 3<br />
people, play soccer. Thus the scenario is basically bi-dimensional and the relative distances<br />
are small in comparison to the involved dynamic. Furthermore the system is composed usually<br />
by relatively few agents, less than 100, and the communication is thought as complete and<br />
faster than the decisional process of the robots. These features are not those a space engineer<br />
could expect from a Space MAS, SMAS. The scenario is three dimensional, the distance<br />
could be large respect to the dynamic, thus the environment may be addressed as sparse, the<br />
agents would be more than 1000 and, in the end, the communication could not be complete,<br />
see Table 1.2.<br />
Dimensions<br />
Distances in compar-<br />
ison to dynamics<br />
MAS SMAS<br />
Typically 2, could be 3 in UAVs sce-<br />
narios<br />
Small<br />
3<br />
Large<br />
Number of agents 10 - 100 10 - 1000<br />
Communication Often complete Often incomplete<br />
Environment Dense Typically sparse<br />
Table 1.2: Comparison between MAS and SMAS<br />
Thus, the challenges are several: using the methods of computer science, try to model a<br />
SMAS and, eventually, change the tools in order to fit them to the different scenario. It is<br />
also possible, and highly recommended in some cases, to develop completely new algorithms.<br />
1.2 Proposed Missions<br />
This is the framework in which the most important space agencies are in. In particular,<br />
in the following sections an overview of the most promising missions of both NASA and ESA<br />
is provided. This list is not complete but it could give a good insight on what is going on in<br />
space research.<br />
1.2.1 Prisma and Swarm missions<br />
Prisma Mission<br />
Prisma mission [1] provides a technology demonstration mission for the in-flight validation<br />
of sensor technologies and guidance/navigation strategies for spacecraft formation flying and<br />
rendezvous. Prisma is originating from an initiative of the Swedish National Space Board<br />
(SNSB) and the Swedish Space Corporation (SSC) and provides a precursor mission for<br />
critical technologies related to advanced formation flying and In-Orbit-Servicing. Prisma<br />
mission launch window is fixed for the beginning of 2009.
4 CHAPTER 1. INTRODUCTION<br />
Figure 1.1: The Prisma mission<br />
The Prisma test bed comprises the fully maneuverable micro-satellite Main as well as<br />
the smaller passive sub-satellite Target. The two spacecraft will be injected into a Sun-<br />
synchronous dusk-dawn orbit at an altitude of 700 km.<br />
The mission objectives of Prisma may be divided into the validation of sensor and actu-<br />
ator technology related to formation flying as well as the demonstration of experiments for<br />
formation flying and rendezvous.<br />
ESA’s magnetic field mission Swarm<br />
The objective of the Swarm mission [2], scheduled for 2010, is to provide the best ever survey<br />
of the geomagnetic field and its temporal evolution, and gain new insights into improving the<br />
knowledge of the Earth’s interior and climate.<br />
The Swarm concept consists of a constellation of three satellites in three different polar<br />
orbits between 400 and 550 km altitude. High-precision and high-resolution measurements<br />
of the strength and direction of the magnetic field will be provided by each satellite. In<br />
combination, they will provide the necessary observations that are required to model various<br />
sources of the geomagnetic field. GPS receivers, an accelerometer and an electric field instru-<br />
ment will provide supplementary information for studying the interaction of the magnetic<br />
field with other physical quantities describing the Earth system - for example, Swarm could<br />
provide independent data on ocean circulation.<br />
The multi-satellite Swarm mission will be able to take full advantage of a new generation<br />
of magnetometers enabling measurements to be taken over different regions of the Earth<br />
simultaneously. Swarm will also provide monitoring of the time-variability aspects of the<br />
geomagnetic field, this is a great improvement on the current method of extrapolation based<br />
on statistics and ground observations. The geomagnetic field models resulting from the<br />
Swarm mission will further our understanding of atmospheric processes related to climate<br />
and weather and will also have practical applications in many different areas, such as space
1.2. PROPOSED MISSIONS 5<br />
weather and radiation hazards.<br />
Figure 1.2: The Swarm mission concept<br />
1.2.2 Terrestrial Planet Finder Interferometer<br />
The Terrestrial Planet Finder Interferometer (TPF-I) mission, by NASA, will search for<br />
habitable worlds around nearby stars and look for indicators of the presence of life. Working<br />
with infrared wavelengths, TPF-I complements the search made by the Terrestrial Planet<br />
Finder Coronagraph (TPF-C) in visible wavelengths. This combination provides the strongest<br />
possible confirmation of the presence of indicators of habitable worlds, Figure 1.3.<br />
TPF-I is in the pre-formulation phase of its development. The observatory mission concept<br />
includes five formation flying spacecraft: four 4-meter-class mid-infrared telescopes and one<br />
combiner spacecraft to which the light from the four telescopes is relayed to be combined and<br />
detected. The observatory will be deployed beyond the Moon’s orbit for a mission life of 5<br />
to 10 years.<br />
New technologies are being developed to allow spectroscopic measurements of light from<br />
extrasolar planets, including:<br />
1. formation flying telescopes to work together as one extended observatory, providing<br />
unprecedented angular detail and sensitivity that no telescope on the ground could<br />
ever achieve;<br />
2. starlight-suppression technology so that light from a planet’s star will be dimmed by a<br />
factor of a million, making the planet’s light visible;
6 CHAPTER 1. INTRODUCTION<br />
3. new cryogenic coolers, making it possible for a new generation of detectors to find<br />
Earth-like planets.<br />
In this context it is important to note that the formation flying concept is one of the main<br />
challenge of the project, [LLJB07], [3].<br />
1.2.3 ANTS project<br />
Figure 1.3: The TPF interferometer<br />
The Autonomous Nano-Technology Swarm (ANTS) concept mission, by NASA, will involve<br />
the launch of a swarm of autonomous pico-class (approximately 1kg) spacecraft that will<br />
explore the asteroid belt for asteroids with certain scientific characteristics. Figure 1.4 gives<br />
an overview of the ANTS mission. In this mission, a transport ship, launched from Earth,<br />
will travel to a point in space where net gravitational forces on small objects (such as pico-<br />
class spacecraft) are negligible (a Lagrangian point). From this point, 1000 spacecraft, that<br />
have been manufactured en route from Earth, will be launched into the asteroid belt. This<br />
environment presents a large risk of destruction for large (traditional) spacecraft. Even with<br />
pico-class spacecraft, 60 to 70 percent of them are expected to be lost. Because of their small<br />
size, each spacecraft will carry just one specialized instrument for collecting a specific type<br />
of data from asteroids in the belt.<br />
To implement this mission, a heuristic approach is being considered, which provides for<br />
a social structure to the spacecraft that uses a hierarchical behavior analogous to colonies or<br />
swarms of insects, with some spacecraft directing others. Artificial intelligence technologies<br />
such as genetic algorithms, neural nets, fuzzy logic, and on-board planners are being investi-<br />
gated to assist the mission to maintain a high level of autonomy. Crucial to the mission will<br />
be the ability to modify its operations autonomously, to reflect the changing nature of the<br />
mission and the distance and low bandwidth communications back to Earth. Approximately<br />
80 percent of the spacecraft will be workers that will carry the specialized instruments (e.g.,<br />
a magnetometer, x-ray, gamma-ray, visible/IR, neutral mass spectrometer) and will obtain<br />
specific types of data. Some will be coordinators (called rulers or leaders) that have rules that
1.2. PROPOSED MISSIONS 7<br />
decided the types of asteroids and data the mission is interested in and that will coordinate<br />
the efforts of the workers. The third type of spacecraft are messengers that will coordinate<br />
communication between rulers and workers, and communications with the mission control<br />
center on Earth.<br />
Figure 1.4: An overview of ANTS mission<br />
The swarm will form sub-swarms, each under the control of a ruler, which contains models<br />
of the types of science that are to be pursued. The ruler will coordinate workers, each of which<br />
uses its individual instrument to collect data on specific asteroids and feeds this information<br />
back to the ruler, who will determine which asteroids are worth examining further. If the<br />
data matches the profile of a type of asteroid that is of interest, an imaging spacecraft will be<br />
sent to the asteroid to ascertain the exact location and to create a rough model to be used<br />
by other spacecrafts for manœuvering around the asteroid. Other teams of spacecrafts will<br />
then coordinate to finish mapping the asteroid to form a complete model, [RHRT06], [4].<br />
1.2.4 APIES feasibility study<br />
APIES (Asteroid Population Investigation & Exploration Swarm) is a mission developed by<br />
EADS Astrium in response to an European Space Agency (ESA) Call for Ideas for swarm<br />
missions, based on the utilization of a large number of spacecrafts working cooperatively<br />
to achieve the mission objectives. APIES is intended to be the first interplanetary swarm
8 CHAPTER 1. INTRODUCTION<br />
mission, designed to explore the asteroid main belt. This is one the least known parts of the<br />
Solar System, yet holding vital information about its evolution and planet formation. APIES<br />
aims to characterize a statistically significant sample of asteroids, exploring the main belt in<br />
great detail, measuring mass and density and imaging over 100 of these objects, at a stroke<br />
more than doubling the number of Solar System bodies visited by man-made spacecraft.<br />
Using the latest advances in system miniaturization, propulsion, onboard autonomy and<br />
communications, the APIES mission can achieve these ambitious goals within the framework<br />
of a standard ESA mission. APIES has completed a Mission Feasibility Study as part of<br />
the General Studies Programme (GSP) of ESA, whose purpose is to evaluate novel missions,<br />
concepts, methods, and to identify their research and development needs beyond currently<br />
running programmes.<br />
Figure 1.5: The HIVE carrier spacecraft with the BEEs<br />
In the baseline concept, the target orbit for the APIES swarm is based on a HIVE helio-<br />
centric circular orbit at 2.6 AU. This orbit selection is the result of a trade-off between the<br />
need of achieving a high rate of asteroid flybys (hence targeting a high density region of the<br />
asteroid belt) and that of adequately sampling the diversity of the asteroid population (and<br />
so targeting a Main Belt zone where the population is mixed, with representatives of most<br />
of the known asteroid spectral classes). To achieve the final operational orbit, it is envisaged<br />
that the APIES swarm will be transported by the HIVE carrier spacecraft, with the BEEs,<br />
the spacecrafts, deployed only after reaching the asteroid belt, figure 1.5. APIES is designed<br />
for a Soyuz/Fregat launch, capable of injecting a mass of up to 1420 kg into a Mars flyby<br />
trajectory. The HIVE, still carrying the BEEs, will take advantage of a Mars gravity assist<br />
and then use its own Solar Electric Propulsion (SEP) system to reach its 2.6 AU final circu-<br />
lar heliocentric orbit. It has been estimated that with the Soyuz/Fregat launcher and Mars<br />
gravity assist, an SEP system can deliver about 850 kg of payload (total ad-up mass for the<br />
BEEs, which are thought to be less than 50 kg each) to a circular orbit at 2.6 AU within a
1.2. PROPOSED MISSIONS 9<br />
3-year transfer time. An additional 3-4 years may then be needed for the deployment of the<br />
BEEs swarm to its nominal operational formation.<br />
After reaching the asteroid belt, the BEEs will separate from the HIVE and create a<br />
swarm ’cloud’ centered on the HIVE, [D’A04], [2].<br />
ANTS APIES<br />
Launch date 2025 −<br />
Objective Asteroid belt Asteroid belt<br />
Spacecraft mass 1 kg 50 kg<br />
Spacecraft number 1000 19<br />
Table 1.3: Comparison between ANTS and APIES missions
10 CHAPTER 1. INTRODUCTION
CHAPTER<br />
2<br />
FORMULATION OF THE PROBLEM<br />
It is possible to make things of great complexity out of things that are very simple.<br />
There is no conservation of simplicity.<br />
Stephen Wolfram (1959 - )<br />
In this chapter the formulation of the problem is described. First of all, the chosen sce-<br />
narios for SMAS are outlined, then connected critical issues are presented. To try to make<br />
first design hypotheses, several MAS techniques are analyzed in details, in particular in the<br />
framework of formal methods. In the end, the work is outlined and described in its parts.<br />
2.1 Typical scenarios<br />
As stated, the proposed missions are basically of two type:<br />
1. formation flying systems;<br />
2. exploration missions.<br />
These scenarios have characteristic features which have to be understood before a rea-<br />
sonable design could start. This is typical: given a problem, the solution could be sought.<br />
In space research, however, it is not so obvious. Often for cost reasons a good design is not<br />
the one perfect for one and only one mission, but it is the one which could be used for many<br />
different aims. It is crucial, in this view, to try to develop a solution which is general enough<br />
to be used for a family of problems.<br />
Another very peculiar issue in space research is the system scalability. As it has been<br />
shown, the future SMAS are going to be very large in comparison with MAS. Thus the<br />
scalability of the algorithms has to be proved.<br />
11
12 CHAPTER 2. FORMULATION OF THE PROBLEM<br />
To analyze the algorithms which have to be developed, two missions have been selected,<br />
one per type, namely<br />
1. Prisma mission;<br />
2. main asteroid belt exploration.<br />
2.1.1 Formation flying in Prisma Mission<br />
The mission objectives of Prisma may be divided into the validation of sensor and actuator<br />
technologies related to formation flying and the demonstration of experiments for formation<br />
flying and rendezvous. It will support and enable the demonstration of autonomous space-<br />
craft formation flying, homing, and rendezvous scenarios, as well as close-range proximity<br />
operations. The mission schedule foresees a launch of the two spacecraft in 2009. Both Main<br />
and Target will be injected by a Dnepr launcher into a sun-synchronous orbit at 700-km<br />
altitude and 98.2 deg inclination. A dusk-dawn orbit with a 6 or 18 h nominal local time at<br />
the ascending node (LTAN) is targeted. Following a separation from the launcher, the two<br />
spacecraft will stay in a clamped configuration for initial system checkout and preliminary<br />
verification. Once the spacecraft are separated from each other, various experiment sets for<br />
formation flying and in-orbit servicing will be conducted within a minimum targeted mission<br />
lifetime of eight months. Spacecraft operations will be performed remotely from Solna, near<br />
Stockholm, making use of the European Space and Sounding Rocket Range (Esrange) ground<br />
station in northern Sweden. The Sband ground-space link to Main supports commanding<br />
with a bit rate of 4 kbps and telemetry with up to 1 Mbps. In contrast, communication with<br />
the Target spacecraft is only provided through Main acting as a relay and making use of<br />
a Main-Target intersatellite link (ISL) in the ultrahigh-frequency (UHF) band with a data<br />
rate of 19.2 kbps. The Main spacecraft has a wet mass of 150 kg. In contrast to the highly<br />
maneuverable Main spacecraft, Target is a passive and much simpler spacecraft, with a mass<br />
of 40 kg [GM07], [PG05], [DM06]. Hence the mission key features can be summarized as<br />
follows<br />
� 2 s/cs;<br />
� communication architecture: sensor → relay → Earth;<br />
� Earth environment/perturbations.<br />
2.1.2 Main asteroid belt exploration<br />
ANTS proposed spacecraft<br />
The proposed spacecraft for ANTS mission has, [4]<br />
� power: 100 mW battery;<br />
� material: 1 kg, 100 m 2 /kg;
2.1. TYPICAL SCENARIOS 13<br />
� locomotion: solar sail.<br />
Sail achieves dynamic attitude control through capability for dynamic change in its mor-<br />
phology, thus changes the effective area and distribution of solar reflectivity to change its<br />
acceleration and momentum vectors to achieve required orbit and orientation.<br />
Asteroid belt<br />
The asteroid belt is the region of the Solar System located roughly between the orbits of<br />
the planets Mars and Jupiter. It is occupied by numerous irregularly shaped bodies called<br />
asteroids or minor planets. The asteroid belt region is also termed the main belt to distinguish<br />
it from other concentrations of minor planets within the Solar System, such as the Kuiper<br />
belt and scattered disk.<br />
Figure 2.1: Main asteroid Belt<br />
More than half the mass within the main belt is contained in the four largest objects:<br />
Ceres, 4 Vesta, 2 Pallas, and 10 Hygiea. All of these have mean diameters of more than
14 CHAPTER 2. FORMULATION OF THE PROBLEM<br />
400 km, while Ceres, the main belt’s only dwarf planet, is about 950 km in diameter. The<br />
remaining bodies range down to the size of a dust particle.<br />
The asteroid belt formed from the primordial solar nebula as a group of planetesimals,<br />
the smaller precursors of the planets. Between Mars and Jupiter, however, gravitational<br />
perturbations from the giant planet imbued the planetesimals with too much orbital energy<br />
for them to accrete into a planet. Collisions became too violent, and instead of sticking<br />
together, the planetesimals shattered. As a result, most of the main belt’s mass has been<br />
lost since the formation of the Solar System. Some fragments can eventually find their way<br />
into the inner Solar System, leading to meteorite impacts with the inner planets. Asteroid<br />
orbits continue to be appreciably perturbed whenever their period of revolution about the<br />
Sun forms an orbital resonance with Jupiter. At these orbital distances, a Kirkwood gap<br />
occurs as they are swept into other orbits [1].<br />
Even if perturbed, main asteroid orbital parameters are known with a reasonable uncer-<br />
tainty; for example Ceres semi-axis is known with 10 −9 1 − σ relative error. Nonetheless,<br />
since mass measurement is more complex, the physical parameters have significatively more<br />
uncertainty, also in the order of 50%, thus this is the most relevant problem in the perturba-<br />
tion model determination. In Table 2.1, the first ten more massive asteroids, which will be<br />
included in the perturbation model [2] [3].<br />
Name Mass [ M⊙ × 10 −10 ] a [AU] e [deg] i [deg] Ω [deg] ω [deg] θ [deg]<br />
Ceres 4.39 ± 0.04 2.7659 0.07976 10.58 80.40 73.15 215.80<br />
Vesta 1.69 ± 0.11 2.3619 0.08936 7.13 103.91 150.18 341.59<br />
Pallas 1.59 ± 0.05 2.7716 0.23075 34.84 173.13 310.34 199.72<br />
Hygiea 0.47 ± 0.23 3.1367 0.11790 3.84 283.45 313.03 91.71<br />
Psyche 0.087 ± 0.026 2.9193 0.13953 3.09 150.34 227.80 141.36<br />
Eunomia 0.042 ± 0.011 2.6436 0.18728 11.73 293.27 97.90 354.91<br />
Hermione 0.0305 ± 0.0013 2.7208 0.08195 6.60 147.93 85.36 154.86<br />
Parthenope 0.0258 ± 0.001 2.4521 0.10019 4.62 125.62 195.25 230.35<br />
Massalla 0.024 ± 0.004 2.4088 0.14276 0.70 206.50 255.49 38.98<br />
Table 2.1: Main asteroids masses and orbital parameters at 2007 – April – 10.0.<br />
The mission<br />
The key features of this mission include<br />
� ≫ 2 s/cs – scalability proof;<br />
� multi communication architecture, typical: sensor → relay → ... → hub → Earth, but<br />
it could be different depending on the agent design;<br />
� asteroid belt environment/perturbations.
2.2. AGENT ARCHITECTURE AND ASSUMPTIONS 15<br />
And the main assumptions on the environment<br />
� reduced two-body gravitational field;<br />
� the gravitational effects of asteroids will be considered as a perturbation force and this<br />
is reasonable if the agents are far from them;<br />
� the rendezvous will not be considered and the agents will be always out of the spheres<br />
of influence of asteroids.<br />
2.2 Agent architecture and assumptions<br />
As it could be understood from these two scenarios, there are different problems to handle.<br />
First of all the spacecrafts have to perform particular missions, they have targets and they<br />
do have not to collide with other objects or each other. This can be called the physical part,<br />
PP, of the control.<br />
Then the spacecrafts are supposed to communicate, both among each other and with<br />
Earth, therefore, since communications are one of the most critical issue in space design, the<br />
communication network deployment and maintenance part, CP, of the control is, at least, as<br />
important as the PP.<br />
Since the multi agent system has to be highly autonomous, both parts have to be thought<br />
in a way they could be very flexible and simple enough to permit real time control by the<br />
agent itself. This is quite obvious since the environment in which the agents will operate is<br />
very dynamic and the Earth – agent communications could take too long. Moreover, this<br />
leads to reduce the communication among the agents as much as possible.<br />
Of course the agents, before even switching on the control, have to decide what to do,<br />
therefore they have to plan in which way they have to act. As stated, this decision has to<br />
be taken with the less communication possible among the agents, then it has to be real time<br />
and robust enough. Hence, the architecture is the one in Figure 2.2.<br />
The main assumptions on the agents are<br />
� they are knowledge and resource bounded, thus they do not have the knowledge of the<br />
whole environment and they have to cooperate to fulfill the tasks;<br />
� they can move use continuous thrust motors, electrical driven in the formation flying<br />
scenario, solar sail driven in the asteroid exploration one, [4];<br />
� they can communicate, both broadcasting or using a peer to peer protocol, but the<br />
range of communication is limited;<br />
� they have sensors to determine their current state and the state of what they sense;<br />
� they can be either explorers or communicators, which means that some agents can have<br />
the capabilities to perform scientific researches – the explorers – but they have lim-<br />
ited communication capabilities; on the contrary other agents cannot perform scientific
16 CHAPTER 2. FORMULATION OF THE PROBLEM<br />
Low Level Control High Level Control<br />
Goal Manager<br />
✄<br />
✄<br />
✄<br />
✄<br />
✄<br />
✄<br />
❈ ❈❈❈❈❈<br />
PP CP<br />
Agent<br />
Figure 2.2: The control architecture.<br />
measurements, but they can communicate in a better way, maybe also with Earth – the<br />
communicators. This implies that explorers cannot communicate each other and they<br />
need communicators to relay data back to Earth; communicators can send information<br />
to other communicators and some of them, the communicator Hubs, can send data back<br />
to Earth, Table 2.2.<br />
Explorers Hubs Communicators<br />
Scientific payload yes no no<br />
Communication only with communicators with all and Earth with all but not with Earth<br />
Table 2.2: Agent composition.<br />
Since, the goal manager and the couple (PP,CP) have to be though in the context of<br />
MAS, the next sections will review some basic concepts of this topic.<br />
2.3 Multi agent techniques<br />
Two paradigms dominate the design of multi agent systems. The first, that will be called<br />
the traditional paradigm, is based on deliberative agents and (usually) central control, while<br />
the second, the swarm paradigm, is based on simple agents and distributed control. In the<br />
past two decades, researchers in the Artificial Intelligence and related communities have, for<br />
the most part, operated within the first paradigm. They focused on making the individual<br />
agents, be they software agents or robots, smarter and more complex by giving them the<br />
ability to reason, negotiate and plan action. In these deliberative systems, complex tasks can<br />
be done either individually or collectively. If collective action is required to complete some
2.3. MULTI AGENT TECHNIQUES 17<br />
task, a central controller is often used to coordinate group behavior. The controller keeps<br />
track of the capabilities and the state of each agent, it decides which agents are best suited<br />
for a specific task, it assigns it to the agents and coordinates communication between them.<br />
Deliberative agents are also capable of collective action in the absence of central control;<br />
however, in these cases agents require global knowledge about the capabilities and the states<br />
of other agents whom they may form a team with. Acquiring such global knowledge may be<br />
expensive and thus impractical for many applications. For instance, a multi agent system<br />
may break into a number of coalitions containing several agents, each coalition being able to<br />
accomplish some tasks more effectively than a single agent can. In one approach to coalition<br />
formation, the agents compute the optimal coalition structure and form coalitions based on<br />
this calculation [LJGM05].<br />
Swarm Intelligence represents an alternative approach to the design of multi agent sys-<br />
tems. Swarms are composed of many simple agents. There is no central controller directing<br />
the behavior of the swarm, rather, these systems are self-organizing, meaning that construc-<br />
tive collective (macroscopic) behavior emerges from local (microscopic) interactions among<br />
agents and between agents and the environment. Self-organization is ubiquitous in nature,<br />
bacteria colonies, amoebas and social insects such as ants, bees, wasps, termites, among oth-<br />
ers, are all examples of this phenomenon. Swarms offer several advantages over traditional<br />
systems based on deliberative agents and central control: specifically robustness, flexibility,<br />
scalability, adaptability, and suitability for analysis. Simple agents are less likely to fail than<br />
more complex ones. If they do fail, they can be entirely pulled out or replaced without signif-<br />
icantly impacting the overall performance of the system. Distributed systems are, therefore,<br />
tolerant of agent error and failure. They are also highly scalable, increasing the number of<br />
agents or task size does not greatly affect performance. In systems using central control,<br />
the high communication and computational costs required to coordinate agent behavior limit<br />
the system size to at most a few dozen agents. Finally, the simplicity of agent’s interactions<br />
with other agents makes swarms amenable to quantitative mathematical analysis. The main<br />
difficulty in designing a swarm is understanding the effect individual characteristics have on<br />
the collective behavior of the system.<br />
Traditional paradigm Swarm paradigm<br />
Control (usually) central distributed<br />
Agent software complex simple<br />
Communication intensive minimal<br />
Time to act long short<br />
Scalability no yes<br />
Reliability poor very good<br />
Table 2.3: Comparison between traditional and swarm paradigm.
18 CHAPTER 2. FORMULATION OF THE PROBLEM<br />
2.3.1 Space multi agent system<br />
In space multi agent systems challenges could be still more. Not least amongst these are<br />
the complex interactions between heterogeneous components, the need for continuous re-<br />
planning, re-configuration and re-optimization, the need for autonomous operation without<br />
intervention from Earth, and the need for assurance of the correct operation of the mission.<br />
In missions such as ANTS [RHRT06], that will be highly autonomous and out of contact<br />
with ground control for extended periods of time, errors in the software may not be observable<br />
or correctable after launch. Because of this, a high level of assurance is necessary for these<br />
missions before they are launched. Testing of space exploration systems is done through<br />
simulations, since it would be impractical or impossible to test them in their final environment.<br />
Although these simulations are of very high quality, often very small errors get through and<br />
can result in the loss of the entire mission, as it is thought to have happened with Mars Polar<br />
Lander Mission [CS00].<br />
Complex missions like these exacerbate the difficulty of finding errors, and will require<br />
new mission verification methods to provide the level of software assurance that for example<br />
NASA requires to reduce risks to an acceptable level. Errors under such conditions can rarely<br />
be found by inputting sample data and checking for correct results. To find these errors<br />
through testing, the software processes involved would have to be executed in all possible<br />
combinations of states (state space) that the processes could collectively be in. Because<br />
the state space is exponential (and sometimes factorial) to the number of states, it becomes<br />
intestable with a relatively small number of processes. Traditionally, to get around the<br />
state explosion problem, testers artificially reduce the number of states of the system and<br />
approximate the underlying software using models. This reduces the fidelity of the model<br />
and may mask potential errors. A significant issue for specifying (and verifying) swarms is<br />
support for analysis and identification of emergent behavior. The idea of emergence is well<br />
known from biology, economics, and other scientific areas. It is also prominent in computer<br />
science and engineering, but the concept is not so well understood by computer scientists<br />
and engineers, although they encounter it regularly. Emergent behavior has been described<br />
as system behavior that is more complex than the behavior of the individual components, [...],<br />
often in ways not intended by the original designers [PV97]. This means that when interacting<br />
components of a system whose individual behavior is well understood are combined within a<br />
single environment, they can demonstrate behavior that can be unforeseen or not explained<br />
from the behavior of the individual components.<br />
2.3.2 Formal methods<br />
Formal methods [Rou06] are proven approaches for assuring the correct operation of complex<br />
interacting systems, being them mathematically-based tools and techniques for specifying and<br />
verifying systems. They are particularly useful for specifying complex parallel and distributed<br />
systems, where the entire system is difficult for a single person to fully understand and<br />
when more than one person was involved in the development. With formal methods, certain
2.3. MULTI AGENT TECHNIQUES 19<br />
properties may be proposed to hold, and prove that they hold. In particular, this is invaluable<br />
for properties that cannot be tested on Earth. By its nature, a good formal specification can<br />
guide researchers to propose and verify certain behaviors (or lack of certain behaviors) that<br />
they would often not think of when using regular testing techniques. Moreover, if properly<br />
applied and used in the development process, a good formal specification can guarantee<br />
the presence or absence of particular properties in the overall system well in advance of<br />
mission launch, or even implementation. Indeed, various formal methods offer the additional<br />
advantage of support for simulation, model checking and automatic code generation, making<br />
the initial investment well worth while. It has been stated that formal analysis is not feasible<br />
for emergent systems, due to their complexity and intractability, and that simulation is the<br />
only viable approach for analyzing emergence of a system [BP03]. For space missions in<br />
general, relying on simulations and testing alone is not sufficient even for systems that are<br />
much simpler than the swarm missions, as noted above. The use of formal analysis would<br />
complement the simulation and testing of these complex systems giving additional assurance<br />
of their correct operation. Given that one mistake can be catastrophic to a system and result<br />
in the loss of money and years of work, development of a formal analysis tool, even at a great<br />
cost, could have huge returns also if only one mission is kept from failing.<br />
Verifying emergent behavior is an area that has been addressed very little by formal meth-<br />
ods, though some work has been done in this area by computer scientists, analyzing biological<br />
systems [SB01] [Tof91]. However, formal methods may provide guidance in determining pos-<br />
sible emergent behaviors that must be considered. Formal methods have been widely used<br />
for test case generation to develop effective test cases. Similar techniques may be used with<br />
formal methods, not to generate a test plan, but to propose certain properties that might or<br />
might not hold, or certain emergent behaviors that might arise.<br />
The Formal Approaches to Swarm Technologies, FAST, project has surveyed formal meth-<br />
ods and formal techniques to determine whether existing formal methods, or a combination<br />
of existing methods, could be suitable for specifying and verifying swarm-based missions and<br />
their emergent behavior [RR04] [RH05] [RR06]. Various methods have been surveyed based<br />
on a small number of criteria that were determined to be important in their application to<br />
intelligent swarms. These include:<br />
� support for concurrency and real time constraints;<br />
� formal basis;<br />
� (existing) tool support;<br />
� past experience in application to agent based and/or swarm based systems;<br />
� algorithm support.<br />
A large number of formal methods, that support the specification of one between, but<br />
not both, concurrent and algorithmic behavior, have been identified. In addition, there
20 CHAPTER 2. FORMULATION OF THE PROBLEM<br />
is a large number of integrated or combination formal methods that have been developed<br />
over recent years, with the goal of supporting the specification of both concurrency and<br />
algorithms. Although the survey identified a few formal methods, to used to specify swarm<br />
based systems, initially only two formal approaches were found that had been used to analyze<br />
the emergent behavior of swarms, namely Weighted Synchronous Calculus of Communicating<br />
Systems (WSCCS) [Tof91] and Artificial Physics [SG99] [SY99a].<br />
The following is a brief description of some specification techniques that have been used<br />
for specifying social, swarm, and emergent behavior:<br />
� Weighted Synchronous Calculus of Communicating Systems (WSCCS), a process alge-<br />
bra, was used by Tofts to model social insects. WSCCS was also used in conjunction<br />
with a dynamical system approach for analyzing the non-linear aspects of social insects.<br />
� X-Machines have been used to model cell biology and modifications, such as Commu-<br />
nicating Stream X-Machines that also seem to have potential for specifying swarms<br />
[Hol88].<br />
� Dynamic Emergent System Modeling Language (DESML), a variant of UML, has been<br />
suggested for use in modeling emergent systems [Kin98].<br />
� Cellular Automata have been used to model systems that exhibit emergent behavior<br />
(land use) [vN96].<br />
� Artificial Physics, which uses physics-based modeling to gauge emergent behavior, has<br />
been used to provide assurance for formation flying as well as other constraints on<br />
swarms.<br />
NASA is currently developing its own formal language based on a mix of the first two<br />
languages and some others, since it thinks that the two are not sufficient by their own.<br />
DESML, though very interesting, has not been chosen because it had not been used or<br />
evaluated outside of the thesis it was developed under. Cellular Automata have not been<br />
selected because they do not have any built in analysis properties for emergent behavior and<br />
because they have been primarily used for simulating emergent systems. Artificial physics,<br />
which is very promising, has not been selected by NASA because of the newness of the<br />
approach [RHRT06].<br />
2.3.3 Artificial Physics<br />
Although Artificial Physics, AP, has not been considered by NASA, it is worth enough to be<br />
analyzed in details, because it could offer several advantages upon the other formal methods.<br />
Moreover, the fact it is quite new is not really a problem, since its newer approach could lead<br />
to reconsider old issues under different perspectives.<br />
AP is a physics oriented approach to construct a coordinated task-allocation algorithm<br />
for cooperative goal-satisfaction [SY99a]. This has to be used by the single agent within the<br />
system and it enables coordination without negotiation and with limited communication.
2.4. WORK’S FRAMEWORK 21<br />
Basically the approach consists in the calculation of a potential function, based on what<br />
a single agent can sense, and in a derivation of the actions that agent has to make. For<br />
this reason the name “Potential Field approach” will be used as well as “Artificial Physic<br />
approach”.<br />
The AP idea can be summarized as<br />
� physics oriented approach, so AP is in the context of formal methods and emergent<br />
behavior can be verified by mathematical tools as statistical mechanics;<br />
� complete reactive behavior, since agents act only as a response of what they sense; this<br />
leads to both scalability and real time algorithms;<br />
� potential function calculation, therefore the approach is very close to the ones used<br />
for example in robot/rover path planners [ZW02], [GC00] [Rei92], [ZS04], [BL92], this<br />
means that space researchers could have a good background to deal with AP.<br />
These properties can respond to the ones expressed before, and since AP could be com-<br />
plete, simple and close enough to previous tested algorithms, it is chosen for this work.<br />
Although the approach is appealing, it has to be kept in mind that it is quite new, thus<br />
only basic test cases have been analyzed in previous works.<br />
2.4 Work’s framework<br />
The work deals with the development of a distributed control for a SMAS using an AP<br />
approach. First of all, the high level control has to be developed, in chapter 3. This is not<br />
the mere application of AP, but a suitable extension of this approach has to be derived to<br />
take into account different capabilities and goals. It is clear that in specifying these properties<br />
some assumptions have to be made. Then, the PP and CP of the control have to be analyzed,<br />
the former in a way which leads to a robust and reliable motion control, chapter 4, the latter<br />
in order to assure the communication network deployment minimizing the communication<br />
effort among the agents, chapter 5. Finally, the perturbation models, which describe the<br />
environments, have to be developed, chapter 6.
22 CHAPTER 2. FORMULATION OF THE PROBLEM<br />
Low Level Control High Level Control<br />
Goal Manager (chap. 3, Algorithm 1)<br />
✄<br />
✄<br />
✄<br />
✄<br />
✄<br />
✄<br />
PP<br />
(chap. 4, Algorithm 2)<br />
❈ ❈❈❈❈❈<br />
Agenti<br />
CP<br />
(chap. 5, Algorithm 3)<br />
Figure 2.3: Work outline.<br />
✛<br />
✛<br />
✲<br />
Agentj<br />
Information sharing<br />
(chap. 3 and chap. 5)<br />
✲<br />
Environment<br />
Targets and Perturbations<br />
(chap. 3 to 6)
CHAPTER<br />
3<br />
THE DECISIONAL LEVEL<br />
People just naturally assume that dogs would be incapable of working together on some sort<br />
of construction project. But what about just a big field full of holes?<br />
Jack Handey (1949 - )<br />
In this chapter the Goal Manager is developed in the framework of Artificial Physics.<br />
First, AP is extended in a suitable way to take into account also multiple tasks and capabilities,<br />
then information sharing mechanism is outlined pointing out key issues.<br />
3.1 Artificial Physics Formulation<br />
Let A = {a1, a2, . . . aN} be the set of agents, which could be spacecrafts but also robots or<br />
whatever. Let G = {g1, g2, . . . , gM} be a set of goals, possibly dynamically changing. These<br />
two sets can be located in a goal space, G, thus A ∈ G and G ∈ G. Let x be the coordinates<br />
of agents and goals in the goal space. Using this notation, both a displacement vector, Dij,<br />
and a metric, dij, can be defined on G as<br />
Dij ≡ xi − xj<br />
dij ≡ ||xi − xj|| 2<br />
(3.1)<br />
(3.2)<br />
Since, in some domains, goals do not have physical properties, the components of Dij are<br />
not necessarily physical distances.<br />
Then it is to assume that:<br />
� the agents have the ability to perceive the displacement vector in the goal space and<br />
they can perceive the properties of other adjacent agents and goals. This may be done<br />
by sensors, integrated into the agents;<br />
23
24 CHAPTER 3. THE DECISIONAL LEVEL<br />
� each agent knows about the types of resources that other agents may have.<br />
These two assumptions are necessary since agents who progress within the goal space need<br />
some information regarding properties of other agents and goals. Moreover it is also to assume<br />
that:<br />
� each agent has a performance capability, mi ∈ R, that can be measured by standard<br />
measurement units, which enable quantification of the agents’ task execution;<br />
� there is a scaling method which is used to represent the displacement of the agents in<br />
the goal space and to evaluate the mutual distances between goals and agents within<br />
this space.<br />
These assumptions are necessary since distances are a significant factor in the AP model.<br />
Moreover it is to assume that goal satisfaction can be achieved progressively. That is, a<br />
goal may be partially satisfied at one instant, and its remaining non satisfied part may be<br />
completed at another point in time.<br />
Then, another basic assumption in AP is that each agent, ai, has a fixed capability,<br />
mi, and each goal requires only a capability to be satisfied. Therefore agents can do only a<br />
single task for goals which require only a single task. This assumption is quite strong and<br />
in the following sections will be removed and the AP approach will be extended in a very<br />
straightforward way.<br />
G<br />
A<br />
• a1<br />
• a3<br />
• an ◦ g1<br />
• an<br />
�<br />
��<br />
• a2<br />
d2n<br />
❅<br />
❅❅❅❅❅<br />
dn4<br />
G<br />
◦ g4<br />
◦ gm<br />
Figure 3.1: Goal space, sets and metric.<br />
Given these assumptions, each agent, ai, in every time instant can compute a potential<br />
function as<br />
◦ g2<br />
◦ g3<br />
Φi = �<br />
Φa(mn, din) + �<br />
Φg(mm, dim) (3.3)<br />
n<br />
m
3.1. ARTIFICIAL PHYSICS FORMULATION 25<br />
where Φa and Φg are suitable potentials, the former takes into account the mutual repulsion<br />
between the agents, since it is not reasonable that many agents perform the same goal, and<br />
it could be in the form<br />
Φa ∝ mn<br />
din<br />
(3.4)<br />
The latter represents the natural attraction towards the goal, which requires a certain amount<br />
of capacity to be fulfilled, mm, and it could be in the form<br />
Φg ∝ − mm<br />
dim<br />
The n and m are the sensed agents and goals.<br />
(3.5)<br />
Example. Just to clarify, an example can be made. Let the goals be the fixing of holes<br />
on a surface, bigger the hole, bigger the value of m. The agents have a certain capability<br />
which is their size, bigger the agent bigger its mi. During time, the agents can eventually fix<br />
a certain amount of a hole, so m for that hole can decrease, and in fact m = m(t).<br />
Once computed the potential function, the agents have to derive the force, since they<br />
have to move towards the minimum of the potential field, which is of course time dependent.<br />
The calculation is straightforward being<br />
F i = −mi∇Φi<br />
(3.6)<br />
The dimensions of vector F i depend on the dimensions of the coordinate vector, x, which<br />
depends on the goal space representation. For example in the scenario of the holes, G is a<br />
subset of R × R and F i is a bidimensional vector.<br />
In Table 3.1 the match between MAS and the shown physical approach is shown.<br />
Multi agent system Physical model<br />
Agent Dynamic particle<br />
Goal Static particle<br />
Agent’s capability Particle’s mass<br />
Agent’s location Particle’s location<br />
Algorithm for goal allocation Formal method for calculating<br />
the evolution of displacement<br />
Table 3.1: The match between MAS and physical model.<br />
The force vector, whom the agent has to act accordingly to, has to be projected by the<br />
same agent in a meaningful space for itself. Let ◦ be the projection transformation, hence<br />
each agent has to compute<br />
Ai = F i ◦ Vi<br />
(3.7)
26 CHAPTER 3. THE DECISIONAL LEVEL<br />
where Ai is the vector of the final actions and Vi is the suitable space for the agent ai.<br />
In the given example, the projection is straightforward since it is only an identity. In fact,<br />
the force vector is indeed a control force, and the example could be regarded as a feedback<br />
control.<br />
3.1.1 AP state of the art<br />
AP has been developed independently by two groups of researchers. The first belongs to<br />
University of Wyoming at Laramie, this is the Spears group; the second is located at Carnegie<br />
Mellon University, Pittsburgh, and it is represented by the person of Onn Shehory. Basically<br />
the approach is the same, but the former group is more focused on real physical goal space,<br />
while the latter is interested in very large MAS for both web and market applications.<br />
Spears. The main idea of Spears group is to try to use the AP approach to move a cluster<br />
of robots in a sort of formation. Therefore the goal space has a complete physical meaning<br />
[SG99]. In Figure 3.2 an example of artificial potential calculation for moving robots: as it<br />
can be seen, the introduction of a virtual agent, which represents the mean motion of the<br />
cluster, is exploited. The Spears group is not the only one which uses the AP in this way,<br />
[EK06].<br />
Figure 3.2: Artificial physics for formations of robots<br />
Shehory. The studies of this second group are both more interesting and more general.<br />
Focused both on web services and market transactions, they imply distributed algorithms<br />
among very large teams of agents. As stated in some papers, the AP approach is practically<br />
the only one capable to deal with very large MAS [SY99a].
3.2. EXTENSION OF AP FORMULATION 27<br />
3.2 Extension of AP formulation<br />
The main strong assumption of AP is that each agent, ai, has a fixed capability, mi, and<br />
each goal requires only a capability to be satisfied. This can be easily removed.<br />
Let the agents, ai ∈ A, i = 1, . . . , n, have multiple capabilities and let C be the matrix of<br />
the total A set capabilities. Then let the goals, gi ∈ G, i = 1, . . . , m, require more tasks to<br />
be fulfilled and let T be the matrix of the total G set tasks.<br />
C can be seen as<br />
⎡<br />
⎢<br />
C ≡ ⎢<br />
⎣<br />
c11 c12 c1k<br />
c21 c22 c2k<br />
. ..<br />
cn1 cn2 cnk<br />
⎤<br />
⎥<br />
⎦<br />
(3.8)<br />
thus C ∈ R n × R k , being k the total number of the capabilities the agents have. Each<br />
coefficient of the C matrix, cij is nothing else but a real number which states how well the<br />
agent ai could perform the task j.<br />
Whereas T can be seen as<br />
⎡<br />
⎢<br />
T ≡ ⎢<br />
⎣<br />
t11 t12 t1q<br />
t21 t22 t2q<br />
. ..<br />
tm1 tn2 tmq<br />
⎤<br />
⎥<br />
⎦<br />
(3.9)<br />
thus T ∈ R m × R q , being q the total number of the tasks the goals require. Each coefficient<br />
of the T matrix, tij is nothing else but a real number which states how much of the task j<br />
serves to fulfil the goal gi. It is straightforward that T is time dependent.<br />
Clearly k � q since otherwise the agents could not perform the goals. Let k = q, with<br />
no loss of generality. By the assumptions of extended AP approach, each agent can write k<br />
potentials which live in the goal space G as<br />
Φ w i = �<br />
αnicnwciw Φa(din) + �<br />
βnitnwciw Φg(dim) , w = 1, . . . , k (3.10)<br />
n<br />
m<br />
where, as before, Φa and Φg are suitable potentials, the former takes into account the<br />
mutual repulsion between the agents, since it is not reasonable that many agents perform<br />
the same goal; the latter represents the natural attraction towards the goal, which requires a<br />
certain amount of capacity to be fulfilled. Note that the potentials have lost their dependence<br />
on capabilities. In the end α and β are coefficients.<br />
For each Φ w i and for each ai, a force vector can be computed as<br />
F w i = −∇Φ w i (3.11)<br />
Then, by a suitable projection transformation, each agent can compute the real action.<br />
Note that, in this case, force vectors could be mutually contrasting, thus the transformation<br />
has to include some conflict detection/solver techniques, F.<br />
�<br />
Ai = F F 1 i ◦ V 1 i , F 2 i ◦ V 2 i , . . . , F k i ◦ V k �<br />
i<br />
(3.12)
28 CHAPTER 3. THE DECISIONAL LEVEL<br />
3.3 Formal methods<br />
As stated, AP formulation is chosen for this work mainly because it could be seen as a formal<br />
method, thus it could verify the presence of emergent behavior. In the new extended AP<br />
version, since classical mechanics does not provide many different properties a body can have<br />
more than inertia properties, statistical mechanics has to be used as suggested by [SY99a],<br />
[DM06]. This parallelism, although it is not difficult to develop, will not be explored in this<br />
work.<br />
3.4 Information propagation and knowledge bounded agents<br />
Since the agents are knowledge bounded, they cannot have a global representation of the<br />
environment and thus they may not have any target. This can overcome through a suitable<br />
information sharing architecture, which have to use the communication less as possible.<br />
Different techniques have been proposed to solve the target assignment problem via in-<br />
formation sharing, for example [SB07a] shows how a broadcast coupled with a distributed<br />
algorithm could be optimal in some cases, namely when the environment is sparse. Other<br />
approaches, [SN08], [YS06], use randomly peer to peer sent packets of information, which<br />
have the advantage to decrease the communication effort.<br />
Even if, the suitable algorithm could be a mix of the two, since the target assignment<br />
frequency is supposed to be low compared to motion dynamics and typical communications,<br />
the broadcast approach is adopted. Moreover, this approach is simpler than the peer to peer<br />
one.<br />
3.4.1 Example<br />
To try to understand how the information sharing mechanism works an example can be made.<br />
Let g1, g2, g3 be three asteroids in a three dimensional space and let ai ∈ A be the agents, the<br />
spacecrafts, randomly located in the space, which have to choose which asteroid reach. Each<br />
ai has three different capabilities, {ci1, ci2, ci3}, which could be, for example: take photos,<br />
communicate, detect water. Each asteroids, gj requires, at the beginning, a certain amount<br />
of capabilities, {tj1, tj2, tj3}, which can be thought as proportional to the asteroid surface. At<br />
the initial time τ0, only a subset of the agents, A ⊆ A, can actually sense the asteroids, and,<br />
supposing the could have a complete knowledge of the object, they can set the {tj1, tj2, tj3}<br />
for each asteroid they sense. Thus each ai ∈ A can define its own TS, which is the sensed T<br />
matrix as:<br />
where j ∈ {sensed object}.<br />
TS =<br />
�<br />
tj1 tj2 tj3<br />
. . .<br />
Then, using the Goal Manager, namely the potential field 3.10 and the following formulae,<br />
with T = TS, they can decide which asteroid reach. Chosen their target, namely j, they<br />
�
3.5. <strong>DI</strong>STRIBUTED SCHEDULING 29<br />
change the {tj1, tj2, tj3} accordingly, subtracting their capacities<br />
{tj1, tj2, tj3}τ1 = {tj1, tj2, tj3}τ0 − {ci1, ci2, ci3} (3.13)<br />
and broadcast the information adding to the packet also the time in which they decided, thus<br />
where TRi = TS at τ1.<br />
Msgi ≡ {TRi , τ1}<br />
When other agents receive the information arriving from different sources, they added it<br />
to what the have sensed, TS, they form a new T as<br />
�<br />
�<br />
T = TS ∪<br />
where the i–th agent is the one who has broadcast them the information.<br />
Then they decide, subtract and broadcast. This process of receiving and sending has a<br />
i<br />
TRi<br />
fixed frequency, depending on the sparsity of the systems.<br />
It has to be noted that in some cases agents have to decide upon conflicting messages,<br />
since the information path is not unique. To solve these conflicts policies could be added<br />
to the agent control, but by now, only a simple policy has been developed: the time policy.<br />
The newest message has to be followed and if more messages than one are the newest, a<br />
random choice is performed. Using this policy the greater the agent number is, the more<br />
accurate the target assignment is. Defining a Sum operator which embodies the policy, the<br />
T determination has to be changed accordingly as<br />
3.4.2 Algorithm<br />
�<br />
T = Sum(TS, SumiTRi )<br />
The final, Goal Manager algorithm is, for each agent, ai the Algorithm 1<br />
3.5 Distributed scheduling<br />
As stated, in most of the cases, in the context of multi agent systems conflicts appear. This is<br />
the case of overlapping requests of the same resource, for example, when two explorers would<br />
like to use the same communication antenna to relay data back to Earth. This problem can be<br />
solved in the framework of distributed scheduling using different approaches, like decoupling<br />
strategies [BL08], but it will not face in this work.
30 CHAPTER 3. THE DECISIONAL LEVEL<br />
Algorithm 1 Goal Manager and sharing mechanism, at τ = τ⋆<br />
1: TS = Sense(Environment)<br />
2: TR = {}<br />
3: repeat<br />
4: Td =GetMsg<br />
5: TR = Sum(TR, Td)<br />
6: until (No more messages)<br />
7: T = Sum(TS, TR)<br />
8: /*Goal Manager Start*/<br />
9: Φw i , eq. 3.10<br />
10: F w i , eq. 3.11<br />
11: Ai, eq. 3.12<br />
12: /*Goal Manager End*/<br />
13: Form TR, eq. 3.13<br />
14: BroadCast Msg ≡ {TR, τ⋆+1}
CHAPTER<br />
4<br />
THE PHYSICAL PART<br />
Go, Traveler. Go anywhere. The universe is a big place, perhaps the biggest.<br />
Philip J. Farmer (1918 - )<br />
In this chapter the physical part of the control is developed. First of all, the tracking<br />
problem is presented as a reasonable solution in the scenarios which have to be explored.<br />
Then different approaches to track the targets are shown, and it is explained how a suitable<br />
anna algorithm for long period dynamics, coupled with a short period dynamic control, is a<br />
reasonable solution which mixes both lightweight and robust features.<br />
4.1 Introduction<br />
The physical part of the control is in charge of moving agents towards dynamic evolving<br />
targets and since, the scenarios involve only physical goals, the problem which has to be<br />
solved could be formulated as:<br />
Given an initial condition on the state variables, x0, of each agent, ai, at the time t0, find<br />
the optimal control u which leads x0 → xf , the target state variables, at the time tf .<br />
The problem which each agent has to solve could be seen in two main ways:<br />
� as a Lambert problem, but, since this involves a lot of computational effort, it is not<br />
reasonable for a real time calculation;<br />
� as a tracking problem/feedback control, to be resolve either using AP approach or, that<br />
is the same in this case, with a suitable feedback control law. In this way, although it<br />
is not necessary, it is convenient for the control effort, to set tf → ∞, thus the problem<br />
becomes an infinite horizon control.<br />
31
32 CHAPTER 4. THE PHYSICAL PART<br />
4.2 Possible solutions<br />
4.2.1 Potential field approach<br />
The potential field approach has been used by ESA in different demonstrative softwares for<br />
SMAS [IP07]. The target position global knowledge is the key assumption of an equilibrium<br />
shaping technique, in order to find an optimal set of parameters which can lead the system<br />
towards a final equilibrium. Since in the scenarios of this work the information is not global<br />
but distributed, this assumption has to be removed and that parameter set has to be tuned<br />
in another way.<br />
Although this could be easily done, problems arising from the use of such a formulation<br />
could be difficult to overcome. The main one is that the control u is far from being optimal,<br />
as long as a suitable term in the potential field is not included. This term would shape<br />
the geometry of the space according to the geodetic lines of the dynamical system, thus it<br />
would exploit the space properties, instead of involving useless control effort. Although this<br />
approach could be easily developed dealing with a linearized system, it is not straightforward<br />
in a complete non linear one, since it involves a sort of dynamic inversion. For that reason<br />
this approach will not be used.<br />
4.2.2 SDRE approach for non linear systems<br />
The use of a feedback control, which takes into account the dynamical system non linearity,<br />
could be a suitable approach for the tracking problem. Initially developed for intelligent<br />
missiles applications, it has revealed some capabilities in space area since few years.<br />
The basic idea is to write the non linear dynamical system in an affine way [Bra04] [PB04],<br />
assuring that the dynamic matrices are not singular for x ∈ {x0, . . . , xf }. Thus, since the<br />
reduced two-body problem in a cartesian coordinate system is<br />
¨r = − κ<br />
r + u + w (4.1)<br />
r3 where κ is the planetary constant and w is the perturbation acceleration vector, it can be<br />
rewritten in the affine shape, introducing the state vector x = {r, ˙r} T as<br />
⎡<br />
03<br />
I3<br />
⎤<br />
⎡<br />
⎢<br />
˙x = ⎢<br />
⎣<br />
− − −− − − −−<br />
− κ<br />
⎥ ⎢ ⎥<br />
⎦ x + ⎣−<br />
− −−⎦<br />
(u + w) (4.2)<br />
I3 03<br />
r3 I3<br />
� �� � � �� �<br />
A(x)<br />
B<br />
Since it is clear that the the matrix A can not be singular, the control problem can be<br />
formulated using a State Dependent Riccati Equation, SDRE, introducing an error variable<br />
e ≡ x − xf and writing a suitable cost function J as<br />
J ≡<br />
� ∞<br />
t0<br />
03<br />
⎤<br />
e ′ Q e + u ′ R u − γ 2 w ′ w dt (4.3)
4.2. POSSIBLE SOLUTIONS 33<br />
Then u has to be found in a H∞ context thus it satisfies a particular steady state Riccati<br />
Equation. Two main problems arise from this kind of approach:<br />
� the use of cartesian coordinates lead to worse results than the use of more suitable set<br />
of coordinates; this has a numerical explanation;<br />
� the control u does not take into account collision avoidance.<br />
Both the first and the second problem can be fixed; in fact for the former a different set<br />
of coordinates has to be chosen, whereas for the latter two main ideas could be used:<br />
� a dynamical separation between long period and short period, thus the long takes in<br />
consideration the tracking problem, and the short the collision avoidance problem. The<br />
latter has to be formulated in a suitable way.<br />
� The control u could be written as<br />
u = −Ky, with y = f(ρa, ρT ) (4.4)<br />
where f(ρa, ρT ) is a suitable function which depends on the distance among agents ρa<br />
and with the target ρT . This leads to an output feedback control [Gad07] which is both<br />
too complex to resolve real time and possibly ill conditioned.<br />
The above considerations lead to the choice of a dynamical separation and the use of a<br />
different set of variables.<br />
4.2.3 Non linear Lyapunov control<br />
Although state dependent, the SDRE approach is nothing else but an extention of the usual<br />
linear control to non linear systems. It could offer some advantages but, since the system<br />
is highly non linear, the use of a non linear control, whose approach is very close to SDRE,<br />
could be more suitable.<br />
First of all, using the non singular equinoctial variables, which are chosen as the set of<br />
coordinate system, to have better numerical results, the equations of motion 4.1, neglecting<br />
the perturbations, can be written as [Naa02]<br />
where<br />
˙x = B(x) u (4.5)
34 CHAPTER 4. THE PHYSICAL PART<br />
� the state vector x = {a, P1, P2, Q1, Q2, l0} T is related to the classical elements set as<br />
�<br />
⎧<br />
⎪⎨<br />
⎪⎩<br />
a = a<br />
P1 = e sin ¯ω<br />
P2 = e cos ¯ω<br />
Q1 = tan i<br />
2<br />
Q2 = tan i<br />
2<br />
sin Ω<br />
cos Ω<br />
l0 = ¯ω + M − nt<br />
⎧<br />
⎪⎨<br />
⎪⎩<br />
a = a<br />
e =<br />
�<br />
P 2 1 + P 2 2<br />
i = 2 tan−1 �<br />
Q2 1 + Q22 Ω = tan −1<br />
ω = tan −1<br />
� Q1<br />
Q2<br />
� P1<br />
M = l − tan −1<br />
P2<br />
�<br />
�<br />
− tan −1<br />
with ¯ω = ω + Ω and l = l0 + nt. Moreover the true longitude, L, can be defined starting<br />
from the mean longitude l as L = ¯ω + θ. In the end l0, which is the mean longitude<br />
at epoch is set in a way so that the equations of motion loose the constant term and<br />
l0 → 0 for t → ∞; in fact l0,0 = l − nlf /nf ;<br />
⎡<br />
⎢<br />
B(x) = ⎢<br />
⎣<br />
with<br />
2a 2 (P2 SL − P1 CL)<br />
h<br />
− pCL<br />
h<br />
pSL<br />
h<br />
2a 2 p<br />
hr<br />
r � P1 + � 1 + p�<br />
�<br />
r SL<br />
h<br />
r � P2 + � 1 + p�<br />
�<br />
r CL<br />
h<br />
0 0<br />
0 0<br />
− pa � (P1SL + P2CL) + 2b<br />
�<br />
a<br />
h(a + b)<br />
�<br />
b = a 1 − P 2 1 − P 2 2 , h = nab ,<br />
r<br />
h =<br />
h<br />
κ(1 + P1SL + P2CL)<br />
� P1<br />
P2<br />
p<br />
ra(1 + r − )(P1CL − P2SL)<br />
h(a + b)<br />
In the framework of Lyapunov control, this theorem holds<br />
�<br />
� Q1<br />
Q2<br />
p<br />
r = 1 + P1SL + P2CL<br />
, C ≡ cos , S ≡ sin<br />
�<br />
(4.6)<br />
0<br />
− P2(Q1CL − Q2SL)r<br />
h<br />
P1(Q1CL − Q2SL)r<br />
h<br />
r(1 + Q2 1 + Q2 2)SL<br />
2h<br />
r(1 + Q2 1 + Q2 2)CL<br />
2h<br />
− r(Q1CL<br />
⎤<br />
⎥<br />
− Q2SL) ⎦<br />
h<br />
(4.7)
4.2. POSSIBLE SOLUTIONS 35<br />
Figure 4.1: Classical orbital parameters
36 CHAPTER 4. THE PHYSICAL PART<br />
Theorem. Given a positive definite function, V = V (x) in Ω, which can be called a Lya-<br />
punov function, if the time derivate of V is negative definite in a subset ¯ Ω of Ω, then the<br />
system is asymptotical stable in ¯ Ω.<br />
Thus choosing V as<br />
V = 1<br />
2 δx′ Q δx , δx = x − xf (4.8)<br />
since the time derivate can be written as<br />
D<br />
Dt V = δx′ Q δ ˙x = δx ′ Q Bu (4.9)<br />
because ˙xf = 0, taking u as<br />
u = − 1<br />
2 K B′ Q ′ δx (4.10)<br />
with K ≻ 0, the time derivate of V is negative definite.<br />
This approach has different advantages, both it involves simple calculations which could<br />
be done in real time and it assures the asymptotical stability of the control. Of course, it has<br />
to be extended for the case in which also perturbations are taken into account and, moreover,<br />
it has to be united to a short period dynamic control for collisions avoidance.<br />
It has to be noticed that the dynamic separation is reasonable since the space environment<br />
is supposed to be sparse, this means that defining a typical free path length, λ, as the path<br />
length an agent has to travel before it encounters another agent, and defining the typical<br />
averaged length which the agent travels in a time step, δℓ, the relation λ ≫ δℓ holds.<br />
4.3 Long period dynamic<br />
4.3.1 Robust non linear Lyapunov control<br />
In this section the work of [Naa02] will be extented, taking into account the perturbations.<br />
The equations of motion are<br />
The problem can be reformulated in the H∞ approach as:<br />
˙x = B(x) [u + w] (4.11)<br />
Find both a definite positive Lyapunov function V whose time derivate<br />
D<br />
V = ∇V ˙x = ∇V B (u + w) (4.12)<br />
Dt<br />
is negative definite, and the minimal u which makes the cost function<br />
L(x, u, w) =<br />
� ∞<br />
stationary with the maximum value of perturbations w.<br />
This statement leads to solve<br />
t0<br />
δx ′ ˜ Q δx + u ′ R u − γ 2 w ′ w dt (4.13)<br />
0 = min<br />
u max<br />
w [L(x, u, w) + ∇ V B (u + w)] (4.14)<br />
which is called the Hamilton – Jacobi – Isaacs, HJI, equation [GG96].
4.3. LONG PERIOD DYNAMIC 37<br />
4.3.2 HJI equation<br />
The equation 4.14 can be resolved with iterative techniques and, in fact, the resulting algo-<br />
rithm is not so different from a SDRE approach.<br />
The result is that [AKL06]<br />
⎧<br />
u = − 1<br />
2 R−1 B ′ (∇V ) ′<br />
⎪⎨<br />
⎪⎩<br />
w =<br />
where<br />
1<br />
2γ 2 B′ (∇V ) ′<br />
0 = (∇ V ) B ′<br />
It is simple to rewrite the last equation as<br />
� �<br />
I3<br />
− R−1 B (∇ V )<br />
γ2 ′ + δx ′ Q˜ δx<br />
(4.15)<br />
0 = 2∇ V B ′ (u + w) + δx ′ ˜ Q δx (4.16)<br />
and seek an iterative solution for ∇ V , starting from u 0 and w 0 [BM00]. For this solution<br />
also a Galerkin approximation could be used, but, since this way leads to sixth order integrals<br />
for the dimension of the state space, it is not selected.<br />
The algorithm 4.15 has two main disadvantages, the first is to find a good starting point<br />
for the convergence, the second is that it includes a pseudo-inverse calculation. The way in<br />
which these problems can be fixed is by adapting the control to the case A = 0 in a new way.<br />
The first step is seeking a solution for the Lyapunov control in the form<br />
V = 1<br />
2 δx′ Q δx (4.17)<br />
which is reasonable, then substituting it in the solution of HJI equation<br />
0 = δx ′ Q B ′<br />
� �<br />
I3<br />
− R−1 B Q δx + δx<br />
γ2 ′ Q˜ δx (4.18)<br />
thus leading to a particular symmetric equation,<br />
0 = δx ′<br />
�<br />
Q B ′<br />
� �<br />
I3<br />
− R−1 B Q +<br />
γ2 ˜ �<br />
Q δx = δx<br />
� �� �<br />
W<br />
′ W δx (4.19)<br />
which is nothing but a particular Riccati equation, which can be called for simplicity anna<br />
equation for the palindrome. Resolving anna implies determining Q, so that<br />
Q B ′<br />
�<br />
�<br />
B Q = ˜ Q (4.20)<br />
− I3<br />
+ R−1<br />
γ2 which could be found using a Newton algorithm approach, since the equation is non linear, but<br />
it would lead to the same problems as before. Since it is basically the same using as control
38 CHAPTER 4. THE PHYSICAL PART<br />
weights the couple ( ˜ Q, R) or (Q, R), the problem can be reformulated, without numerical<br />
problems, assuring that both �<br />
and<br />
− I3<br />
+ R−1<br />
γ2 w ≥ wmax<br />
where wmax is the maximum expected value of the perturbations.<br />
�<br />
≻ 0 (4.21)<br />
The algorithm, which can be called anna algorithm, can be summarized as follows<br />
(4.22)<br />
� Choice a couple (Q, R) and γ which satisfy the definite positiveness of the matrix 4.21<br />
� Calculate both u and w using<br />
⎧<br />
� verify w ≥ wmax<br />
⎪⎨<br />
⎪⎩<br />
u = − 1<br />
2 R−1 B ′ Q δx<br />
w = 1<br />
2γ 2 B′ Q δx<br />
(4.23)<br />
As stated, this approach is the optimal solution of an H∞ problem in which the anna<br />
equation holds<br />
˜Q = Q B ′<br />
�<br />
− I3<br />
+ R−1<br />
γ2 �<br />
B Q<br />
It is important to note that since wmax is known, and u is bounded, in fact u < ū, it is<br />
possible that for some perturbation the control effort could not be sufficient to overcome the<br />
perturbation. However this is a design problem and not really a control problem.<br />
4.3.3 R as a function of the state<br />
In most of the cases, it is reasonable to have a thrust which is almost constant over time. This<br />
can be achieved using a control weight, R, which is modulated on the state, thus R = R(x).<br />
Moreover if R is symmetric and definite positive for each x the control can be proved to be<br />
stable, thus<br />
(∀x(R(x) ≻ 0) ∧ (R(x) = R(x) ′ )) ⇒<br />
� �<br />
D<br />
V ≺ 0<br />
Dt<br />
Proof. The proof is straightforward, since R is symmetric and definite positive, there exists<br />
a Cholesky decomposition thus<br />
R = HH ′<br />
with H = H(x), hence the time derivative of the Lyapunov function V is<br />
D<br />
V = −1<br />
Dt 2 δx′ Q B H H ′ B ′ Q ′ δx<br />
� �� �<br />
y<br />
= − 1<br />
2 y′ y ≺ 0<br />
�
4.4. SHORT PERIOD DYNAMIC 39<br />
4.4 Short period dynamic<br />
As stated, the collision avoidance can be handle using a suitable short dynamic control, thus<br />
each agent trajectory is modified in an optimal way to face possible conflicts. Designing such<br />
a control law is not trivial because it has to be, at least<br />
� real time;<br />
� robust enough to overcome perturbations and model/sensor errors.<br />
It could be thought that a suitable repulsion force, in a potential field way, could solve the<br />
problem easily. This is not the case, since potential functions have local minima which cannot<br />
assure a optimal control. Mainly there are two reasonable approaches, the first is to use an<br />
optimal feedback control, in the fashion of LQR or H∞, coupled with a velocity Potential<br />
field, as in [IP07] or [MY07]. This leads to good results and lightweight control. The key idea<br />
is to define a velocity field that make the spacecraft move far from the obstacles and close to<br />
the targets. Then a feedback control is added to obtain the required velocity.<br />
Another reasonable approach is the one of [IH02], [Arm04], even if it has to be extended<br />
to let it be fully distributed and moreover it has to be integrated into the long dynamic<br />
control law. The concept of [Arm04] approach, which it will be called AA now on, could be<br />
summarize as follows:<br />
Given the initial and the final positions of all the agents, find the optimal control law which<br />
enables them to reach their goals without any collisions.<br />
The AA involves linear simplifications, the global knowledge of where each agent is, and<br />
moreover it assumes to have a reliable perturbation model. Since these three main points are<br />
not acceptable for the problem which has to be faced, some extensions have to be thought<br />
and developed. First of all, the linear behavior assumption can be adopted if the reference<br />
orbit is the actual one, thus it is dynamically changing by the long dynamic control; second,<br />
the algorithm has to be thought as distributed, thus a policy, of a suitable decisional function<br />
has to be found and added. In the end, the control law has to be robust enough to face<br />
uncertainty in the perturbation model.<br />
4.5 Final Algorithm<br />
In this section only the long dynamic algorithm is exploited. Each agent, every time step,<br />
knowing its own target use the Algorithm 2.
40 CHAPTER 4. THE PHYSICAL PART<br />
Algorithm 2 Long dynamic control<br />
1: x ← (Actual State)<br />
2: xf ← (Target)<br />
3: δx = x − xf<br />
4: B = f(x), eq. 4.7<br />
5: R = R(x), section 4.3.3<br />
6: u, eq. 4.23
CHAPTER<br />
5<br />
THE COMMUNICATION PART<br />
Democracy means government by discussion,<br />
but it is only effective if you can stop people talking.<br />
Clement Attlee (1883 - 1967)<br />
This chapter deals with the communication network assurance part of the control, thus,<br />
first of all, the problem is formulated in the context of graph theory and sensor networks.<br />
Then, a standard potential field/ distributed algorithm is presented and extended using a<br />
dynamic – token based approach. Various policies which determine the way in which tokens<br />
are sent and kept are examined on the basis of local connectivity.<br />
5.1 Introduction<br />
Communications are critical in space environment since, once they are lost, the entire mis-<br />
sion could be lost, or at least out of control. Although SMAS are designed to be highly<br />
autonomous, trying to have a almost constant connection with the agents is at least reason-<br />
able and perhaps required, thus the communication network has to be deployed.<br />
Communications can be divided in two main group:<br />
� peer to peer, or P2P, in which the information is sent to one and only one agent per<br />
time;<br />
� broadband, in which the information is just spread everywhere, therefore every agent in<br />
the communication range can have access to it.<br />
Of course the former is less time consuming, given a certain amount of byte to be sent,<br />
or it could send more information, given the time interval. On the contrary, the latter is<br />
simpler since it does not involve complex mechanism to individuate where the other agents<br />
41
42 CHAPTER 5. THE COMMUNICATION PART<br />
are, although this is not a really hard problem. In fact agents know where the others are and<br />
with an electronic phasing antenna, also the mechanical difficulties could be overcome.<br />
Defining:<br />
� graph: a set of vertices V connected together by edges E; therefore the a graph G can<br />
be represented by a V and a connectivity matrix KE, which states which vertice is<br />
connected to which. Hence<br />
G ≡ (V, KE); (5.1)<br />
� k-connectivity: the graph G is k – connected if and only if for each vertice Vi there<br />
exists at least k connections.<br />
7<br />
6<br />
5<br />
5<br />
4<br />
5<br />
Figure 5.1: A 3-connected graph<br />
The control problem for the communication part can be written as:<br />
Given, in every time instant, the position of the explorers, built a k-connected graph with the<br />
communicators.<br />
Where k has to be chosen to assure a good fault tolerance property.<br />
5.2 Artificial Physics and Token based algorithm<br />
Dynamically deploying effective networks is difficult for a variety of reasons. First, the com-<br />
municators will not have a priori knowledge of where the explorers will go, nor of the environ-<br />
ment in which they must deploy. Second, to coordinate their deployment they must maintain<br />
communication with each other or coordinate without communication. Even if communica-<br />
tion between the agents is available, its use has to be minimized, both to make bandwidth<br />
5<br />
3<br />
5<br />
3
5.3. PROBLEM STATEMENT 43<br />
available for explorers to relay information back to a possible hub and to allow commands to<br />
be relayed to agents. Third, there is typically no clearly defined deployment stage, thus the<br />
ad hoc network needs to be maintained for the explorers while the communicators deploy to<br />
their positions. Finally, the agents may need to constantly rearrange to adjust to explorers<br />
movement or failed communicators, since typically they cannot provide coverage to the whole<br />
environment.<br />
A variety of approaches have been developed for this problem. In ad hoc networks and<br />
sensor networks, distributed algorithms, which can assure k - connected graphs [BR05], allow<br />
robust robot positioning [SL02] and provide good coverage [MS01], have been applied in rela-<br />
tively open environments. However those efforts largely ignore situations in which signals are<br />
impeded by obstacles, like walls or asteroids, or in which only a small dynamically changing<br />
part of environment needs coverage.<br />
Artificial Physics and Potential field are lightweight and robust way of positioning agents<br />
in a clustered and complex environment [HS02], often not requiring any communication to<br />
coordinate. However, potential fields are best suited for spreading agents out across an<br />
environment, not focusing them on dynamically changing areas. Hence, for this application,<br />
key extensions to Artificial Physics and potential fields were required to take advantage of<br />
their strengths while meeting problem constraints.<br />
The central idea of this work is to dynamically change the applicable potential fields based<br />
in the current overall needs of the team. If the potential fields can be appropriately varied,<br />
the agents will robustly move to locations where a connected network can be formed.<br />
The key to the dynamic potential field approach is to ensure that each communicators is<br />
influenced by appropriate fields at appropiate times. Specifically, the team must configure<br />
itself so that some communicators move near to the explorers, while others position themselves<br />
to relay massages to and from the hub. To achieve this, each agent sends out requests for<br />
other agents to connect it back to the hub or in the hub’s case, sends out requests to be<br />
connected to the network. These requests are in the form of tokens. When an agent receives<br />
a token it either keeps the token, adds a potential field corresponding to the request for<br />
support represented by the token, or passes the token on to another agent (which faces the<br />
same choice). By controlling the number of tokens each agent sends out, the number of links<br />
the team tries to form with the requester can effectively be controlled. The policy by which<br />
an agent decides to keep a token, and add the corresponding field, or pass the token on,<br />
dictates the effectiveness and the nature of the network.<br />
5.3 Problem Statement<br />
Let S = {S1, . . . , Sn, H} be a set of moving agents, Si, the explorers, and a hub H, and let<br />
C = {C 1 , . . . , C m } be a set of communication agents, the communicators. The basic aim is<br />
to position C to create a network which connects each Si to H.<br />
Si is assumed to be independent of C i but both can move at the same speed. Si and C i<br />
have a maximum range of communication, dc. It is assumed that every agent can sense where
44 CHAPTER 5. THE COMMUNICATION PART<br />
the others are if they are within their communication range and agents can distinguish be-<br />
tween explorers and communicators. This may be done by overhearing messages broadcasted<br />
by other agents. Let S ⊆ S and C ⊆ C be the subsets of explorers and communicators a<br />
agent can sense respectively.<br />
Let<br />
Let x be the position of a generic agent at a given time, while the hub, H is stationary.<br />
Pi(Sk) = {Sk, C i , . . . , C q , H}<br />
be a possible communication path from Sk ∈ S to H, thus the distance between two consec-<br />
utive elements, pi, of Pi(Sk) is at most dc,<br />
|x(pi) − x(pi−1)| ≤ dc<br />
Two paths from the same explorer are different if they involve different communicators:<br />
Pi(Sk) �= Pj(Sk) ⇔ ∄ C i |(C i ∈ Pi(Sk)) ∧ (C i ∈ Pj(Sk))<br />
Among all the Pi(Sk) which have at least a C i in common, a minimal path can be defined<br />
as the one which involves the minimum number of communicators:<br />
minPi(Sk) = Pi(Sk) if |Pi(Sk)|is minimum<br />
All the different paths from the same explorer can be grouped in the local subset of different<br />
paths, P (Sk): P (Sk) = {. . . , Pi(Sk), . . . } where<br />
∀Pi(Sk), Pj(Sk) | (i �= j) ∧ (Pi(Sk), Pj(Sk) ∈ P (Sk)) ⇒<br />
⇒ Pi(Sk) �= Pj(Sk)<br />
Let the local connectivity c be c = |C| and let the connectivity explorer - Hub at a given<br />
time t, be<br />
Ki(t) = |P (Si)|<br />
Thus, Ki(t) = 0 means there are no communication paths from Si to H, thus Si is not<br />
connected. The primary goal of C is to avoid this happening. In Figure 5.2 the connectivity<br />
is 1 for S1 and 2 for S2. Let the global connectivity at a given time be<br />
K(t) is 1 in Figure 5.2.<br />
K(t) = min<br />
i Ki(t)<br />
A communicator is useful if it is on a minimal path. Let the used communicators subset<br />
be the subset:<br />
�<br />
�<br />
�<br />
U = minPi /S<br />
i<br />
i.e., the useful communicators. Define efficiency, E, as the ratio of useful communicators to<br />
total communicators:<br />
E = |U|/|C|
5.4. POSITIONING ALGORITHM 45<br />
H<br />
C 2<br />
C 3<br />
C 4<br />
Figure 5.2: An example of possible network which connects two explorers, S1 and S2 and<br />
communicators C 1 , . . . , C 4 to the hub H. The dashed line are communication links, the black<br />
line is a wall.<br />
In Figure 5.2, E = 3/4 , since C 1 is not on a minimal path.<br />
C 1<br />
Let < K > and < E > be the average global connectivity over time and the average<br />
efficiency over time respectively.<br />
Finally let v be the environment change rate, characterized as the maximum rate a com-<br />
municator has to move to prevent the network breakdown. This gives a rough measure of<br />
the environment difficult for C.<br />
The problem is to:<br />
5.4 Positioning Algorithm<br />
�<br />
�<br />
max min K(t)<br />
0≤t≤tmax<br />
The basic concept of AP and potential field is to overlap fields representing different influences<br />
on the agent. The agent then simply follows the gradient down the resulting field. The basic<br />
potential function, Jj, utilizes the Lennard - Jones formulation [SY99a], resulting in:<br />
Jj( � S, � C) = α �<br />
Si∈ � S<br />
� �6 � �12 dcfs dcfs<br />
−2 +<br />
+ β<br />
rCj Si rCj Si<br />
�<br />
Cq∈ � � �6 � �12 dcfc dcfc<br />
−2 +<br />
rCj Cq rCj Cq C<br />
S 1<br />
S 2<br />
(5.2)<br />
where � S ⊆ S and � C ⊆ C are the subsets of explorers and communicators which influence a<br />
agent, r C j Si and r C j C q are the relative distances between Cj and the agents in those subsets.<br />
The communication distance dc and the fs and fc coefficients determine the function shape.<br />
The coefficients α and β state whether to move further from explorers or communicators. If<br />
| � S| = 1, | � C| = 0, Jj( � S, � C) would have a minimum at a distance dcfs from Si; below that<br />
distance Jj( � S, � C) would increase not to prevent agents from being too near one another,
46 CHAPTER 5. THE COMMUNICATION PART<br />
while above that distance it would increase to keep agents in the communication range. Once<br />
the potential function has been evaluated, the agent moves toward the local minimum.<br />
In the next sections different versions of this approach will be presented. What varies<br />
among the versions are � S and � C, i.e., which agents effect the potential field. First the basic<br />
potential field algorithm, in which � S = S and � C = C, i.e., where the potential field is<br />
influenced by all agents in sensor range. Second, a version where tokens are passed around<br />
the team, with the agent represented by the token being an influence in � S and � C.<br />
5.4.1 Standard algorithm<br />
In the basic algorithm, referred to as standard, � S = S and � C = C, thus every sensed agent<br />
influences the potential field shape. This leads to the agents spreading out the environment,<br />
since Jj( � S, � C) makes the relative distances among agents almost the same. The main problem<br />
with the standard approach is that, when the environment is large, spreading out is not an<br />
acceptable solution, since coverage can not be assured.<br />
5.4.2 Dynamic Potential Fields<br />
The key is to have the communicators move to the parts of the environment where explorers<br />
are, not just anywhere. Since in the standard approach the balance between attractive<br />
and repulsive force, i.e., the potential field gradient, determines the spreading pattern, it is<br />
reasonable that if communicators could cooperate they could turn off useless repulsive forces,<br />
which avoid them to move in critical position, and they could move in better locations. This<br />
approach, which dynamically changes the potential field to follow, will be called dynamic.<br />
The algorithm works as follows: every agent sends a message to N randomly chosen<br />
C i ∈ C, in which they request help maintaining the network. The information is packed in<br />
a token, τ = {x, c, explorer/communicator}. This could be accepted by the other agents or<br />
resent.<br />
By a intelligent choice based on the information on the token, this dynamic token potential<br />
field approach can overcome the problems of the standard approach.<br />
A simple example can show the token algorithm features and better performance over the<br />
standard approach. Let the situation be the one in Figure 5.3 (left), and let the communica-<br />
tors be in equilibrium, i.e., at minima in the potential field. If the S1 moves right and S2 up,<br />
in the standard approach, C 1 tries to follow both sensors breaking the network; moreover C 2<br />
and C 3 repulse each other and they can not help C 1 . In the dynamic approach C 1 follows<br />
S 1 and informs C 2 , which moves to help it maintain the network.<br />
The choice of what tokens to send and, moreover, what to do with them, is made by<br />
defining a policy. Three different policies has been defined, namely C policy (connectivity),<br />
TC policy (threshold connectivity) and RC policy (resend connectivity). In following sections<br />
each policy will be described.<br />
Each communicator follows the algorithm shown in Algorithm 3. For every time step,<br />
they form a token τ = {x, |C|, communicator} (Algorithm 3 line 1), then they keep sending
5.4. POSITIONING ALGORITHM 47<br />
H<br />
C 3<br />
C 2<br />
C 1<br />
Initial situation<br />
S 1<br />
S 2<br />
Standard<br />
H<br />
H<br />
C 2<br />
Token Algorithm<br />
Figure 5.3: Standard vs. Token algorithm behavior in an example. S1 and S2 are moving in<br />
opposite directions causing the network break in the former but not in the latter.<br />
and receiving tokens imax times and they group them in T . imax is fixed by the policy, if<br />
imax = 1 they simply delete the tokens they do not use. The determination of which tokens<br />
are important is made using a policy (Algorithm 3 line 12 - 16), which forms � S, � C and, if<br />
imax > 1, the token τ to be resent. Then they compute the potential function and move<br />
(Algorithm 3 line 18 - 19).<br />
C policy<br />
Agents with a low local connectivity are in the most critical positions of the network, thus<br />
they need more help. For that reason, C policy is based on local connectivity encouraging<br />
communicators to keep the tokens of low connected agents and move toward them. It works<br />
as follows: let NR be the number of received tokens, the communicator sorts the received NR<br />
tokens so that the first has the lowest c and the last the highest (Algorithm 4 - line 3), then<br />
it determines the subsets � S and � C using M ≤ NR agents, which the first M tokens refer to<br />
(Algorithm 4 - lines 5 - 11). Then it deletes the remaining tokens since for C policy imax = 1.<br />
TC policy<br />
The policy is the same of C policy, but includes a criterion for determine whether a com-<br />
municator is useful or not. Since each agent can not know if it is on a minimal path, this<br />
criterion is based on the number of received token, NR. If this is very high it means that<br />
the communicator is very useful, since a lot of agents request its help; if there was only a<br />
C 3<br />
C 3<br />
C 2<br />
C 1<br />
C 1<br />
S 2<br />
S 2<br />
S 1<br />
S 1
48 CHAPTER 5. THE COMMUNICATION PART<br />
Algorithm 3 Token Algorithm<br />
1: τ = {x, |C|, communicator}<br />
2: imax ← Policy<br />
3: for i = 0 to imax do<br />
4: for k = 0 to N do<br />
5: Send(τ) → Random(C r ∈ C)<br />
6: end for<br />
7: T = {∅}<br />
8: repeat<br />
9: TA =GetToken ← (C r ∈ C ∨ S r ∈ S)<br />
10: T = T ∪ TA<br />
11: until (No more messages)<br />
12: if imax > 1 then<br />
13: ( � S, � C, τ) ← Policy(T )<br />
14: else<br />
15: ( � S, � C) ← Policy(T )<br />
16: end if<br />
17: end for<br />
18: J = J ( � S, � C)<br />
19: x ← x + (∇xJ ) dx<br />
Algorithm 4 C policy<br />
1: imax = 1<br />
2: NR ← |T |<br />
3: Sort(T ): c(T1) ≤ c(TNR )<br />
4: � S = {∅}, � C = {∅}<br />
5: for j = 1 to M ≤ NR do<br />
6: if communicator(Tj) = 0 then<br />
7: S � = { S, � S(Tj)}<br />
8: else<br />
9: C � = { C, � C(Tj)}<br />
10: end if<br />
11: end for
5.5. POTENTIAL FIELD AND MOTION CONTROL 49<br />
agent connected to it, it would receive at least N tokens. On the other hand, if few tokens<br />
are received, the communicator is in an useless position.<br />
Therefore, when a comm receives less tokens than a specified threshold, it computes a<br />
different potential function where the repulsive part is neglected. This is done to allow that<br />
communicator to move closer to other communicators to reach, eventually, critical location<br />
in the network.<br />
The threshold can be different for explorers and communicators, in the sense that the<br />
repulsive part of the � S subset is neglected if the number of received tokens NR is less than<br />
TS, whereas the one of the � C subset is neglected if the number of tokens is less than TC.<br />
Thus the potential function is<br />
�Jj( � S, � C, NR) = α �<br />
Si∈ � S<br />
Algorithm 5 TC policy<br />
1: C - policy(from line 1 to line 11)<br />
2: Change J to � J ( � S, � C, NR)<br />
RC policy<br />
� �6 � �12 dcfs<br />
dcfs<br />
−2 + (tokens ≥ TS)<br />
+<br />
rCj Si<br />
rCj Si<br />
β �<br />
� �6 � �12 dcfc<br />
dcfc<br />
−2 + (tokens ≥ TC)<br />
rCj Cq rCj Cq C q ∈ � C<br />
The policy is the same of C policy, but imax > 1, thus, after communicator determined � S<br />
(5.3)<br />
and � C it resends some of the tokens it did not use. It keeps on sending and receiving tokens,<br />
until i = imax, when this condition is satisfied, it computes the potential field using the last<br />
determined � S and � C. This allows to better determine � S and � C, since it spreads the infor-<br />
mation of where the low local connectivity areas are. In fact, in low local connectivity areas<br />
communicators could be not sufficient to satisfy all the helping requests, while in high local<br />
connectivity ones, communicators are less useful. By passing the not used tokens through<br />
the network this can be avoided making less useful communicators move where it is needed.<br />
The choice of which tokens have to be resent is made by the connectivity value. In<br />
particular, each communicator forms a new token, which carries not only its information, but<br />
also Q tokens, from M + 1 to M + Q (Algorithm 6 line 13 - 15).<br />
5.5 Potential Field and Motion Control<br />
In the previous sections the shaping of suitable potential fields have dictated where each<br />
spacecraft has to move, which is towards the minimum of their local potential field. This<br />
means, that giving a fixed spatial step at every time instant, each agent knows the location<br />
in space which has to be reach. The key issue is, then, how to actually reach that location,<br />
since the environment will be perturbed and the model/sensors will be affected by errors.
50 CHAPTER 5. THE COMMUNICATION PART<br />
Algorithm 6 RC policy<br />
1: imax = number > 1<br />
2: NR ← |T |<br />
3: Sort(T ): c(T1) ≤ c(TNR )<br />
4: � S = {∅}, � C = {∅}<br />
5: for j = 1 to M ≤ NR do<br />
6: if communicator(Tj) = 0 then<br />
7: S � = { S, � S(Tj)}<br />
8: else<br />
9: C � = { C, � C(Tj)}<br />
10: end if<br />
11: end for<br />
12: τ = {x, |C|, communicator}<br />
13: for j = M + 1 to Q ≤ NR − M do<br />
14: τ = {τ, x(Tj), c(Tj), communicator/explorer(Tj)}<br />
15: end for<br />
Although this control will not develop here, it seems reasonable that a suitable feedback<br />
robust control can be an answer, since the fixed spatial step is usually small compared to the<br />
global dynamics. That feedback control will be not different from a relative motion control<br />
in the fashion of [MY07], among the others.
CHAPTER<br />
6<br />
PERTURBATION MODELS<br />
The true felicity of life is to be free from anxiety and perturbations<br />
to understand and do our duties to God and man, and to enjoy the present without any<br />
serious dependence on the future<br />
Lucius Annaeus Seneca (4 BC - 65)<br />
Here the perturbation models are analyzed and derived, both for Earth environment, drag,<br />
J2, J22, and for Asteroid main belt.<br />
6.1 Formation Flying<br />
The Prisma mission involves sun-synchronous orbits, it involves small satellites and a 700 km<br />
altitude perigee. For these reasons the significative perturbations are<br />
� atmospheric drag effect;<br />
� gravitational effects, namely J2 and J22.<br />
6.1.1 Atmospheric drag effect<br />
The acceleration due to atmospheric drag, aD, can be compute as [PS06]<br />
aD = − 1 S<br />
ρ<br />
2 m CDv 2 ˆv (6.1)<br />
where ρ is the density, S is the surface, CD is the drag coefficient, m the mass, v the velocity<br />
modulus and ˆv is the velocity unit vector.<br />
51
52 CHAPTER 6. PERTURBATION MODELS<br />
6.1.2 Gravitational effects, J2 and J22<br />
The external geo-potential function at any point P specified by the spherical coordinates<br />
(r, δ, λ) can be expressed as[Cho02], [PS06]<br />
U = − κ<br />
�<br />
1 −<br />
r<br />
∞�<br />
n=2<br />
Jn<br />
� �n RE<br />
r<br />
Pn(sin δ) +<br />
∞�<br />
n�<br />
n=2 m=1<br />
Jnm<br />
� � �<br />
n<br />
RE<br />
Pnm(sin δ) cos m(λ − λT )<br />
r<br />
where κ is the planetary constant, RE is the Earth radius, and Pi, Pij suitable spherical<br />
functions.<br />
For the J2 term, considering the spherical triangle ABC,<br />
C<br />
A<br />
λ<br />
θ<br />
i<br />
Figure 6.1: Spherical Triangle.<br />
sin δ = sin i sin θ<br />
Thus substituting in the geo-potential function and calculating the partial derivatives in the<br />
orbital frame<br />
fr = ∂U<br />
∂r<br />
fθ = 1 ∂U<br />
r ∂θ<br />
R<br />
= −3κJ2<br />
2 E<br />
r4 B<br />
δ<br />
� 2 2 �<br />
1 − 3 sin i sin θ<br />
R<br />
= −3κJ2<br />
2 E<br />
r4 sin2 i sin θ cos θ<br />
fh = 1 ∂U<br />
r sin θ ∂i<br />
= −3κJ2<br />
r4 sin i cos i sin θ<br />
As regards J22, the term which needs to be written is<br />
R 2 E<br />
P22(sin δ) cos 2(λ − λT ) = 3(1 − sin 2 δ) cos 2(λ − λT )<br />
2<br />
(6.2)<br />
(6.3)
6.1. FORMATION FLYING 53<br />
Considering the spherical triangles ABC:<br />
Thus<br />
The second part can be written as<br />
but<br />
sin δ = sin i sin θ<br />
sin λ =<br />
cos i sin θ<br />
cos δ<br />
P22(sin δ) cos 2(λ − λT ) = 3(1 − sin 2 i sin 2 θ) cos 2(λ − λT )<br />
cos 2(λ − λT ) = 1 − 2 sin 2 (λ − λT ) = 1 − 2(sin λ cos λT − cos λ sin λT ) 2 =<br />
= 1 − 2(sin 2 λ cos 2 λT + cos 2 λ sin 2 λT − 2 sin λ cos λ sin λT cos λT )<br />
sin 2 λ = cos2 i sin 2 θ<br />
1 − sin 2 i sin 2 θ<br />
and, by the spherical triangles ABC<br />
Hence<br />
cos 2(λ − λT ) =<br />
or also<br />
thus<br />
cos 2(λ − λT ) =<br />
sin λ cos λ =<br />
cos i sin θ<br />
cos δ<br />
cos θ<br />
cos δ<br />
= cos i sin θ cos θ<br />
1 − sin 2 i sin 2 θ<br />
= 1 − 2(sin 2 λ cos 2 λT + cos 2 λ sin 2 λT − 2 sin λ cos λ sin λT cos λT ) =<br />
= 1 −<br />
= 1 − 2(sin 2 λ cos 2 2λT + sin 2 λT − sin λ cos λ sin 2λT ) =<br />
= 1 − 2<br />
2<br />
1 − sin 2 i sin 2 θ<br />
P22(sin δ) cos 2(λ − λT ) =<br />
� cos 2 i sin 2 θ<br />
1 − sin 2 i sin 2 θ cos2 2λT + sin 2 λT −<br />
cos i sin θ cos θ<br />
1 − sin2 i sin2 sin 2λT<br />
θ<br />
� cos 2 i sin 2 θ cos 2 2λT + sin 2 λT (1 − sin 2 i sin 2 θ)<br />
�<br />
− cos i sin θ cos θ sin 2λT )<br />
= 3(1 − sin 2 i sin 2 θ − 2 � cos 2 i sin 2 θ cos 2 2λT + sin 2 λT (1 − sin 2 i sin 2 θ)<br />
Rearranging the similar terms<br />
P22(sin δ) cos 2(λ − λT ) =<br />
− cos i sin θ cos θ sin 2λT ))<br />
= 3(1−2 sin 2 λT −sin 2 i sin 2 θ(1−2 cos 2 2λT −2 sin 2 λT )−2 sin 2 θ cos 2 2λT +2 cos i sin θ cos θ sin 2λT ) =<br />
= 3(c1 + c2 sin 2 i sin 2 θ + c3 sin 2 θ + c4 cos i sin θ cos θ)
54 CHAPTER 6. PERTURBATION MODELS<br />
Thus, in the orbital frame<br />
fr = ∂U<br />
∂r<br />
fθ = 1 ∂U<br />
r ∂θ<br />
fh = 1 ∂U<br />
r sin θ ∂i<br />
6.2 Asteroid belt<br />
R<br />
= 3κJ22<br />
2 E<br />
r4 3(c1 + c2 sin 2 i sin 2 θ + c3 sin 2 θ + c4 cos i sin θ cos θ)<br />
R<br />
= −κJ22<br />
2 E<br />
r4 3(2c2 sin 2 i sin θ cos θ + 2c3 sin θ cos θ + c4 cos i cos 2θ)<br />
R<br />
= −κJ22<br />
2 E<br />
r4 3(2c2 sin i cos i sin θ − c4 sin i cos θ)<br />
(6.4)<br />
The perturbation model for the asteroid belt can be derived considering the perturbative<br />
gravitational forces of the most massive objects. As shown in chapter 2 this model will<br />
include the most massive 10 asteroids. Thus the perturbative field in each point x induced<br />
by each asteroid i can be written as<br />
w(x) = − �<br />
i<br />
mi<br />
||x − xi|| 3 (x − xi) (6.5)<br />
The w has to be written in the orbital frame with a suitable rotation matrix.<br />
In Figure 6.2, the orbit of the selected asteroids.<br />
1<br />
0<br />
−1<br />
−6<br />
−4<br />
−2<br />
0<br />
2<br />
4<br />
Figure 6.2: First ten asteroid orbits, dimensions in AU.<br />
6<br />
The final distances between spacecrafts and asteroids will be in the order of 0.01 AU,<br />
because below that distance it is more convenient a feedback linearized approach for the<br />
control; therefore the maximum expected perturbation is in the order of 10 −8 m/s 2 , which<br />
can be handle easily with solar sail propulsion ∼ 10 −6 m/s 2 , as in [1].<br />
Since the uncertainties on the masses are quite important, in closer rendezvous, a method<br />
to handle this problem has to be used and applied in order to make a correct valuation of the<br />
−4<br />
−2<br />
0<br />
2<br />
4
6.2. ASTEROID BELT 55<br />
maximum perturbative force, thus the correct design of the control law. The Taylor series<br />
method of [Ber99], [Fer08] could be chosen.
56 CHAPTER 6. PERTURBATION MODELS
CHAPTER<br />
7<br />
RESULTS<br />
Telling the future by looking at the past assumes that conditions remain constant.<br />
This is like driving a car by looking in the rearview mirror.<br />
Herb Brody (1957 - )<br />
The main results are here shown. First, the goal manager behavior in the example pre-<br />
sented in chapter 3; second the non linear Lyapunov control in several tests in the framework<br />
of Prisma mission. Third the communication network deployment in two significative sce-<br />
nario; finally an asteroid belt example, in which the features of the whole control algorithm<br />
are shown.<br />
7.1 Introduction<br />
In this chapter the main results are presented, in particular<br />
� a goal manager simulation for the example shown in chapter 3; here the fully scalable<br />
behavior of task assignment is shown and the sharing information mechanism tested;<br />
� several tests for the non linear Lyapunov controller in the framework of Prisma mis-<br />
sion; here the algorithm robustness is proved using, among the others, a Montecarlo<br />
simulation;<br />
� two wall scenario, 2D and 3D, for the communication network deployment; here the<br />
behavior of token based algorithm is shown and proved to out-perform the standard<br />
one;<br />
� an asteroid belt scenario, in which the controls are coupled and shown.<br />
57
58 CHAPTER 7. RESULTS<br />
7.1.1 Scalability proof<br />
First of all the global agent algorithm has to be proved to be scalable, thus it has to be<br />
proved that the computational time for each agent does not increase increasing the agent<br />
number. This is quite straightforward, since the single pieces of the algorithm are completely<br />
distributed, hence the agent number do not affect the computation time. It has to be note<br />
that the communication and information sharing mechanism, in particular the number of<br />
messages each agent receives depends on the environment density and not directly on the<br />
number of agents. Since the scenario environments are supposed to be sparse, there is no<br />
scalability issue and thus the global algorithm is scalable.<br />
7.2 Goal Manager example<br />
As an example of the properties of the Goal Manager, the example of section 3.4.1, will be<br />
exploited. Let the environment be closed in a box, [−1, 1] × [−1, 1] × [−1, 1], as in Figure<br />
7.1, and let the red dots be three different asteroids, with a fixed initial T ∈ R 3×3 matrix,<br />
where each tij = 1/3. Let N be the agent number randomly located in the box and let C be<br />
a R N×3 random capacities matrix. C is normalized thus the global team capacities are equal<br />
to 1. Let dc = 1 be the communication/sensing range among the agents. Moreover let the<br />
agents move towards the chosen asteroid using straight lines.<br />
As a performance index can be chosen the final T matrix on the initial one, which repre-<br />
sents how much the goals have been satisfied. In Figure 7.1, the dynamical behavior of a set<br />
of N = 50 agents is depicted; the agents are the black dots, while the gray dashed lines are<br />
the communication links.<br />
In Figure 7.2, the first column of the final T on the initial one, is shown for 100 simulation<br />
runs. This represents how well the Goal Manager can assign a target to the agents, the lower<br />
the values the better the Goal Manager is; in particular a positive tij/(tij)0 means that the<br />
goal still needs tij/(tij)0 capacity to be fulfilled, whereas a negative tij/(tij)0 results in a<br />
excess of capacity. The choice of the first column does not affect the results evaluation, since<br />
every column is almost the same starting with a random choice on C. In Figure 7.2, the line<br />
thickness represents the results standard deviation.<br />
The shown results assure a good Goal Manager performance for N, which can fulfill the<br />
tasks within a 10% errors. This can be acceptable, since in the real case, the agents may<br />
have more capacities than necessary. Moreover, it has to be noted that the agent number<br />
is not significant per se but it is has to be compared to the initial connectivity, i.e. the<br />
communication distance. Few agents with a long communication distance could perform<br />
better than lot of agents with a very limited communication range.
7.2. GOAL MANAGER EXAMPLE 59<br />
1<br />
0.5<br />
0<br />
−0.5<br />
−1<br />
1<br />
0.5<br />
0<br />
−0.5<br />
−1<br />
1<br />
0.5<br />
0<br />
−0.5<br />
−1<br />
1<br />
1<br />
1<br />
0.5<br />
0.5<br />
0.5<br />
0<br />
0<br />
0<br />
−0.5<br />
−0.5<br />
−0.5<br />
−1<br />
−1<br />
(a) τ = 1<br />
−1<br />
−1<br />
(c) τ = 5<br />
−1<br />
−1<br />
(e) τ = 9<br />
−0.5<br />
−0.5<br />
−0.5<br />
0<br />
0<br />
0<br />
0.5<br />
0.5<br />
0.5<br />
1<br />
1<br />
1<br />
1<br />
0.5<br />
0<br />
−0.5<br />
−1<br />
1<br />
0.5<br />
0<br />
−0.5<br />
−1<br />
1<br />
0.5<br />
0<br />
−0.5<br />
−1<br />
1<br />
1<br />
1<br />
0.5<br />
0.5<br />
0.5<br />
0<br />
0<br />
0<br />
−0.5<br />
−0.5<br />
−0.5<br />
−1<br />
−1<br />
(b) τ = 3<br />
−1<br />
−1<br />
(d) τ = 7<br />
−1<br />
−1<br />
(f) τ = 11<br />
Figure 7.1: Dynamical behavior of a set of N = 50 agents: the agents are the black dots, the<br />
asteroids are the red dots, while the gray dashed lines are the communication links.<br />
−0.5<br />
−0.5<br />
−0.5<br />
0<br />
0<br />
0<br />
0.5<br />
0.5<br />
0.5<br />
1<br />
1<br />
1
60 CHAPTER 7. RESULTS<br />
t ij /(t ij ) 0 value<br />
0.25<br />
0.2<br />
0.15<br />
0.1<br />
0.05<br />
0<br />
−0.05<br />
−0.1<br />
−0.15<br />
−0.2<br />
−0.25<br />
Goal Manager performances<br />
First Asteroid<br />
Second Asteroid<br />
Third Asteroid<br />
30 40 50 60 70 80 90 100<br />
Agent number, N<br />
Figure 7.2: Goal Manager performances, the line thickness represents the results standard<br />
deviation.<br />
7.3 Formation flying scenarios<br />
The Prisma mission has been used to test the physical part of the distributed algorithm, in<br />
particular it has been supposed that the Target spacecraft has to rendezvous with the Main<br />
one in operative orbit, which is the real case, from the initial orbit in which it is left by the<br />
launcher. In fact, the spacecrafts have impulsive rocket motors, but here low thrusters have<br />
been considered. At the beginning the nominal Main orbital parameters are [GM07]<br />
where RE is the Earth radius.<br />
ā ē ī ¯ω ¯ Ω ¯ θ<br />
RE+ 700 km 0 98.2π/180 0 0 0<br />
Table 7.1: Main initial orbital parameters.<br />
7.3.1 Unperturbed and perturbed results<br />
Three cases have been tested, Table 7.2<br />
The simulations parameters are: R = 1E − 4, ∆tI = 60 s, ∆tC = 60 s, where I and<br />
C mean Integration time step and Control time step, Q = diag{1/a 2 e, 1, 1, 10, 10, 1}, with<br />
ae = (aT + aM)/2, being aT the Target satellite semiaxis and aM the Main one.<br />
The drag due to the atmosphere and gravitational perturbations due to J2 and J22 have
7.3. FORMATION FLYING SCENARIOS 61<br />
a e i ω Ω θ<br />
1 st test – RE+ 600 km ē ī ¯ω ¯ Ω π/2<br />
2 nd test – – RE+ 500 km ē ī ¯ω ¯ Ω π/2<br />
3 rd test – • RE+ 400 km ē ī ¯ω ¯ Ω π/2<br />
Table 7.2: Tests initial conditions.<br />
been considered and since the actual orbit of Main has to be a sun-synchronous, the precession<br />
of the ascending node has not been controlled.<br />
Since the perturbation magnitude is ∼ 1 mm/s 2 and the control should be at least ∼ 1<br />
cm/s 2 , for a 100 Kg spacecraft, the use of low thrust electrical motors is, in fact, unpractical<br />
and perhaps infeasible. Here, only the control performances in terms of relative distances<br />
and Lyapunov function have been taken into account, leaving the thruster detailed design for<br />
further developments.<br />
In Figure 7.3, the unperturbed relative distances – in-track/cross-track/radial – are shown<br />
for the different tests. The chosen time interval is t ∈ [18, 21]h. While in Figure 7.4, the<br />
perturbed relative distances – in-track/cross-track/radial – are shown for the different tests.<br />
The chosen time interval is t ∈ [18, 21]h.<br />
From the graphs appear that the control can lead the Target spacecraft to a close approach<br />
with the Main spacecraft. The relative distance is within less than 5 km and this is reasonable<br />
to start a close rendezvous approach via linear feedback control.<br />
In Figure 7.5 the variation of Normalized Lyapunov functions for the unperturbed and<br />
perturbed case in the third test. The two curve cannot be distinguished. The normalization<br />
is made on the initial value.<br />
7.3.2 Montecarlo analysis<br />
As a final test on the algorithm quality a Monte Carlo analysis has been performed using, as<br />
state space, the uncertainties on the initial position of the Target spacecraft as<br />
a e i ω Ω θ<br />
RE+ 700km ± 100 km ē + 0.1 ī ± 10 ◦ ¯ω ± 10 ◦ ¯ Ω ± 10 ◦ π/2 ± 10 ◦<br />
Table 7.3: Monte Carlo initial condition intervals.<br />
In Figure 7.6 the relative distance results in the time frame t ∈ [20, 21]h. There, to make<br />
the graphs readable, only discrete instant trajectories are shown; for each simulation, the<br />
continuous trajectory would start from the top and it would stop at the bottom, as the one<br />
shown in black. Thus each simulated trajectory passes through the 0 km in track point. The<br />
results show a good performance of the control algorithm, allowing a close approach of less<br />
than 5 km.
62 CHAPTER 7. RESULTS<br />
Cross track [km]<br />
Radial [km]<br />
0.4<br />
0.3<br />
0.2<br />
0.1<br />
0<br />
−0.1<br />
−0.2<br />
Relative unperturbed distance for t ∈ [18, 21] h<br />
−0.3<br />
25 30 35 40 45<br />
In track [km]<br />
50 55 60 65<br />
−0.2<br />
−0.25<br />
−0.3<br />
−0.35<br />
−0.4<br />
−0.45<br />
−0.5<br />
Relative unperturbed distance for t ∈ [18, 21] h<br />
−0.55<br />
25 30 35 40 45<br />
In track [km]<br />
50 55 60 65<br />
Figure 7.3: Results: Close approach of the spacecrafts for different initial altitude: – 600 km,<br />
– – 500 km, – • 400 km, unperturbed case.
7.3. FORMATION FLYING SCENARIOS 63<br />
Cross track [km]<br />
Radial [km]<br />
2<br />
1.5<br />
1<br />
0.5<br />
0<br />
−0.5<br />
−1<br />
−1.5<br />
Relative perturbed distance for t ∈ [18, 21] h<br />
−2<br />
−100 −50 0<br />
In track [km]<br />
50 100<br />
2<br />
1.5<br />
1<br />
0.5<br />
0<br />
−0.5<br />
−1<br />
Relative perturbed distance for t ∈ [18, 21] h<br />
−1.5<br />
−100 −50 0<br />
In track [km]<br />
50 100<br />
Figure 7.4: Results: Close approach of the spacecrafts for different initial altitude: – 600 km,<br />
– – 500 km, – • 400 km, perturbed case.
64 CHAPTER 7. RESULTS<br />
Normalized Lyapunov function J<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
0.4<br />
0.3<br />
0.2<br />
0.1<br />
0<br />
0 5 10 15 20 25<br />
Time [h]<br />
Figure 7.5: Normalized Lyapunov Function
7.3. FORMATION FLYING SCENARIOS 65<br />
Cross track [km]<br />
Radial [km]<br />
2<br />
1.5<br />
1<br />
0.5<br />
0<br />
−0.5<br />
−1<br />
Relative distance for t ∈ [20, 21] h<br />
−1.5<br />
−40 −20 0 20 40<br />
In track [km]<br />
60 80 100 120<br />
2<br />
1.5<br />
1<br />
0.5<br />
0<br />
−0.5<br />
−1<br />
Relative distance for t ∈ [20, 21] h<br />
−1.5<br />
−40 −20 0 20 40<br />
In track [km]<br />
60 80 100 120<br />
Figure 7.6: Relative positions reached after 20 h. The black line is a representative trajectory.
66 CHAPTER 7. RESULTS<br />
7.4 Communication network deployment<br />
Two main scenarios have been analyzed to test the performance of the algorithm, namely<br />
� a 2D wall scenario;<br />
� a 3D wall scenario.<br />
Other, more complex, scenarios can be found in [SS08].<br />
7.4.1 2D Environment<br />
This apparently simple wall scenario tests the basic functionality of the algorithms. If the<br />
communicators do not move north of the wall, connectivity will be lost. Thus, spreading out<br />
is an inadequate strategy.<br />
The initial and a final state are shown in Figure 7.4.1, notice the explorers, black dots,<br />
move from west to east. In Figures 7.8 and 7.9 the obtained average results for 200 simulation<br />
runs are presented. The parameters for the experiments were:<br />
� for the scenario: |S| = 6, dc = 10, v = 0.016;<br />
� for the potential function: fs = 1/2, fc = 1/2, α = 100 |C|/|S|, β = 1;<br />
� for the policies: N = 2, (M, Q) = ([NR/2], 0) for C and TC policies, N = 2, (M, Q) =<br />
(1, 1) for RC policy, TS = 4, TC = 3, imax = 3.<br />
In the graphs B means Baseline, C C policy and so on. Bars are the value of average<br />
global connectivity of average efficiency. Black bars on the top of the bars are the standard<br />
deviations. Lines in the bars are the final value of K.<br />
(a) Initial State (b) Final State<br />
Figure 7.7: Simple wall simulations: black dots are explorers, white dots the communicators,<br />
dashed lines communication links, the triangle the hub. The box ticks distance is 5.<br />
Both < K > and < E > are significantly higher using the token algorithm than in the<br />
baseline. Moreover, note that in the |C| = 8, case the baseline can not assure a final global<br />
connectivity greater than one, while the other algorithms can. TC policy performs best<br />
here, which was expected since it makes communicators move in the north side of the wall<br />
neglecting the repulsive force of communicators already there. This was less decisive for<br />
14 communicators because with an increasing number of communicators resending is more<br />
important since it allows a better understand of critical locations.
7.4. COMMUNICATION NETWORK DEPLOYMENT 67<br />
Average Global Connectivity<br />
8<br />
7<br />
6<br />
5<br />
4<br />
3<br />
2<br />
1<br />
0<br />
B C TC RC B C TC RC<br />
|C| = 8 |C| = 14<br />
Figure 7.8: < K > for different |C|: B means Baseline, C C - policy and so on. Black bars<br />
on the top of the bars are the average standard deviations, whereas black lines in the bars<br />
are the final value of K.<br />
Average Efficiency<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
0.4<br />
0.3<br />
0.2<br />
0.1<br />
0<br />
B C TC RC B C TC RC<br />
|C| = 8 |C| = 14<br />
Figure 7.9: < E > for different |C|: B means Baseline, C C - policy and so on. Black bars<br />
on the top of the bars are the average standard deviations.
68 CHAPTER 7. RESULTS<br />
7.4.2 3D Environment<br />
This 3D wall scenario tests the basic functionality of the algorithms in an environment closer<br />
to the real one, since the wall could be seen as an asteroid. Moreover to test code robustness,<br />
agents were disabled at a rate, kR. The initial and a final state are shown in Figure 7.4.2,<br />
notice the explorers, black dots, move from west to east. In Figures 7.11 and 7.12 the obtained<br />
average results for 50 simulation runs are presented. The parameters for the experiments were:<br />
� for the scenario: |S| = 3, |C| = 12, dc = 10, v = 0.02;<br />
� for the potential function: fs = 1/2, fc = 1/2, α = 100 |C|/|S|, β = 1;<br />
� for the policies: N = 2, (M, Q) = ([NR/2], 0) for C policy, N = 2, (M, Q) = (1, 1) for<br />
RC policy, imax = 3.<br />
In the graphs B means Baseline, C C policy and so on. Bars are the value of average<br />
global connectivity of average efficiency. Black bars on the top of the bars are the standard<br />
deviations. Lines in the bars are the final value of K.<br />
10<br />
5<br />
0<br />
−5<br />
−10<br />
−10<br />
−5<br />
0<br />
5<br />
10 −10<br />
(a) Initial State<br />
−5<br />
0<br />
5<br />
10<br />
10<br />
5<br />
0<br />
−5<br />
−10<br />
−10<br />
−5<br />
0<br />
5<br />
10 −10<br />
(b) Final State<br />
Figure 7.10: 3D wall simulations: black dots are explorers, white dots the communicators,<br />
dashed lines communication links, the triangle the hub.<br />
Both < K > and < E > are significantly higher using the token algorithm than in the<br />
baseline. Moreover, note that the baseline can not assure a final global connectivity greater<br />
than one, while the other algorithms can. The code robustness is proven.<br />
−5<br />
0<br />
5<br />
10
7.4. COMMUNICATION NETWORK DEPLOYMENT 69<br />
Average Global Connectivity<br />
10<br />
9<br />
8<br />
7<br />
6<br />
5<br />
4<br />
3<br />
2<br />
1<br />
0<br />
B C RC B C RC B C RC<br />
Kill rate = 0.00 Kill rate = 0.16 Kill rate = 0.33<br />
Figure 7.11: < K > for different |C|: B means Baseline, C C - policy and so on. Black bars<br />
on the top of the bars are the average standard deviations, whereas black lines in the bars<br />
are the final value of K.<br />
Average Efficiency<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
0.4<br />
0.3<br />
0.2<br />
0.1<br />
0<br />
B C RC B C RC B C RC<br />
Kill rate = 0.00 Kill rate = 0.16 Kill rate = 0.33<br />
Figure 7.12: < E > for different |C|: B means Baseline, C C - policy and so on. Black bars<br />
on the top of the bars are the average standard deviations.
70 CHAPTER 7. RESULTS<br />
7.5 Asteroid belt scenario<br />
An Asteroid belt scenario has been developed to test both the single controls’ performance<br />
and how the two spacecraft types behave together. The scenario consists in three different<br />
explorers which have to follow two asteroids, one communicator Hub and a set of communi-<br />
cators; the initial data for the explorers, hub and asteroids are summarized in Table 7.5.<br />
a [AU] e [ ◦ ] i [ ◦ ] ω [ ◦ ] Ω [ ◦ ] θ [ ◦ ] Target<br />
1 st explorer 2.75 0 2 0 0 0 1 st asteroid<br />
2 nd explorer 2.8 0 2 0 0 0 1 st asteroid<br />
3 rd explorer 2.775 0 2 0 0 0 2 nd asteroid<br />
Hub 2.8 0 2 0 0 0 target orbit<br />
1 st asteroid 2.85 0 2 0 0 -2<br />
2 nd asteroid 2.8 0 2 0 0 -2<br />
target orbit 2.9 0 2 0 0 -2<br />
It has been assumed that<br />
Table 7.4: Explorers/Hub/Asteroids initial conditions.<br />
� the targets were pre-imposed by the goal manager and they were minor asteroids, here<br />
arbitrarily located, as in [1];<br />
� the spacecrafts had a mass of 1 kg and solar sail propulsion system as in [1];<br />
� the time frame was 3 years as in [2].<br />
7.5.1 The physical part<br />
To obtain smoother control results R is considered as a function of the state as<br />
�<br />
||δx||<br />
R = αRI3<br />
||δx0||<br />
where δx0 is the initial value of δx. The other simulations parameters are: αR = 0.25,∆tI =<br />
1 day, ∆tC = 1 day, where I and C mean Integration time step and Control time step,<br />
Q = diag{1/a 2 e, 1, 1, 1, 1, 1}, with ae = (aT + aM)/2, being aT the Target orbit semiaxis and<br />
aM the spacecraft one.<br />
The results are depicted in Figures 7.13 - 7.14.<br />
In Figures 7.15 - 7.16 the control behavior for the analyzed spacecrafts. Note that the<br />
control effort is almost constant and, moreover, always positive for fr, which is a necessary<br />
condition for solar sail propulsion. The sail area with this control is ∼ 20 m 2 , which is in the<br />
order of the one proposed by NASA, [1].
7.5. ASTEROID BELT SCENARIO 71<br />
z [AU]<br />
2<br />
Asteroid<br />
Hub<br />
Explorers<br />
1<br />
y [AU]<br />
0<br />
−1<br />
−2<br />
−2<br />
−1<br />
0<br />
x [AU]<br />
Figure 7.13: Trajectory of Asteroids/Hub/Explorers in a sun-centered inertial reference<br />
frame. Asteroids in blue, Hub in red, Explorers in black.<br />
y [AU]<br />
0.05<br />
0<br />
−0.05<br />
−0.1<br />
−0.15<br />
−0.2<br />
−0.25<br />
−0.3<br />
−0.35<br />
Relative positions from Hub [AU]<br />
−0.1 0 0.1 0.2<br />
x [AU]<br />
0.3 0.4<br />
Figure 7.14: Trajectory of Asteroids/Hub/Explorers in a Hub-centered inertial reference<br />
frame. Asteroids in blue, Hub in red, Explorers in black.<br />
1<br />
2
72 CHAPTER 7. RESULTS<br />
Control [nN]<br />
Control [nN]<br />
10<br />
8<br />
6<br />
4<br />
2<br />
0<br />
10<br />
8<br />
6<br />
4<br />
2<br />
0<br />
HUB Control<br />
0 0.5 1 1.5<br />
Time [y]<br />
2 2.5 3<br />
0 0.5 1 1.5<br />
Time [y]<br />
2 2.5 3<br />
Normalized Lyapunov Function J<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
0.4<br />
s/c 2 Control<br />
f r<br />
f θ<br />
f h<br />
f r<br />
f θ<br />
f h<br />
Control [nN]<br />
Control [nN]<br />
10<br />
8<br />
6<br />
4<br />
2<br />
0<br />
10<br />
8<br />
6<br />
4<br />
2<br />
0<br />
s/c 1 Control<br />
0 0.5 1 1.5<br />
Time [y]<br />
2 2.5 3<br />
s/c 3 Control<br />
0 0.5 1 1.5<br />
Time [y]<br />
2 2.5 3<br />
Figure 7.15: Control effort for Hub/Explorers.<br />
Normalized Lyapunov Function<br />
J HUB<br />
J s/c 1<br />
J s/c 2<br />
J s/c 3<br />
0 0.5 1 1.5<br />
Time [y]<br />
2 2.5 3<br />
Figure 7.16: Normalized Lyapunov Function for Hub/Explorers.<br />
f r<br />
f θ<br />
f h<br />
f r<br />
f θ<br />
f h
7.5. ASTEROID BELT SCENARIO 73<br />
7.5.2 The communication part<br />
The communicators were initially spread randomly in a box, [−0.05, 0.05]×[−0.05, 0.05]×[0, 0]<br />
AU , around the Hub; in Figure 7.5.2, notice the initial and a final graph configuration. In<br />
Figures 7.18 - 7.19 the obtained average results for 50 simulation runs are presented. The<br />
parameters for the experiments were:<br />
� for the scenario: |S| = 4, |C| = 6 − 10, dc = 0.25 AU, v = 0.8;<br />
� for the potential function: fs = 1/2, fc = 1/2, α = 100 |C|/|S|, β = 1;<br />
� for the policies: N = 2, (M, Q) = ([NR/2], 0) for C policy, N = 2, (M, Q) = (1, 1) for<br />
RC policy, imax = 3.<br />
In the graphs B means Baseline, C C policy and so on. Bars are the value of average<br />
global connectivity of average efficiency. Black bars on the top of the bars are the standard<br />
deviations. Lines in the bars are the final value of K.<br />
y [AU]<br />
0.1<br />
0.05<br />
0<br />
−0.05<br />
−0.1<br />
−0.15<br />
−0.2<br />
−0.25<br />
−0.3<br />
−0.35<br />
−0.4<br />
−0.2 −0.1 0 0.1 0.2<br />
x [AU]<br />
0.3 0.4 0.5 0.6<br />
(a) Initial State<br />
y [AU]<br />
0.1<br />
0.05<br />
0<br />
−0.05<br />
−0.1<br />
−0.15<br />
−0.2<br />
−0.25<br />
−0.3<br />
−0.35<br />
−0.4<br />
−0.2 −0.1 0 0.1 0.2<br />
x [AU]<br />
0.3 0.4 0.5 0.6<br />
(b) Final State<br />
Figure 7.17: Asteroid belt scenario: black dots are explorers, white dots the communicators,<br />
dashed lines communication links, the triangle the hub.<br />
Both < K > and < E > are higher using the token algorithm than in the baseline.<br />
The differences among the algorithms are slight here because the environment is completely<br />
without obstacles, but, even so, only the RC policy can assure a final connectivity greater<br />
then one. The algorithms behavior to a change of |C| depends on the interaction between<br />
Hub/Explorers and Communicators, and it is not monotonic here, because the explorers<br />
relative velocity varies with time. In fact in more clustered environments the communicators<br />
receive less tokens from explorers relatively to those of other communicators, thus they are<br />
less reactive to rapid velocity increase. To avoid such a situation a proactive algorithm could<br />
be used as in [SS08].<br />
As a final remarks on the scenario, a communication distance of dc = 0.25 AU can assure<br />
a data amount in the order of 1 MB per day – e.g. a high resolution image – with a 20 W, 1
74 CHAPTER 7. RESULTS<br />
Average Global Connectivity<br />
8<br />
7<br />
6<br />
5<br />
4<br />
3<br />
2<br />
1<br />
0<br />
B C RC B C RC B C RC<br />
|C| = 6 |C| = 8 |C| = 10<br />
Figure 7.18: < K > for different |C|: B means Baseline, C C - policy and so on. Black bars<br />
on the top of the bars are the average standard deviations, whereas black lines in the bars<br />
are the final value of K.<br />
Average Efficiency<br />
1<br />
0.99<br />
0.98<br />
0.97<br />
0.96<br />
0.95<br />
0.94<br />
0.93<br />
0.92<br />
0.91<br />
0.9<br />
B C RC B C RC B C RC<br />
|C| = 6 |C| = 8 |C| = 10<br />
Figure 7.19: < E > for different |C|: B means Baseline, C C - policy and so on. Black bars<br />
on the top of the bars are the average standard deviations.
7.5. ASTEROID BELT SCENARIO 75<br />
m diameter, Ka antenna. The data rate can be improved considering different Hub trajectory<br />
and thus less communication range.
76 CHAPTER 7. RESULTS
CHAPTER<br />
8<br />
FINAL REMARKS<br />
A positive attitude may not solve all your problems,<br />
but it will annoy enough people to make it worth the effort.<br />
Herm Albright (1876 - 1944)<br />
The conclusions and some remarks are included in this chapter. The future developments<br />
are also outlined.<br />
8.1 Thesis final remarks<br />
The thesis results show good algorithm performances in terms of reliability, robustness and<br />
wide application areas; in particular<br />
� the goal manager can handle fairly complex scenario with multiple tasks and capabili-<br />
ties;<br />
� the non linear Lyapunov control framework is rich enough to be both robust and a good<br />
starting point to tune and optimize the trajectory control;<br />
� in the communication network deployment, the proposed token based algorithm out-<br />
performs the standard one.<br />
8.2 Future developments<br />
8.2.1 Decisional Level<br />
Reactive Goal Manager has been investigating for years in the computer science field, mostly<br />
using potential fields or other techniques. As it appears clear, to better develop the one of<br />
this thesis, several things have to be studied as<br />
77
78 CHAPTER 8. FINAL REMARKS<br />
� the projection operator has to be fully defined in various applicative contexts;<br />
� the conflict detection and solver has to be developed; one way could be the use a<br />
distributed scheduling [BL08];<br />
� the information sharing mechanism has to be improved, perhaps using a token based<br />
approach, and low connected graphs have to be taken into account.<br />
8.2.2 Physical part<br />
The trajectory control, using a long dynamic approach, has been fully characterized, even if<br />
several parameters could be tuned to obtained some sort of optimal behavior. On the other<br />
hand, the short period dynamic control has to be developed, in particular<br />
� the approach has to be decided, focusing on reactive algorithm, like [MY07] and it has<br />
to be extended properly;<br />
� the long and short period have to be integrated.<br />
8.2.3 Communication part<br />
The communication network deployment has been studied and assured using a token based<br />
algorithm. Mainly the following points are open issues<br />
� find the optimal policy tuning the different parameters;<br />
� integrate the feedback control law for motion control;<br />
8.2.4 Missions<br />
Real mission scenarios have to be tested, as<br />
� large formation flight mission, such as TPF, in environment as a Lagrangian point;<br />
� asteroid scenario with goal manager integration.
1 Introduction<br />
REFERENCES<br />
[Bro99] R. A. Brooks. Cambrian Intelligence, The Early History of the New AI. The MIT Press, 1999.<br />
[D’A04] P. D’Arrigo. The APIES Mission. Technical report, ESA, 2004.<br />
[KE01] J. Kennedy and R. C. Eberhart. Swarm Intelligence. The Morgan Kaufmann, 2001.<br />
[LLJB07] P. R. Lawson, O. P. Lay, K. J. Johnston, and C. A. Beich. Terrestrial Planet Finder Interferometer<br />
Science Working Group Report. Technical report, JPL - NASA, 2007.<br />
[RHRT06] C. A. Rouff, M. G. Hinchey, J. L. Rash, and W. Truszkowski. Verifying Future Swarm-Based<br />
Missions. SpaceOps 2006 Conference AIAA 2006-5555, 2006.<br />
[Syc98] K. Sycara. Multiagent systems. AI - Magazine, 1998.<br />
[Woo02] M. Wooldridge. An introduction to MultiAgent Systems. Wiley, 2002.<br />
Web sites<br />
[1] http://www.weblab.dlr.de/rbrt/GpsNav/Prisma/Prisma.html (2008)<br />
[2] http://www.esa.it (2007)<br />
[3] http://planetquest.jpl.nasa.gov/TPF (2007)<br />
[4] http://ants.gsfc.nasa.gov (2007)<br />
2 Formulation of the problem<br />
[BL92] B. Barraquand, J. Langlois and J. Latombe. Numerical Potential Field Techniques for Robot Path<br />
Planning. IEEE Transactions on Systems, Man, and Cybernetics, 22(2), March – April 1992.<br />
[BP03] S. Brueckner and H. V. D. Parunak. Resource-aware exploration of the emergent dynamics of<br />
simulated systems. Proceedings of Autonomous Agents and Multi Agent Systems (AAMAS), pages<br />
781–788, 2003.<br />
[CS00] C. Albee A. Battel S. Brace R. Burdick G. Burr P. Dippoey D. Lavell J. Leising C. MacPherson<br />
D. Menard W. Rose R. Sackheim R. Casani, J. Whetsler and A. Schallenmuller. Report on the<br />
Loss of the Mars Polar Lander and Deep Space 2 Missions. Jet Propulsion Laboratory, California<br />
Institute of Technology, (JPL D-18709), 2000.<br />
[d’I04] M. d’Inverno. The dMARS Architecture: A Specification of the Distributed Multi-Agent Reasoning<br />
System. Autonomous Agents and Multi-Agent Sytems (AAMAS), pages 5–53, 2004.<br />
79
80 REFERENCES<br />
[DM06] E. D’Amico, S. Gill and O. Montenbruck. Relative Orbit Control Design For The Prisma Formation<br />
Flying Mission. AIAA Guidance, Navigation, and Control Conference, August 21–24 2006.<br />
[EK06] G. H. Elkaim and R. J. Kelbley. A Lightweight Formation Control Methodology for a Swarm of<br />
Non-Holonomic Vehicles. IEEE, 2006.<br />
[GC00] S. Ge and Y. Cui. Path Planning for Mobile Robots Using New Potential Functions. 3rd Asian<br />
Control Conference, July 4–7 2000.<br />
[GM07] S. Gill, E. D’Amico and O. Montenbruck. Autonomous Formation Flying for the PRISMA Mission.<br />
Journal of Guidance, Control and Dynamics, 44(3):671 – 681, May – June 2007.<br />
[Hol88] W.M.L. Holcombe. X-Machines as a Basis for System Specification. Software Engineering, 3(2):69–<br />
76, 1988.<br />
[HR01] J. Hinchey, M. Rash and C.A. Rouff. Verification and Validation of Autonomous Systems. NASA,<br />
2001.<br />
[Kin98] J. R. Kiniry. The Specification of Dynamic Distributed Component Systems. Master’s thesis,<br />
California Institute of Technology, 1998.<br />
[LAM02] A. Li, L. Martinoli and Y. S. Abu-Mostafa. Emergent Specialization in Swarm Systems. IDEAL,<br />
LNCS 2412, pages 216–266, 2002.<br />
[Ld95] M. Luck and M. d’Inverno. A Formal Framework for Agency and Autonomy. pages 254–260, 1995.<br />
AAAI Press/MIT Press.<br />
[LJGM05] K. Lerman, C. Jones, A. Galstyan, and M. J. Matari c. Analysis of Dynamic Task Allocation in<br />
Multi-Robot Systems. University of Southern California, Los Angeles, 2005.<br />
[PG05] B. Persson, S. Jacobsson and E. Gill. Prisma - Demonstration Mission For Advanced Rendezvous<br />
And Formation Flying Technologies And Sensors. IAC, 56th International Astronautical Congress,<br />
October 17–21 2005.<br />
[PV97] H. V. D. Parunak and R. Vanderbok. Managing Emergent Behaviour in Distributed Control<br />
Systems. Proceedings of ISA-Tech’97, 1997.<br />
[Rei92] M.B. Reid. Path Planning Using Optically Computed Potential Fields. NASA, Ames, 1992.<br />
[RH05] W. Rash J. Rouff, C.A. Truszkowski and M. Hinchey. A Survey of Formal Methods for Intelligent<br />
Swarms. NASA Goddard Space Flight Center, 2005.<br />
[RHRT06] C. A. Rouff, M. G. Hinchey, J. L. Rash, and W. Truszkowski. Verifying Future Swarm-Based<br />
Missions. SpaceOps 2006 Conference AIAA 2006-5555, 2006.<br />
[Rou00] C. A. Rouff. Experience Using Formal Methods for Specifying a Multi-Agent System. IEEE, 2000.<br />
[Rou06] C. A. Rouff. Agent Technology from a Formal Perspective. NASA Monographs in Systems and<br />
Software Engineering. Springer, 2006.<br />
[RR04] A. Hinchey M. Truszkowski W. Rouff, C. A. Vanderbilt and J. Rash. Formal Methods for Swarm<br />
and Autonomic Systems. Proc. 1st International Symposium on Leveraging Applications of Formal<br />
Methods (ISoLA), 2004.<br />
[RR06] M. Truszkowski W. Rouff, C. A. Hinchey and J. Rash. Experiences Applying Formal Approaches in<br />
the Development of Swarm-Based Space Exploration Systems. International Journal on software<br />
Tools for Technology Transfer. Special Issue on Formal Methods in Industry, 2006.
REFERENCES 81<br />
[SB01] G. B. Sumpter, D. J. T. Blanchard and D. S. Broomhead. Ants and Agents: a Process Algebra<br />
Approach to Modelling Ant Colony Behaviour. Bulletin of Mathematical Biology, 63(5):951–980,<br />
September 2001.<br />
[SG99] W. M. Spears and D. F. Gordon. Using artificial physics to control agents. Proc. IEEE International<br />
Conference on Information, Intelligence, and Systems, November 1999.<br />
[SK96] O. Shehory and S. Kraus. Cooperative goal-satisfaction without communication in large-scale<br />
agent-systems. ECAI 96. 12th European Conference on Artificial Intelligence, 1996.<br />
[Spe04a] W. Spears. Distributed, Physics-Based Control of Swarms of Vehicles. Autonomous Robots, 17:137–<br />
162, 2004.<br />
[Spe04b] W. Spears. Physicomimetics for Mobile Robot Formations. AAMAS, July 19–23 2004.<br />
[Spe05] W. Spears. An Overview of Physicomimetics. Computer Science Department, University of<br />
Wyoming, Laramie, 2005.<br />
[SS04] A. Strurm and O. Shehory. A Framework for Evaluating Agent-Oriented Methodologies. AOIS,<br />
LNAI 3030, pages 94–109, 2004.<br />
[SS06] W. Spears, W. Kerr and D. Spears. Fluid-Like Swarms with Predictable Macroscopic Behavior.<br />
Computer Science Department, University of Wyoming, Laramie, 2006.<br />
[SW05] W. Spears, D. Kerr and Spears W. Physics-Based Robot Swarms For Coverage Problems.<br />
IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), August 2005.<br />
[SY99b] Kraus Shehory and Yadgar. Goal-satisfaction in large-scale agent-systems: a transportation ex-<br />
ample. Carnegie Mellon University, 1999.<br />
[SY99a] O. Shehory, S. Kraus and O. Yadgar. Emergent Cooperative Goal-Satisfaction in Large-Scale<br />
Automated-Agent Systems. Artificial Intelligence, 110(1):1–55, 1999.<br />
[Tof91] C. Tofts. Describing social insect behavior using process algebra. Transactions on Social Computing<br />
Simulation, pages 227–283, 1991.<br />
[vN96] J. von Neumann. Theory of Self-Reproducing Automata. University of Illinois Press, Urbana,<br />
Illinois, 1996.<br />
[ZS04] T. Zhang, P. Lu and L. Song. Soccer robot path planning based on the artificial potential field<br />
approach with simulated annealing. Robotica, 22:563–566, 2004.<br />
[ZW02] Q. Yin B. Zhuang, X. Meng and H. Wang. Robot Path Planning by Artificial Potential Field Op-<br />
Web sites<br />
timization Based on Reinforcement Learning with Fuzzy State. 4th World Congress on Intelligent<br />
Control and Automation, June 10–14 2002.<br />
[1] http://en.wikipedia.org/wiki/Asteroid belt (2008)<br />
[2] http://ssd.jpl.nasa.gov/sbdb.cgi#top (2008)<br />
[3] http://aa.usno.navy.mil/faq/docs/asteroid masses (2008)<br />
[4] http://ants.gsfc.nasa.gov (2007)
82 REFERENCES<br />
3 The Decisional part<br />
[BL08] A. Brambilla and M. Lavagna. A Decentralized Approach to Cooperative Situation Assessment<br />
in Multi-Robot Systems. 7th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS<br />
2008), May 12–16 2008.<br />
[DM06] A. DeMartino and M. Marsili. Statistical mechanics of socio-economic systems with heterogeneous<br />
agents. Journal of Physics A: Mathematical and General, 39(43):R465 – R540(1), October 2006.<br />
[EK06] G. H. Elkaim and R. J. Kelbley. A Lightweight Formation Control Methodology for a Swarm of<br />
Non-Holonomic Vehicles. IEEE, 2006.<br />
[SB07a] S.L. Smith and F. Bullo. Monotonic target assignment for robotic networks. IEEE Transacion on<br />
Automatic Control, June 2007. Submitted.<br />
[SB07b] S.L. Smith and F. Bullo. Target assignment for robotic networks: asymptotic performance under<br />
limited communication. Autonoumous Control Conference, July 2007.<br />
[SG99] W. M. Spears and D. F. Gordon. Using artificial physics to control agents. Proc. IEEE International<br />
Conference on Information, Intelligence, and Systems, November 1999.<br />
[SK96] O. Shehory and S. Kraus. Cooperative goal-satisfaction without communication in large-scale agent-<br />
systems. ECAI 96. 12th European Conference on Artificial Intelligence, 1996.<br />
[SN08] P. Farinelli A. Sycara K. Settembre, G. Scerri and D. Nardi. A Decentralized Approach to Coop-<br />
erative Situation Assessment in Multi-Robot Systems. 7th Int. Conf. on Autonomous Agents and<br />
Multiagent Systems (AAMAS 2008), May 12–16 2008.<br />
[Spe04a] W. Spears. Distributed, Physics-Based Control of Swarms of Vehicles. Autonomous Robots, 17:137–<br />
162, 2004.<br />
[Spe04b] W. Spears. Physicomimetics for Mobile Robot Formations. AAMAS, July 19–23 2004.<br />
[Spe05] W. Spears. An Overview of Physicomimetics. Computer Science Department, University of<br />
Wyoming, Laramie, 2005.<br />
[SS04] A. Strurm and O. Shehory. A Framework for Evaluating Agent-Oriented Methodologies. AOIS,<br />
LNAI 3030, pages 94–109, 2004.<br />
[SS06] W. Spears, W. Kerr and D. Spears. Fluid-Like Swarms with Predictable Macroscopic Behavior.<br />
Computer Science Department, University of Wyoming, Laramie, 2006.<br />
[SW05] W. Spears, D. Kerr and Spears W. Physics-Based Robot Swarms For Coverage Problems. IEEE/RSJ<br />
Int. Conf. on Intelligent Robots and Systems (IROS), August 2005.<br />
[SY99a] O. Shehory, S. Kraus and O. Yadgar. Emergent cooperative goal-satisfaction in large-scale<br />
automated-agent systems. Artificial Intelligence, 110(1):1–55, 1999.<br />
[SY99b] S. Shehory, O. Kraus and O. Yadgar. Goal-satisfaction in large-scale agent-systems: a transportation<br />
example. Carnegie Mellon University, 1999.
REFERENCES 83<br />
4 The Physical part<br />
[AKL06] J. Abu-Khalaf, M. Huang and F.L. Lewis. Nonlinear H2/H∞ Constrained Feedback Control.<br />
Springer, 2006.<br />
[AM90] B. Anderson and J. Moore. Optimal Control. Prentice Hall, 1990.<br />
[Arm04] Armellin. Formation fliying control. Master’s thesis, Politecnico di Milano, 2004.<br />
[BB95] T. Basar and P. Bernhard. H∞ - Optimal Control and Related Minimax Design Problems.<br />
Birkhaeuser, 1995.<br />
[BM00] R.W. Beard and T.W. McLain. Successive Galerkin Approximation Algorithms for Non Linear and<br />
Robust Control. International Journal of Control, 2000.<br />
[Bra04] A. Bracci. Nuovi sviluppi nelle tecniche di controllo nonlineare SDRE. Master’s thesis, Università<br />
di Pisa, 2004.<br />
[Clo97] J.R. Cloutier. State-Dependent Riccati Equation Techniques: An Overview. Proceedings of the<br />
American Control Conference, June 1997.<br />
[EA01] Erdem and Alleyne. Experimental Real - Time SDRE Control of an Underactuated Robot. Proced-<br />
ings of the 40th IEEE Conference on Decision and Control, 2001.<br />
[Gad07] J. Gadewadikar. H-Infinity Output-Feedback COntrol: Application to Unmanned Aerial Vehicle. PhD<br />
thesis, The University of Texas at Arlington, May 2007.<br />
[GC06] F.L. Subbarao Q. Peng L. Gadewadikar, J. Lewis and T. Chen. H-Infinity Static Output-Feedback<br />
Control for Rotorcraft. AIAA Guidance, Navigation, and Control Conference and Exhibit, August<br />
2006.<br />
[GG96] F. Garofalo and L. Glielmo. Robust Control via Variable Structure and Lyapunov Techniques.<br />
Springer, 1996.<br />
[GL06] J. Gadewadikar and F. L. Lewis. Aircraft flight controller tracking design using H-Infinity static<br />
output-feedback. Transactions of the Institute of Measurement and Control, 28(5):429 – 440, 2006.<br />
[IH02] M. Inalhan, G. Tillerson and J. How. Relative Dynamics and Control of Spacecraft Formations in<br />
Eccentric Orbits. Journal of Guidance, Control and Dynamics, 2002.<br />
[IP07] Dario Izzo and Lorenzo Pettazzi. Autonomous and distributed motion planning for satellite swarm.<br />
Journal of Guidance, Control and Dynamics, 30(2):449 – 459, 2007.<br />
[JB05] A. Jaganath, C. Ridley and D.S. Bernstein. A SDRE-Based Asymptotic Observer for Nonlinear<br />
Discrete-Time Systems. American Control Conference, June 2005.<br />
[MC02] T. Crawford L.S. Menon, P.K. Lam and V.H.L. Cheng. Real-Time Computational Methods for<br />
SDRE Nonlinear Control of Missiles. Proceedings of the American Control Conference, May 2002.<br />
[Mei90] L. Meirovitch. Dynamics and Control of Structures. Wiley, 1990.<br />
[MO03] G.D. Menon, P.K. Sweriduk and E.J. Ohlmeyer. Optimal Fixed-Interval Integrated Guidance-<br />
Control Laws For Hit-To-Kill Missiles. AIAA Guidance, Navigation, and Control Conference and<br />
Exhibit, August 2003.
84 REFERENCES<br />
[MY07] M. McCamish, S. Romano and X. Yun. Autonomous distributed control algorithm for multiple<br />
spacecraft in close proximity operations. AIAA Guidance, Navigation, and Control Conference and<br />
Exhibit, 20 – 23 August 2007.<br />
[Naa02] Bo J. Naasz. Classical Element Feedback Control for Spacecraft Orbital Manoeuvers. Master’s<br />
thesis, Virginia Polytechnic Institute and State University, May 2002.<br />
[PB04] R.E. Palumbo, N.F. Brian and R.A. Blauwkamp. Integrated Guidance and Control for Homing<br />
Missiles. Johns Hopkins APL Technical Digest, 25(2), 2004.<br />
[PJ99] N.F. Palumbo and P. Jackson. Integrated Missile Guidance and Control: a State Dependent Riccati<br />
Differential Equation Approach. Proceedings of the 1999 IEEE International Conference on Control<br />
Applications, August 1999.<br />
[Szn00] M. Sznaier. Receding Horizon Control Lyapunov Function Approach to Suboptimal Regulation of<br />
Nonlinear Systems. Journal of Guidance, Control and Dynamics, 23(3), 2000.<br />
[WK03] Bogdanov Wan and Kieburtz. Model Predictive Neural Control for Aggressive Helicopter Manoeu-<br />
vers, 2003.<br />
[XO06] S.N. Xin, M. Balakrishnan and E.J. Ohlmeyer. Integrated Guidance and Control of Missiles With<br />
θ-D Method. IEEE Transactions On Control Systems Technology, 14(6), November 2006.<br />
5 The Communication part<br />
[BR05] E. Taghi M. Bredin, J. Demaine and D. Rus. Deploying Sensor Networks with Guaranteed Capacity<br />
and Fault Tolerance. MobiHoc’05, May 25–27 2005.<br />
[CB04] M. Chandrashekar, K. Raissi and J. Baras. Providing Full Connectivity in Large Ad-Hoc Networks<br />
by Dynamic Placement of Aerial Platforms. MILCOM 2004 - 2004 IEEE Military Communications<br />
Conference, October 31 – November 3 2004.<br />
[HS02] M.J. Howard, A. Matari´c and G.S. Sukhatme. Mobile Sensor Network Deployment using Potential<br />
Fields: A Distributed, Scalable Solution to the Area Coverage Problem. In Proceedings of the<br />
6th International Symposium on Distributed Autonomous Robotics Systems (DARS02), June 25–27<br />
2002.<br />
[KP05] A. Krishnamurthy and R. Preis. Satellite Formation, a Mobile Sensor Network in Space. Proceedings<br />
of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS’05), April<br />
3–8 2005.<br />
[MA07] R. Martijn and B. Andreas. Multi–robot exploration under the constraints of wireless networking.<br />
Control Engineering Practice, 15(4):435–445, 2007.<br />
[MCT02] H. Murrieta-Cid, R. Gonzglez-Bafiost and B. Tovar. A Reactive Motion Planner to Maintain Visi-<br />
bility of Unpredictable Targets. Proceedings of the 2002 IEEE lnternational Conference on Robotics<br />
and Automation, May 2002.<br />
[MS01] F. Potkonjak M. Meguerdichian, S. Koushanfar and M. Srivastava. Coverage Problems in Wireless<br />
Ad-hoc Sensor Networks. Proceedings IEEE INFOCOM 2001. Twentieth Annual Joint Conference<br />
of the IEEE Computer and Communications Societies, April 22–26 2001.<br />
[Par02] L.E. Parker. Distributed Algorithms for Multi-Robot Observation of Multiple Moving Targets.<br />
Autonomous Robots, 12:231–255, 2002.
REFERENCES 85<br />
[SL02] J. Savarese, C. Rabaey and K. Langendoen. Robust Positioning Algorithms for Distributed Ad-Hoc<br />
Wireless Sensor Networks. USENIX Technical Annual Conference, June 2002.<br />
[SO07] P. Scerri and S. Owens. A Decentralized Approach to Space Deconfliction. IEEE Fusion Conference,<br />
2007.<br />
[SS08] A. Simonetto, P. Scerri and K. Sycara. A Mobile Network for Mobile Sensors. IEEE Fusion<br />
Conference, June 2008.<br />
[VS07] P. Velagapudi and P. Scerri. Maintaining Shared Belief in a Large Multiagent Team. IEEE Fusion<br />
Conference, 2007.<br />
[XS05] Y. Xu and P. Scerri. An Integrated Token-Based Algorithm for Scalable Coordination. AAMAS,<br />
July 25 – 29 2005.<br />
[YS06] B. Yu and P. Scerri. Scalable and Reliable Data Delivery in Mobile Ad Hoc Sensor Networks.<br />
AAMAS, 2006.<br />
6 Perturbation models<br />
[Ber99] M. Berz. Academic Press. Modern map methods in particle beam physics, 1999.<br />
[Cho02] V. Chobotov. Orbital Mechanics. AIAA – Education Series, 2002.<br />
[Fer08] D. Ferraro. Taylor Series Methods. Master’s thesis, Politecnico di Milano, 2008. to be submitted.<br />
[PS06] G. Parissenti and A. Simonetto. Perturbed Orbits. Report, Orbital Mechanics, 2006.<br />
7 Results<br />
[GM07] S. Gill, E. D’Amico and O. Montenbruck. Autonomous Formation Flying for the PRISMA Mission.<br />
Journal of Guidance, Control and Dynamics, 44(3):671 – 681, May – June 2007.<br />
[SS08] A. Simonetto, P. Scerri and K. Sycara. A Mobile Network for Mobile Sensors. IEEE Fusion Confer-<br />
Web sites<br />
ence, June 2008.<br />
[1] http://ants.gsfc.nasa.gov (2007)<br />
[2] http://www.esa.it (2007)<br />
8 Final remarks<br />
[BL08] A. Brambilla and M. Lavagna. A Decentralized Approach to Cooperative Situation Assessment in<br />
Multi-Robot Systems. 7th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2008),<br />
May 12–16 2008.<br />
[MY07] M. McCamish, S. Romano and X. Yun. Autonomous distributed control algorithm for multiple<br />
spacecraft in close proximity operations. AIAA Guidance, Navigation, and Control Conference and<br />
Exhibit, 20 – 23 August 2007.
86 REFERENCES
APPEN<strong>DI</strong>X<br />
A<br />
SISTEMI MULTI AGENTE PER APPLICAZIONI SPAZIALI<br />
“Buongiorno Mastro Antonio” disse Geppetto “Che fai lì sul pavimento?<br />
“Insegno l’alfabeto alle formiche”<br />
Carlo Collodi (1826 - 1890)<br />
Questa tesi è un lavoro di ricerca sui sistemi multi agente in campo spaziale, in partico-<br />
lare per missioni di volo di formazione ed esplorative. Per prima cosa si analizza lo stato<br />
del’arte, in secondo luogo si propone un controllo distribuito robusto sia di alto livello sia di<br />
basso livello. Il secondo include un controllo sulla traiettoria e la formazione di una rete di<br />
comunicazioni.<br />
A.1 Introduzione<br />
I sistemi multi agente sono studiati da discreto tempo in campo informatico, dove i con-<br />
testi applicativi sono diversi, dai web services, ad applicazioni multi robot, come scenari<br />
di salvataggio o militari. Da qualche anno la loro versatilità e affidabilità ha focalizzato<br />
l’attenzione del settore spaziale, in particolare perché tali sistemi sono relativamente poco<br />
costosi e possono permettere alti tassi di perdite tra i satelliti che compongono la formazione<br />
senza compromettere l’adempimento della missione. Sia ESA che NASA stanno proponendo<br />
missioni, come grandi formazioni di satelliti per telescopi interferometrici o esplorazione della<br />
cintura di asteroidi, dove il paradigma multi agente è l’aspetto fondamentale. Per questo<br />
motivo i metodi classici di controllo centralizzato devono essere rivisti in ottica distribuita e<br />
multi agente, questa è anche l’idea fondamentale della tesi: studiare come gli algoritmi multi<br />
agente possano entrare nell’ambiente spaziale in modo verificabile e robusto.<br />
i
ii SISTEMI MULTI AGENTE<br />
A.1.1 Principali Contributi<br />
I principali contributi innovativi alla tesi sono diversi: in primo luogo la formulazione del<br />
controllo di alto livello – Goal Manager – è una estensione di una tecnica formale conosciuta<br />
come Fisica Artificiale. In secondo luogo, il controllo della traiettoria sviluppa un controllo<br />
non lineare alla Lyapunov per inquadrarlo in un contesto robusto, H∞, risolvendo con una<br />
opportuna semplificazione una particolare equazione di Riccati. Infine, per la parte di assicu-<br />
razione di una rete di comunicazione, le prestazioni di un metodo a campi a potenziale sono<br />
migliorate introducendo pacchetti di informazione random, chiamati token.<br />
A.2 Formulazione del Problema<br />
Il problema viene viene formulato come un problema di controllo robusto e distribuito: dato<br />
uno scenario, scrivere un algoritmo di controllo che consenta ad ogni agente di decidere<br />
autonomamente o con un minimo utilizzo di comunicazione, cosa fare e come fare a farlo,<br />
cioè come muoversi e come comunicare.<br />
Gli scenari studiati sono due: un volo di formazione di due satelliti, missione ESA -<br />
Prisma, dove le parte fondamentale è assicurare una distanza relativa in un certo intervallo<br />
a fronte di diverse perturbazioni, quali atmosferica e gravitazionale; poi, l’esplorazione della<br />
cintura degli asteroidi, dove uno sciame di satelliti è spedito per raccogliere informazioni. In<br />
questo caso la non conoscenza dell’ambiente impone che il sistema di controllo sia robusto<br />
per far fronte ad incertezze di modello e perturbazioni esterne.<br />
Il controllo di ogni agente è diviso in due parti, la parte di alto livello, che deve decidere<br />
che cosa fare, e la parte di basso livello, che invece decide come farlo. Per il controllo di alto<br />
livello si usa una approccio simile ai campi a potenziale, chiamato Fisica Artificiale (Artificial<br />
Physics). Tale metodo è selezionato perché, in primo luogo, è inquadrabile in un contesto<br />
formale, cosa importante per la verifica dell’algoritmo; in secondo luogo, richiede un minimo<br />
uso di comunicazione, cosa fondamentale in campo spaziale. La parte di basso livello è divisa<br />
ulteriormente in una parte di controllo della traiettoria e una parte di controllo della creazione<br />
di una rete di comunicazioni.<br />
A.3 Controllo di Alto Livello<br />
L’idea alla base dell’approaccio a Fisica Artificiale consiste nella formazione di un campo a<br />
potenziale sulla basse delle informazioni che arrivano dal mondo esterno. La scelta su cui si<br />
basa il controllo di Alto Livello è poi ridotta al calcolo del gradiente di tale campo.<br />
Definendo un opportuno insieme di agenti A, una misura delle capacità che ogni agente<br />
ha, cij, un opportuno insieme di obiettivi G, le capacità che ogni obiettivo richiede per essere<br />
soddisfatto, tij e una metrica che misura quanto gli agenti siano vicini tra di loro o all’obiettivo<br />
dij, si riesce a introdurre un potenziale nella forma<br />
Φ w i = �<br />
n<br />
αnicnwciw Φa(din) + �<br />
m<br />
βnitnwciw Φg(dim) , w = 1, . . . , k (A.1)
SISTEMI MULTI AGENTE iii<br />
Il quale estende la formulazione tradizionale di Fisica Artificiale a contesti con più ca-<br />
pacità/obiettivi. Da un calcolo del gradiente, con eventualmente una risoluzione di conflitti,<br />
ogni agente riesce, in modo completamente autonomo a decidere quale obiettivo perseguire.<br />
Siccome di suppone che gli agenti non abbiamo una conoscenza completa dell’ambiente,<br />
è importante notare che le informazioni che servono ad ogni sonda per costruire il proprio<br />
campo a potenziale locale sono di due tipi, da un lato quelle ricavate dai sensori, dall’altro<br />
quelle inviate dalle altre sonde. Questo implica l’esistenza di una rete di comunicazioni e un<br />
protocollo attraverso il quale l’informazione è condivisa. Per questioni di velocità di risposta,<br />
si sceglie un approccio broadcast per il protocollo. Inoltre, l’informazione è opportunamente<br />
modificata viaggiando da agente ad agente per tener conto delle scelte personali ed eventuali<br />
conflitti.<br />
A.4 Controllo Robusto della Traiettoria<br />
L’acquisizione della traiettoria che porta ogni singolo agente nel luogo desiderato è formulata<br />
dividendo la dinamica in lungo e corto periodo.<br />
La parte di lungo periodo tiene in considerazione la traiettoria globale ed è sviluppata<br />
con variabili equinoziali non singolari nel contesto di un controllo non lineare alla Lyapunov.<br />
Inoltre per tenere presente le perturbazioni tale approccio è esteso usando una tecnica H∞<br />
che rende robusto l’algoritmo. L’equazione di Hamilton – Jacobi – Isaacs risultante<br />
0 = min<br />
u max<br />
w [L(x, u, w) + ∇ V B (u + w)] (A.2)<br />
porta ad una particolare equazione di Riccati, che viene denominata equazione di anna, che<br />
può essere risolta con opportune semplificazioni e permette di derivare le condizioni necessarie<br />
e sufficienti per la stabilità. Il controllo viene scritto come<br />
u = − 1<br />
2 R−1 B ′ Q δx (A.3)<br />
Per la parte di corto periodo si propongono due possibili soluzioni, entrambe note in<br />
letteratura. La prima consiste in un accoppiamento controllo ottimo e campi a potenziale di<br />
velocità, la seconda in una ottimizzazione vincolata della dinamica discretizzata via matrici<br />
di convoluzione. Entrambe andrebbero adattate ed integrate al modello di lungo periodo.<br />
A.5 Architettura di Comunicazione<br />
L’architettura di comunicazione, o meglio come gli agenti designati a formare la rete di<br />
comunicazione tra gli esploratori e Terra si devono muovere, è formulata in un contesto di<br />
campi a potenziale. Questo si combina perfettamente con l’idea di Fisica Artificiale. Siccome<br />
i campi a potenziale non consentono, da un lato, di focalizzare gli agenti sul loro dovere<br />
ma, generalmente, li distribuiscono equamente nello spazio, e dall’altro, non minimizzano le<br />
comunicazioni in modo drastico, è sviluppato un approccio a potenziale dinamico basato su
iv SISTEMI MULTI AGENTE<br />
singoli pacchetti random di informazione, i token. Questo consente di migliorare le prestazioni<br />
sia per quanto riguarda la connettività globale sia per la tolleranza a rotture di singoli agenti.<br />
L’idea chiave dell’approccio dinamico è usare i singoli token per spedire attraverso la<br />
rete di comunicazione richieste di aiuto; mediante l’imposizione di una opportuna politica<br />
ogni agente sceglie quale agente è più in difficoltà e decide quale pacchetto di informazione<br />
seguire e quale rimbalzare ad altri. Le politiche analizzate sono diverse, ma includono tutte<br />
il concetto di connettività locale.<br />
A.6 Modelli per le Perturbazioni<br />
Due i modelli di perturbazione sviluppati, uno per le missioni di volo di formazione vicino<br />
Terra, che include sia la resistenza atmosferica sia effetti gravitazionali come J2 e J22, ed uno<br />
per le missioni nella cintura degli asteroidi. Quest’ultimo scrive il campo perturbativo come<br />
somma dei campi gravitazionali dovuti ai primi dieci più massivi asteroidi.<br />
A.7 Risultati<br />
I risultati si dividono in quattro gruppi, quelli del controllo di alto livello, applicato ad<br />
un esempio abbastanza generale; quelli del controllo della traiettoria, applicato a diverse<br />
simulazioni nel contesto della missione Prisma; quelli per la rete di comunicazione, applicati<br />
a scenari a due e tre dimensioni; infine, quelli per il controllo globale, applicato ad uno<br />
scenario di missione verso gli asteroidi.<br />
A.7.1 Controllo di alto livello<br />
Il controllo di alto livello e il meccanismo di condivisione delle informazioni, porta a buoni<br />
risultati, in particolare riesce a assegnare gli obiettivi con errori inferiori al 10% in uno scenario<br />
con 3 asteroidi e 3 capacità/obiettivi, Figura A.1.<br />
A.7.2 Controllo della traiettoria<br />
Il controllo della traiettoria è applicato alla missione Prima e, in particolare, si vuole garan-<br />
tire che il satellite che insegue riesca ad avere una distanza relativa rispetto a quello che<br />
precede, entro la decina di kilometri. Questo permetterebbe di sviluppare in sede successiva<br />
un controllo più fine di rendezvous. Diverse simulazioni e test sono stati condotti, il più sig-<br />
nificativo è una analisi statistica alla Montecarlo su incertezze delle variabili di stato iniziali<br />
dell’inseguitore, a simulare un’incertezza sulla posizione in cui il lanciatore lascerà il satellite.<br />
ā ē ī ¯ω ¯ Ω ¯ θ<br />
RE+ 700 km 0 98.2π/180 0 0 0<br />
Table A.1: Parametri orbitali del satellite Main.
SISTEMI MULTI AGENTE v<br />
Valore t ij /(t ij ) 0<br />
0.25<br />
0.2<br />
0.15<br />
0.1<br />
0.05<br />
0<br />
−0.05<br />
−0.1<br />
−0.15<br />
−0.2<br />
−0.25<br />
Prestazioni del controllo di alto livello<br />
Primo Asteroide<br />
Secondo Asteroide<br />
Terzo Asteroide<br />
30 40 50 60 70 80 90 100<br />
Numero di agenti, N<br />
Figure A.1: Risultati per il Goal Manager; la larghezza delle linee rappresenta la deviazione<br />
standard.<br />
dove RE è il raggio terrestre.<br />
a e i ω Ω θ<br />
RE+ 700km ± 100 km ē + 0.1 ī ± 10 ◦ ¯ω ± 10 ◦ ¯ Ω ± 10 ◦ π/2 ± 10 ◦<br />
Table A.2: Incertezza sui parametri del satellite Target per la simulazione Montecarlo.<br />
In Figura A.2 le distanze relative dopo 20h. I risultati mostrano un comportamento<br />
robusto dell’algoritmo di controllo consentendo un avvicinamento sotto i 5 km.
vi SISTEMI MULTI AGENTE<br />
Cross track [km]<br />
Radial [km]<br />
2<br />
1.5<br />
1<br />
0.5<br />
0<br />
−0.5<br />
−1<br />
Distanza relativa per t ∈ [20, 21] h<br />
−1.5<br />
−40 −20 0 20 40<br />
In track [km]<br />
60 80 100 120<br />
2<br />
1.5<br />
1<br />
0.5<br />
0<br />
−0.5<br />
−1<br />
Distanza relativa per t ∈ [20, 21] h<br />
−1.5<br />
−40 −20 0 20 40<br />
In track [km]<br />
60 80 100 120<br />
Figure A.2: Posizioni relative raggiunte dopo 20 h. La linea nera è una delle traiettorie, le<br />
altre sono ad essa simili, e tutte passano per il punto 0 km in track.
SISTEMI MULTI AGENTE vii<br />
A.7.3 Dispiegamento della rete di comunicazione<br />
Il dispiegamento della rete di comunicazione viene testato su due scenari significativi, uno<br />
bidimensionale, l’altro tridimensionale, mostrando come gli algoritmi token portino ad un<br />
notevole aumento delle prestazioni, sia in termini di connettività globale media (nel tempo)<br />
< K >, che di efficienza media < E > del grafo.<br />
Nelle Figure A.3 - A.4, sono presentati i risultati per lo scenario tridimensionale.<br />
Connettività globale media<br />
10<br />
9<br />
8<br />
7<br />
6<br />
5<br />
4<br />
3<br />
2<br />
1<br />
0<br />
B C RC B C RC B C RC<br />
Kill rate = 0.00 Kill rate = 0.16 Kill rate = 0.33<br />
Figure A.3: < K > per un differente numero di comunicatori, |C|: B sta per algoritmo Base,<br />
C per politica C, RC per politica RC. Le barre di errore nere sono la deviazione standard, le<br />
linee nere i valori finali di K.<br />
A.7.4 Cintura degli asteroidi<br />
Uno scenario all’interno della cintura degli asteroidi è sviluppato per mostrare come il con-<br />
trollo globale degli agenti funzioni assieme; in particolare si scelgono due asteroidi da rag-<br />
giungere, tre esploratori che devono raggiungerli, un agente hub che deve comunicare con<br />
Terra, e una serie di agenti di comunicazione tra esploratori e hub. I risultati, mostrati nelle<br />
Figure A.5 - A.7.4, portano ad un controllo robusto sulla traiettoria e ad un dispiegamento<br />
della rete di comunicazione tale da assicurare connettività almeno pari ad uno.
viii SISTEMI MULTI AGENTE<br />
Efficienza media<br />
1<br />
0.9<br />
0.8<br />
0.7<br />
0.6<br />
0.5<br />
0.4<br />
0.3<br />
0.2<br />
0.1<br />
0<br />
B C RC B C RC B C RC<br />
Kill rate = 0.00 Kill rate = 0.16 Kill rate = 0.33<br />
Figure A.4: < E > per un differente numero di comunicatori, |C|: B sta per algoritmo Base,<br />
C per politica C, RC per politica RC. Le barre di errore nere sono la deviazione standard.<br />
y [AU]<br />
0.05<br />
0<br />
−0.05<br />
−0.1<br />
−0.15<br />
−0.2<br />
−0.25<br />
−0.3<br />
−0.35<br />
Posizioni relative dall’Hub [AU]<br />
−0.1 0 0.1 0.2<br />
x [AU]<br />
0.3 0.4<br />
Figure A.5: Traiettorie per esploratori (nero), Hub (rosso) e asteroidi (blu) in un sistema di<br />
riferimento inerziale centrate nell’Hub.
SISTEMI MULTI AGENTE ix<br />
y [AU]<br />
0.1<br />
0.05<br />
0<br />
−0.05<br />
−0.1<br />
−0.15<br />
−0.2<br />
−0.25<br />
−0.3<br />
−0.35<br />
−0.4<br />
−0.2 −0.1 0 0.1 0.2<br />
x [AU]<br />
0.3 0.4 0.5 0.6<br />
(a) Stato Iniziale<br />
y [AU]<br />
0.1<br />
0.05<br />
0<br />
−0.05<br />
−0.1<br />
−0.15<br />
−0.2<br />
−0.25<br />
−0.3<br />
−0.35<br />
−0.4<br />
−0.2 −0.1 0 0.1 0.2<br />
x [AU]<br />
0.3 0.4 0.5 0.6<br />
(b) Stato Finale<br />
Figure A.6: Scenario Asteroidi: i cerchi neri sono gli esploratori, quelli bianchi i comunicatori,<br />
le linee tratteggiate i collegamenti della rete di comunicazione, il triangolo, l’Hub.<br />
A.8 Sviluppi Futuri<br />
Tra gli sviluppi futuri si possono elencare<br />
� estensione del controllo di alto livello a piccole formazioni e quando la connettività<br />
iniziale è piccola;<br />
� dinamica di corto periodo per il controllo della traiettoria;<br />
� controllo di movimento linearizzato per l’assicurazione della rete di comunicazione.
that’s all folks