Graphics for a 3D Driving Simulator - Robotics UWA

L E H R S T U H L F Ü R R E A L Z E I T - C O M P U T E R S Y S T E M E
TECHNISCHE UNIVERSIT ÄT MÜNCHEN
UNIV.-PROF. DR.-ING. G. F ÄRBER
Graphics for a 3D Driving Simulator
Johannes Georg Brand
Bachelor Thesis

Graphics for a 3D Driving Simulator
Bachelor Thesis
Supervised by the Institute for Real-Time Computer Systems
Technische Universität München
Prof. Dr.-Ing. Georg Färber
Conducted at Robotics and Automation Lab
Center for Intelligent Information Processing Systems
University of Western Australia
Perth
Advisor: Assoc. Prof. Dr. rer. nat. habil. Thomas Bräunl
Adrian Boeing
Dipl.-Ing. Philipp Harms
Author: Johannes Georg Brand
Eulenweg 2
85356 Freising
Submitted 3rd March 2008

Contents
List of Figures v
List of Tables vii
List of Symbols viii
1 Introduction 1
2 Related Work 6
3 AutoSim Framework 15
4 World and Robot Creation 25
5 Terrain Modeling 29
iii

iv Contents
6 Road Construction 40
7 Rendering Methods 47
8 Conclusion and Future Work 50
A Tutorials 52
B The Configuration Files 59
Bibliography 70

List of Figures
v

vi List of Figures

List of Tables
vii

List of Symbols
viii

Abstract
Due to the continuously growing amount of traffic on our roads, and with it, an increasing
risk of car accidents, the theory and design of robotic cars that take away the risk of
human driver errors has become an area of active research among car manufacturers,
suppliers and universities. Even if various competitions for autonomous driving cars have
introduced some excellent testing venues in real world environments, decent simulators for
testing the robotic programs without hardware do not exist. In the light of this an extensible,
modular and flexible open source component-based 3D driving simulator framework called
AutoSim is presented in this thesis. The framework enables rapid and interactive development
of robotic car algorithms and allows researchers to develop, test and experiment
with autonomous vehicle software without the need for a physical vehicle. Using a testing
environment like AutoSim can tremendously increase the speed of development and thus
save costs by reducing time to market.
Constructing a simulator framework includes meeting a large range of demands for creating
a high-level 3D environment. A large 3D scene has to be adapted for execution on limited
computer hardware and nevertheless provide the user all features desired for interaction
with the tested robots. This thesis follows the ideas of methods like GeoMipMap and Geometric
Clipmaps by applying them to the simulator’s requirements and performing parallel
operations on the graphics card. The road system in AutoSim is designed in order to display
an accurate representation of a real world street web, implemented through self-developed
methods for procedurally constructing road meshes out of 2D world road data and transforming
them by height data. Finally, the thesis introduces approaches for designing a 3D
simulator framework which includes concepts for transferring and managing data as well
as multiple rendering methods.
x

Zusammenfassung
Aufgrund einer beständig anwachsenden Verkehrsdichte auf unseren Straßen und eines
damit verbundenen immer größer werdenden Unfallrisikos, ist das Entwickeln von Roboter
Autos, welche die Fehler des menschlichen Fahrers als Unfallrisiko entfernen, ein aktiv umforschtes
Gebiet von Fahrzeugherstellern, Lieferanten und Universitäten geworden. Auch
wenn verschiedene Wettbewerbe schon exzellente Testumgebungen für autonome Fahrzeuge
bereitgestellt haben, gibt es noch immer keine geeigneten Simulatoren um Roboter
Programme ohne Hardware zu testen. Im Angesicht dessen wird in dieser Bachelorarbeit
ein erweiterbares, modular aufgebautes und flexibel anwendbares Open Source 3D
Fahrsimulator Framework mit dem Namen AutoSim präsentiert. Das Framework ermöglicht
schnelles und interaktives Entwickeln von Programmen für Roboter und erlaubt Forschern
ohne Gebrauch eines physikalischen Fahrzeugs zu entwickeln, zu testen und zu experimentieren.
Eine Testumgebung wie AutoSim kann die Entwicklungszeit eines solchen
Programmes enorm beschleunigen und auch wegen eines verkürzten Time-to-Market’s
Kosten einsparen.
Das Entwickeln eines Fahrsimulator Frameworks erfordert einer großen Menge von Ansprüchen
einer hochwertigen 3D Anwendung gerecht zu werden. Eine große 3D Szene
muss an die Limitierungen der Computer Hardware angepasst werden und trotzdem alle
gewünschten Fähigkeiten zur Interaktion mit den Robotern bereitstellen. Diese Bachelorarbeit
übernimmt dazu Ideen von Methoden wie GeoMipMap oder Geometric Clipmaps
um sie an die Anforderungen des Simulators anzupassen und dabei parallele Operationen
auf der Grafikkarte auszuführen. Das Straßensystem von AutoSim ist entworfen um das
echte Straßennetz möglichst genau wiederzugeben, indem aus 2D Weltdaten durch selbst
entwickelte Methoden prozedurale Straßenmeshes erstellt werden, um sie dann durch Höhendaten
zu verändern. Abschließend führt die Arbeit noch einige Ansätze auf die das
Entwickeln eines 3D Fahrsimulator Frameworks betreffen und geht dabei auf Konzepte wie
das Übertragen und Verwalten von Daten sowie mehrere Render Methoden ein.
xi

1 Introduction
1.1 Motivation
Sojourner, Mars rover from Pathfinder mission [29].
1

2 1 INTRODUCTION

1.1 MOTIVATION 3
Stanley, 2005 Grand Challenge winner from Stanford University [4]

4 1 INTRODUCTION
1.2 Objectives

1.3 THESIS OUTLINE 5
1.3 Thesis Outline

2 Related Work
2.1 Literature Review
2.1.1 Design Patterns
6

2.1 LITERATURE REVIEW 7
•
•
2.1.2 Terrain Rendering
!"#$%& !"#$%&'()%*
'()*$%&
-.(/%&0)(112
34"1/%&054#)$".#26
34"1/%&7894"1/:(&$2;>>
+ ,
+",-&%.%/'()*$%&
Builder Design Pattern

8 2 RELATED WORK
ROAM Method

2.1 LITERATURE REVIEW 9
Geometrical MipMapping
2.1.3 Geometric Data Systems
Geometry Clipmap

10 2 RELATED WORK
2.2 Simulators
OpenStreetMap
2.2.1 RARS

2.2 SIMULATORS 11
RARS Screenshot [26]
2.2.2 TORCS

12 2 RELATED WORK
TORCS Screenshot [31]
2.2.3 Racer

2.2 SIMULATORS 13
Racer Screenshot [25]
2.2.4 SubSim

14 2 RELATED WORK
The Subsim AUV Simulator [1]

3 AutoSim Framework
3.1 Used Libraries
15

16 3 AUTOSIM FRAMEWORK

3.2 FRAMEWORK ARCHITECTURE 17
3.2 Framework Architecture
•
•
•
•
!"#$%&'(%)*+)*
23)*4*$5*6'!47
23)*4*$5*6'
23)*4*$5*6'
23)*4*$5*6'
,)#-$*.
!"#$%&'(/0&)1#
!"#$%&'(/0&)1#
!"#$%&'(/0&)1#
23)*4*$5*6'!47
/0&)1#23)*4*$5*6'!47
23)*4*$5*6'
23)*4*$5*6'
23)*4*$5*6'
AutoSim during runtime

18 3 AUTOSIM FRAMEWORK
3.3 Program Description
3.3.1 AutoSimServer
Graphical User Interface of the AutoSim Server

3.3 PROGRAM DESCRIPTION 19
3.3.2 AutoSimClient
AutoSimClient Graphical User Interface

20 3 AUTOSIM FRAMEWORK
3.3.3 The UserProgram
3.3.4 OsmManipulator

3.3 PROGRAM DESCRIPTION 21
User Program
•
•
•
•

22 3 AUTOSIM FRAMEWORK
3.4 Client Software Design
Main Window OsmManipulator

3.4 CLIENT SOFTWARE DESIGN 23
+,-%&'()"*
,#&-5&-417
3
89$$*&'#+&1-
:)6;0(

24 3 AUTOSIM FRAMEWORK

4 World and Robot Creation
4.1 World Creation
25

26 4 WORLD AND ROBOT CREATION
!"#$%&'#()*+",
-"."*&'#()*+", /(##)+,&'#()*+",
2+,*(#3)4(5
0(1(,-.%+)"#
*+%"&,0(1(,-.%+)"#
0 0
/(#+)'($)"#
1
0(1(,'($)"#
*+%"&,/(#+)-.%+)"#
0
2+,*(#3)4(5
/(#+)-.%+)"#
0
0
!"##$%&'($)"#
World Creation
0 0
2+,*(#3)4(5
!"##$%&-.%+)"#
*+%"&,!"##$%&-.%+)"#

4.2 ROBOT CREATION 27
•
•
•
•
4.2 Robot Creation

28 4 WORLD AND ROBOT CREATION
!"#"$%
&'($)%
*+,-.+)%
0+1$2/++0
!"#$%&!&'()*
./'))-)
!"#$%&!&'()*
333
Robot Creation
("#"$
+,--.)*
$('-0+(
!"#$%&!&'()*
$('-0+(.'4
/$01,2%32-&02)*
•
•
•
•
•
•
•
•
•
•
•
333
333

5 Terrain Modeling
5.1 Bilinear Interpolation
P (x, y)
Q11 = (x1, y1) Q12 = (x1, y2) Q21 = (x2, y1) Q22 = (x2, y2)
R1 = (x, y1) R2 = (x, y2)
f(R1) ≈ x2 − x x − x1
f(Q11) + f(Q21)
x2 − x1
x2 − x1
x2 − x1
f(R2) ≈ x2 − x x − x1
f(Q12) + f(Q22)
29
x2 − x1

30 5 TERRAIN MODELING
"
"
"
$
#
% ##
! ! !
#
% #$
& $
'
& #
Bilinear Interpolation
f(R1) f(R2)
P
f(P ) ≈ y2 − y y − y1
f(R1) + f(R2)
y2 − y1
y2 − y1
1 (x1, y1) (0, 0)
a1 = Q11
a2 = Q21 − Q11
a3 = Q12 − Q11
a4 = Q11 − Q21 − Q12 + Q22
Q11
% $$
$
f(P ) ≈ a1 + a2x + a3y + a4xy
% $#

2)5!9:;$!&*(!2)5!!:;$

32 5 TERRAIN MODELING
•
256
Example of predefined names for varying inputs
Output Structure
/ / Vertex shader output s t r u c t u r e
struct VS_OUTPUT
{
f l o a t 4 P o s i t i o n : POSITION ; / / v e r t e x p o s i t i o n
f l o a t 4 D i f f u s e : COLOR0; / / v e r t e x d i f f u s e c o l o r
f l o a t 2 TexCoord : TEXCOORD0; / / tex coords
} ;
VS_OUTPUT vertexMain (VS_INPUT IN )
{
VS_OUTPUT Output ;
Preparing data

5.3 APPLYING HEIGHT DATA TO THE WORLD 33
Output . P o s i t i o n = mul ( IN . vPosition , mWorldViewProj ) ;
return Output ;
}
5.3 Applying Height Data to the World

34 5 TERRAIN MODELING
5.4 Graphics Terrain
5.4.1 Terrain Mesh

5.4 GRAPHICS TERRAIN 35
!"#$ !"#% &'()
(*+,- -.,/012+/-'
Tile Mesh Construction
!"#!$%&'$()"*&("*$
-+..$-("+.&)%$)&).'&($%%)".&#)/
Terrain Gap
*+,$%&'$()"*&("*$
Gap in rendered Terrain

36 5 TERRAIN MODELING
Terrain Mesh Wireframe Terrain
5.4.2 Terrain Vertex Shader

5.4 GRAPHICS TERRAIN 37
10
10
256 130m ∗ 130m

38 5 TERRAIN MODELING
#
"
#
Flipping Edges
5.5 Creation of Static Objects
α
!
"
!
α = arccos( a ·b |a|| )
b|
a b
360

5.5 CREATION OF STATIC OBJECTS 39
0
360

6 Road Construction
6.1 Splines
t
p(t) = (2t 3 −3t 2 +1)pi +(t 3 −2t 2 +t)mi +(−2t 3 +3t 2 )pi+1 +(t 3 −t 2 )mi+1 t ∈ [0; 1]
pi pi+1
mi mi+1
t
40

6.1 SPLINES 41
h00(t) = 2t 3 − 3t 2 + 1
h10(t) = t 3 − 2t 2 + t
h01(t) = −2t 3 + 3t 2
h11(t) = t 3 − t 2
Hermite Basis Functions [36]
t 0 h00(t) 0
pi t = 1 h01(t) 0
pi+1 −1
mi =
mi+1 =
(1 − t)(1 + b)(1 + c)
(1 − t)(1 − b)(1 − c)
(pi − pi−1) + (pi+1 − pi)
2
2
(1 − t)(1 + b)(1 − c)
(1 − t)(1 − b)(1 + c)
(pi+1 − pi) + (pi+2 − pi+1)
2
2
t ∈ [−1; 1]
b ∈ [−1; 1]
c ∈ [−1; 1]
0
(C 1 )

42 6 ROAD CONSTRUCTION
Spline [33] Kochanek Bartels Parameters [37]
0 1
=
C
ds =
b
a
˙x(t) 2 + ˙y(t) 2 dt
˙x(t) ˙y(t) x y ˙p(t)
6.2 Road Data

6.3 ROAD GENERATION 43
class Road
{
public :
Vertex m_Position ;
VECTOR roadSegment ;
} ;
class I n t e r s e c t i o n
{
public :
Vertex m_Position ;
VECTOR i n t e r s e c t i o n V e r t i c e s ;
VECTOR i n t e r s e c t i o n I n d i c e s ;
} ;
Road Data Streets in Perth
6.3 Road Generation

44 6 ROAD CONSTRUCTION
left handed and right handed coordinate system [35]
0

6.3 ROAD GENERATION 45
!"#$%&
,+-&*&%.
!.)&&.
'()*+#
Road Construction Constructed Road Part
x 2
(x − 1) 2

46 6 ROAD CONSTRUCTION
2',,$1/*',%3'*,/0
(440$/
!"#$#%&'"#
.,/$)0$1/*',
Offset Fading
()*+*,"-%&'"#
T-junction after Road Generation
y(t) = (x − 1) 2 y(t) = x 2

7 Rendering Methods
7.1 Triangulation
v0 v1
v2 v1 v3 v2 v2 v3
v4 v3 v5 v4
vi vi+1
v1 v2 v0
v2 v3 v0 v3 v4 v0
47

48 7 RENDERING METHODS
for ( i n t i =0; i

7.3 GEOMIPMAP 49
7.3 GeoMipMap
456/.'
#/01+.'3
#+,01)2/3
!"#$ !"#% &''&''& ()*+,+-./
! "#$%&'()* +#$%&'()* ,#$%&'()*
GeoMipMap

8 Conclusion and Future Work
50

51

A Tutorials
A.1 The AutoSimServer kick start guide
Graphical User Interface of the AutoSim Server
52

A.2 WORKING WITH THE AUTOSIMCLIENT 53
A.2 Working with the AutoSimClient
AutoSimClient

54 A TUTORIALS
A.3 How to write a User Program
A.3.1 Workings of the User Program

A.3 HOW TO WRITE A USER PROGRAM 55
User Program
A.3.2 The User Program API
Parts section of a robot configuration file
namespace UserProgramAPI
{
SimDeviceError setData ( SimDeviceName device , DeviceData ∗data , i n t dataSize
) ;
SimDeviceError getData ( SimDeviceName device , DeviceData ∗data , i n t dataSize
) ;

} ;
56 A TUTORIALS
A.3.3 The Client User Program API
Parts section of a robot configuration file
namespace ClientUserProgramAPI
{
typedef unsigned i n t ∗ VirtualCameraImage ;
i n t getImageHeight ( ) ;
i n t getImageWidth ( ) ;
VirtualCameraImage getImage ( UserProgramAPI : : SimDeviceName camera ) ;
void unlockImage ( ) ;
} ;
A.3.4 A Simple Example
Parts section of a robot configuration file
#include " UserProgramAPI . h " / / i n c l u d e the user program API d e f i n i t i o n s
#include / / f o r the Sleep ( ) f u n c t i o n

A.3 HOW TO WRITE A USER PROGRAM 57
/ / f o r convenience : i n c l u d e the UserProgramAPI namespace
using namespace UserProgramAPI ;
/ / Entry p o i n t f o r the user program .
/ / Do not change argument l i s t or r e t u r n value !
i n t main ( i n t argc , char ∗ argv [ ] )
{
/ / device names of an a c t u a t o r and a sensor t h a t are defined
/ / i n the corresponding robot d e s c r i p t i o n f i l e
SimDeviceName i n d i c a t o r = " chassis . i n d i c a t o r _ l i g h t _ b a c k _ l e f t " ;
SimDeviceName i n c l i n o m e t e r = " chassis . i n c l i n o 0 " ;
/ / the robot name t h a t the user program belongs to i s
/ / always the f i r s t argument s t r i n g
RobotName robot = argv [ 0 ] ;
/ / v a r i a b l e s to s t o r e the data from the devices
f l o a t i n t e n s i t y = 0.0 f ;
f l o a t angle = 0.0 f ;
f l o a t previousAngle = 0.0 f ;
/ / get the c u r r e n t angle from the i n c l i n o m e t e r and make the
/ / l e f t i n d i c a t o r l i g h t b l i n k i f the robot t u r n s l e f t
while ( true )
{
GET_DATA( robot+" . " + i n c l i n o m e t e r , angle ) ;
i f ( ( angle − previousAngle ) < 0.0 f | | i n t e n s i t y == 1.0 f )
i n t e n s i t y = 0.0 f ;
else
i n t e n s i t y = 1.0 f ;
previousAngle = angle ;
SET_DATA( robot+" . " + i n d i c a t o r , i n t e n s i t y ) ;
Sleep (300) ;
}
return 0;
}

58 A TUTORIALS
A.4 Manipulate an OSM file in 6 steps
OsmManipulator

B The Configuration Files
B.1 General Syntax
Examplary Configuration file structure
< P o s i t i o n x= " 0.0 " y= " 0.0 " z= " 0.0 " / >
< / Box>
< / Parts>
< P o s i t i o n x= " 0.0 " y= " 3.0 " z= " −4.0 " / >
< / Camera>
< / GyroscopeSensor>
< / Devices>
< / Robot>
59

60 B THE CONFIGURATION FILES
B.2 Customizing a World File
•
•
•
•
B.3 General info on OSM Files

B.3 GENERAL INFO ON OSM FILES 61
Highways

62 B THE CONFIGURATION FILES
Landuse
B.4 The Robot File
House Node
Parts section of a robot configuration file
< P o s i t i o n x= " 0.0 " y= " 0.0 " z= " 0.0 " / >
< / Box>
< P o s i t i o n x= " 0.0 " y= " 0.0 " z= " 10.0 " / >
< / Box>
< S p h e r i c a l L i n k name= " towbar " >

B.4 THE ROBOT FILE 63
< Child name= " t r a i l e r " / >
< P o s i t i o n x= " 0.0 " y= " 0.0 " z= " 5.0 " / >
< / S p h e r i c a l L i n k >
< / Parts>
Devices section of a robot configuration file
< DriveActuator name= " drive_chassis " >
< / DriveActuator>
< DriveActuator name= " drive_chassis " / >
< P o s i t i o n x= " −0.80 " y= " −0.45 " z= " 1.35 " / >
< Steering value= " t r u e " / >
< / WheelDevice>
< P o s i t i o n x= " 0.0 " y= " 1.0 " z= " 1.4 " / >
< D i r e c t i o n x= " 0.0 " y= " −0.25 " z= " 1.0 " / >
< / Camera>
< / GyroscopeSensor>
< / Devices>

64 B THE CONFIGURATION FILES
•
•
•
•
•
•
•
•
•
•
•
•
•
•
B.5 The Map Setup File

B.5 THE MAP SETUP FILE 65
OsmSetup
< !−− Lat and lon coordinates of the s i m u l a t i o n world center−−>
< / OsmSetup>
RoadDimensions
< !−− S p e c i f y i n g how much ( i n meters ) the roads are l i f t e d above the t e r r a i n .
I f t h i s value i s set to low graphic problems may occur .−−>
< !−− Height of the curbs i n distance to the road lanes i n meters .−−>
< !−− Number of l e v e l s of d e t a i l constructed f o r the roads . At l e a s t 1 l e v e l
of d e t a i l i s constructed . −−>
< L e v e l s O f D e t a i l value= " 3.0 " / >
< / RoadDimensions>
HighwayDimensions
< t r u n k width = " 12.0 " leftPavementWidth = " 1.5 " rightPavementWidth = " 1.5 " / >
< t r u n k _ l i n k width = " 4.0 " leftPavementWidth = " 0.0 " rightPavementWidth = " 0.0 " / >
< p r i m a r y _ l i n k width = " 4.0 " leftPavementWidth = " 0.0 " rightPavementWidth = " 0.0 "
/ >
< t e r t i a r y width = " 6.0 " leftPavementWidth = " 1.5 " rightPavementWidth = " 1.5 " / >
< u n c l a s s i f i e d width = " 6.0 " leftPavementWidth = " 0.0 " rightPavementWidth = " 0.0 "
/ >

66 B THE CONFIGURATION FILES
< t r a c k width = " 3.0 " leftPavementWidth = " 1.5 " rightPavementWidth = " 1.5 " / >
< r e s i d e n t i a l width = " 8.0 " leftPavementWidth = " 1.5 " rightPavementWidth = " 1.5 " /
>
< s e r v i c e width = " 3.0 " leftPavementWidth = " 1.5 " rightPavementWidth = " 1.5 " / >
< / HighwayDimensions>
!
(
" #
'
Triangle Fan
Textures
< I n t e r s e c t i o n f i l e = " media / roads / lane_withoutmarks .JPG" / >
< / RoadTextures>
< TerrainTextures name= " c i r c l e _ t o w n " >
< !−− Texture f i l e name f o r the t e r r a i n t e x t u r e−−>
< !−− S p e c i f y i n g how o f t e n the t e x t u r e i s repeated on the t e r r a i n . Only has to
be changed to a value bigger than one
&
$
%

B.5 THE MAP SETUP FILE 67
i f no t e x t u r e f o r the complete t e r r a i n i s used . The used t e x t u r e should be
seamless then . −−>
< !−− Folder path and f i l e type of the skybox t e x t u r e s .
I n s i d e the f o l d e r have to be s i x t e x t u r e f i l e s of the declared f i l e t y p e :
1. l e f t . ∗
2. f r o n t . ∗
3. r i g h t . ∗
4. back . ∗
5. top . ∗
6. bottom . ∗ −−>
< / TerrainTextures>
TerrainDimensions
< !−− Size of the loaded t e r r a i n i n meters . The t e r r a i n has to be at l e a s t big
enough to load a l l the s t r e e t data of the OSM f i l e and to contain a l l the
t i l e s the g r a p h i c a l r e p r e s e n t a t i o n of the t e r r a i n i s made of ( can be set
i n the f o l l o w i n g values ) . A T e r r a i n bigger than 3000m∗3000m may e f f e c t
long loading times and slow physics . −−>
< !−− Size of one t i l e of the g r a p h i c a l t e r r a i n r e p r e s e n t a t i o n i n meters .
T i l e S i z e has to be a m u l t i p l e of HeightDataPerArea ! −−>
< T i l e S i z e value = " 64.0 " / >
< !−− D e t a i l value of the graphic t e r r a i n ’ s center t i l e . (maximum = 7) −−>
< L e v e l s O f D e t a i l value = " 3 . 0 " / >
< H e i g h t D i v i s i o n C o e f f i c i e n t value ="10.0" / >

68 B THE CONFIGURATION FILES
B.6 The House File List
File
House List
< r e s i d e n t i a l >
< F i l e path= " models / b u i l d i n g s / oldschoolhouse " type= " 3ds " / >
< / house>
< F i l e path= " models / b u i l d i n g s / r e s i d e n t i a l /2 " type= " obj " / >
< / house>
< / r e s i d e n t i a l >
< f o r e s t >
< t r e e name= " tree0 " >
< F i l e path= " models / nature / t r e e s / tree0 " type= " 3ds " / >

B.7 GENERAL MODEL FILE INFORMATION 69
< / t r e e >
< / f o r e s t >
B.7 General Model File Information

Bibliography
70

Bibliography 71