Multi Level Surface Mapping using Sparse Point Clouds - Student ...

Autonomous Systems Lab 

Prof. Roland Siegwart 

Semester-Thesis 

Multi Level Surface 

Mapping using Sparse Point 

Clouds 

Spring Term 2010 

Supervised by: 

Stephan Weiss 

Rudolph Triebel 

Author: 

Pascal Strupler

Contents 

Abstract 

Symbols 

iii 

v 

1 Introduction 1 

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 

1.2 Goals and limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 

1.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 

2 Introduction to the MLS framework 3 

3 Adaptions and optimizations 7 

3.1 Median filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 

3.2 Region growing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 

3.3 Orphan points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 

3.4 Pyramid artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 

3.5 Hole plastering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 

4 Integration into ROS 23 

4.1 Module layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 

4.2 MLS module usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 

5 Simulation and testing 25 

5.1 Recorded data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 

5.2 Artificial setups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 

6 Result discussion 31 

6.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 

6.2 Computational costs . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 

6.3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 

6.4 Conclusion and outlook . . . . . . . . . . . . . . . . . . . . . . . . . 37 

Bibliography 39 

i

Abstract 

The sFly project at the Autonomous Systems Lab, ETH Zürich is all about quadrotors 

navigating and mapping urban environments. In order to perform the mapping 

task satisfactorily, we have to implement an appropriate mapping framework. A 

first goal of this semester thesis was to adapt a multi level surface framework (originally 

designed to work with laserscan data) to process inaccurate sparse pointclouds 

delivered by the quadrotors SLAM algorithm. In the beginning the originally provided 

MLS framework had to be optimized concerning outliers and sparsity. I 

therefore implemented three outlier rejection algorithms, including a multiple level 

median filter processing incoming pointclouds. In order to deplete the second issue 

of sparsity, a convenient hole plastering algorithm was also used. As a second goal 

we wanted to retain the benefit of the current quadrotors modular operating system 

ROS. Therefore the original MLS framework was integrated into a ROS package, 

providing a modular, universal mapping system also capable of doing sequential 

online mapping. As a result of this work, we are able to do online multiple level 

surface mapping. Even sparse pointclouds can deliver an impression of the environment 

while marking areas of high uncertainty. Latter areas could then be explored 

by flying closer in order to systematically map the environment. 

iii

Symbols 

Acronyms and Abbreviations 

ETH 

MLS 

ASL 

ROS 

SLAM 

sFly 

EMAV 

Eidgenössische Technische Hochschule 

Multi Level Surface 

Autonomous Systems Lab, ETH Zürich 

Robotic Operating System 

Simultaneous localization and mapping 

Swarm Fly, a MAV project by a consortium including ETH Zürich 

European conference and competition of micro aerial vehicles 

v

Chapter 1 

Introduction 

1.1 Motivation 

At the sFly project at ETH Zürich the goal is to develop several small and safe 

quadrotors which can fly autonomously in urban environments and which can be 

used to assist humans in tasks like rescue and monitoring. The MAVs are stabilized 

by estimating the degrees of freedom using a downward mounted video camera. The 

estimation is taken from a visual SLAM (Simultaneous Localization And Mapping) 

framework. The SLAM task consists of several sub tasks such as detection of 

points of interest in the image, finding these points again in a next camera image 

and estimating the actual degrees of freedom out of these points. Besides from 

estimating the attitude of the quadrotor, we can also use the resulting pointcloud 

of the SLAM algorithm to map the environment. Mapping yields two key benefits: 

First of all we can generate a map for visualization of the environment. Second we 

can use the resulting mapping for path planning, which is essential when navigating 

the quadrotor through unknown terrain. 

Up until now both attitude estimation and mapping could be implemented. Preliminary 

work at ASL used elevation maps on the pointcloud to represent the topology 

of a mapped area. This has the drawback that underpassings cannot be modeled. 

To overcome this issue, the motivation of this work was to implement multi level 

surface (MLS) mapping as proposed in the paper Multi-Level Surface Maps for 

Outdoor Terrain Mapping and Loop Closing [1]. As a result, the sFly quadrotor 

could therefore get the possibility to fly through openings like windows or underpassings 

of bridges, which wouldn’t be possible with the current elevation map. An 

additional benefit of the proposed MLS map would be the better representation 

of vertical surfaces. Walls as an example could be modeled with vertical surface 

patches, which is not possible with an elevation map implementation. Here walls 

can only be approximated by inclined surface patches. 

Urban environments are based on highly multilevel structures like buildings, bridges 

or tunnels. Consequently the success of the sFly project relies on a working implementation 

of multi level surface mapping. 

1.2 Goals and limitations 

The MLS maps framework proposed in the paper [1] has already been implemented 

at the ASL of ETH Zürich. It was successfully used to do mapping and SLAM loop 

closing with pointclouds from a laserscan device. In the sFly project pointclouds 

are produced by finding correspondences of feature points among subsequent video 

1

Chapter 1. Introduction 2 

frames. Hence our pointclouds differ substantially from pointclouds generated by a 

laserscan device concerning density and accuracy. 

While laserscan devices have constant and relatively high resolution, cloud points 

extracted from the video feed of the quadrotor are irregular and generally sparse. 

This irregularity depends on several factors like the quality of the video feed, the 

way the environment is textured and composed, light conditions, the maximum 

number of processed feature points and so on. 

Another problem is the feature point correspondence. This matching is often wrong 

and leads to false coordinate values (mostly in z-direction) and therefore produces 

a lot of outliers in the resulting pointcloud. 

The first goal of this work was to implement algorithms to optimize the MLS framework 

for pointclouds with such imperfections mentioned before. As we will see later, 

rejecting outliers and dealing with the sparseness will be crucial for improving the 

quality of the resulting maps. 

The sFly quadrotors are running on an open-source, meta-operating system called 

ROS (Robot Operating System [3]). It provides services like hardware abstraction, 

low-level device control, implementation of commonly-used functionality, messagepassing 

between processes, and package management. Modules are then performing 

different tasks while also communicating among each other through topics or services. 

Hence another goal of this thesis was to create a ROS package of modules 

which can be used by the sFly project to convert generated pointclouds into multi 

level surface maps. The optimized MLS framework will then be included in this 

package to provide the necessary functions. The last objective was to have sequential, 

online mapping: The quadrotor constantly expands the pointcloud with new 

points while flying around. Batches of those new points have to be converted into 

small, partial MLS maps which then get added to the global MLS map. Of course 

it’s preferable to achieve this on some realtime basis. 

Due limited time, it wasn’t part of this thesis to study dynamic map updates. For 

example handling mapped cars which drive around and appear at another place 

over time. Only static environments were considered. It was also never planned to 

do actual live tests having a quadrotor flying around. Through the whole thesis I 

worked with pre-recorded data from the SLAM algorithm. 

1.3 Methodology 

This report is organized as follows: In the first chapter Introduction to the MLS 

framework on page 3 I will give a short introduction into the existing MLS framework 

used for laserscan data. I will point out the problems encountered by just processing 

inaccurate, sparse pointclouds from the quadrotor with this original framework. 

Then I optimized the framework by implementing some algorithms for outlier 

rejection and dealing with the sparseness of the pointcloud. These implementations 

and its parameter settings will be explained in detail in chapter Adaptions and optimizations 

on page 7. Afterwards I worked on integrating the framework into a 

ROS package in order to test sequential, online mapping and later have a simple solution 

to replace the sFly projects current mapping package. I described this work 

in chapter Integration into ROS on page 23. Finally I could simulate the whole 

mapping package with some recorded and artificial pointcloud data which is shown 

in chapter Simulation and testing on page 25. The result- and problem-discussion of 

this work as well as an outlook will be done in chapter Result discussion on page 31.

Chapter 2 

Introduction to the MLS 

framework 

There are different solutions for mapping in 3D environments. Until now, elevation 

maps were implemented for the sFly project. But this was never a definite solution. 

The problem with elevation maps is the constraint of having only one connected 

surface. Like that we can’t model any sort of underpassings like bridges, tunnels, 

balconies and so on. But those elements are very common in urban areas we would 

like to map and to do path planning in the future. Imagine for example a simple 

obstacle like a crane. If the quadrotor approaches from the top and maps the crane 

arm, the resulting surface would build an impassable wall from the ground up to 

the crane arm. Even if we would measure additional points of the ground below 

the crane, we’d had the choice of either discarding the ground, discarding the crane 

arm or connect ground points with crane points through several, almost vertical 

surfaces. Another disadvantage is the lack of vertical surfaces in elevation maps. 

Because of this limitation we wouldn’t be able to map walls and other vertical 

surfaces satisfactorily. 

Therefore we had to come up with another type of map which supports more than 

just one height-level of surfaces and is capable of modeling vertical walls. Grid based 

maps like 3D occupancy maps (like proposed in [2]) or multi-level surface maps are 

two of several possible choices. In 3D occupancy maps the whole space is partitioned 

into cubic 3D cells. Each of those cells indicates the probability of occupancy by 

some obstacle. The multi-level surface map described in [1] also partitions its space 

using a cell grid, but only in x and y direction. Surface patches in 3D space are 

then defined by the x and y coordinates of the cell and a continuous height value 

in z direction. In order to represent surface patches with a thickness (cuboids) 

an additional value for depth defines the surface patches thickness underneath it. 

Compared to 3D occupancy grids, this map design features two main advantages: 

We can save storage because we don’t have 3D cells with a value in empty spaces and 

we can represent several, vertically connected and occupied 3D grid cells by just two 

values - the upper end as z-value and the depth. Besides from that, MLS maps have 

a continuous z-space and can therefore map the environment more accurately in this 

direction. Solving the problems of elevation maps and having clear advantages over 

3D occupancy maps we decided to use the multi-level surface maps discussed in [1] 

for the sFly project. 

The whole MLS framework was already implemented by Rudolph Triebel and is 

working well on pointclouds generated by laser scanners. Directly processing pointclouds 

from the sFly SLAM algorithm with it however doesn’t work that well. In the 

image 2.2 you can see that the resulting map visualization of a recorded pointcloud 

3

Chapter 2. Introduction to the MLS framework 4 

(image 2.1) is quite poor. There is just too much gaps and unconnected triangle 

surface patches to recognize anything of the captured environment. This issue has 

mainly three reasons: 

1. Sparseness of the pointcloud: The sFly SLAM algorithm can only extract 

feature points in distinct areas in the images like edges or corners with high 

contrast. Now depending on the image quality and the structure of the captured 

environment, there may be areas with no or only few extracted feature 

points, which leads to gaps and holes in the pointcloud. We also have a maximum 

number of extracted points per frame; hence as we fly away from the 

ground or objects, the number of extracted features per area decreases. Unlike 

the sFly SLAM a laserscan device always returns the same amount of 

measurement, which is often a lot higher than the number of tracked features 

in a camera image. Also the angle between measurements is constant and independent 

of the environment, which leads to a pointcloud with more evenly 

spread points. 

2. Outliers: When the sFly SLAM searches for correspondent feature points in 

subsequent frames, it can happen that this matching is wrong. As a consequence, 

the calculated distance of the point to the camera will be wrong. Due 

to the fact that the wide-angle camera of the quadrotor is mostly directed 

downwards during flight, this wrong value will affect the z-value of a point for 

the most part. 

3. Visualization: The map isn’t that sparse like it appears in image 2.2. As 

you can see in another representation of the map in image 2.3, there is a lot 

more data given. The cause of this lies in the visualization algorithm used 

by the framework. The polygon patches you can see in 2.2 are actually three 

connected neighboring surface patches. If there are less than three neighboring 

surface patches, nothing will be displayed in the resulting map. 

Therefore in the first part of my thesis, I extended the framework with several 

algorithms in order to reduce the first two issues. I implemented a median filter, an 

orphan point algorithm, and a pyramid artifacts algorithm for outlier rejection as 

well as a hole plastering algorithm to reduce sparseness of the map. Region growing 

is also implemented and was intended to identify planes for easier outlier rejection. 

In the end it wasn’t used because of the good performance of the median filter. 

These mentioned algorithms will be explained in detail in the next chapter. 

Figure 2.1: Given pointcloud

5 

Figure 2.2: Unoptimized surface map 

Figure 2.3: Unoptimized surface map, alternative visualization with the surface 

patches as black dots with its depths as orange lines

Chapter 2. Introduction to the MLS framework 6

Chapter 3 

Adaptions and optimizations 

3.1 Median filter 

One of the most important filters for outlier rejection is the non-linear median 

filter. It’s often used to reduce noise, while preserving edges. Like described in the 

previous chapter, the errors in the pointclouds from the sFly SLAM will be mostly 

in z-direction. In the image 3.1 we can see how a 1D pountcloud is affected by a 

median filter with a window width of 5. The window is iteratively slided over the 

original pointcloud on the left side of the image. At each position, the height value 

of the actual point and its two neighbors on each side are saved into an array. Next 

we sort the values and then assign the height value in the middle of the array to 

the current point. The result can be seen in the image 3.1 on the right. The planes 

contain less noise but edges are preserved. 

Figure 3.1: Principle of median filtering 

Now in our 3D pointcloud, this filter is a bit more complicated as we have to 

consider several planes on top of each other, but the main principle stays the same. 

A flowchart of the algorithm is shown in image 3.2. We cycle through all points in 

our pointcloud and consider neighbors inside a certain radius nbRadius. In this list 

of neighbors we only keep the closest points up to a certain number maxNb. On 

the other hand, if there are too few neighbors (less than minNb), we won’t consider 

this point for our map any longer - it’s quite certain that this is an outlier and 

the neighborhood is too sparse to do median filtering. Once the closest neighbors 

are found, we sort them according to their height values. Finally we can assign 

the value in the middle of the sorted array to the current point and save it into a 

new pointcloud. In the end, when we have processed all points, the resulting new 

pointcloud will be median filtered and ready to be converted into a MLS map. An 

overview of all parameters and its functions is given in table 3.1 

Next, we will see some examples of how good the median filter performs. In order to 

test the performance, I implemented a test module to synthesize pointclouds. This 

module is explained in detail in the chapter Simulation and testing on page 25. In 

7

Chapter 3. Adaptions and optimizations 8 

Figure 3.2: Flowchart median filtering 

the image 3.3 such an artificial pointcloud with 10% outliers is given. The resulting 

map without median filtering on the right of the image is quite noisy and the craggy. 

If we process this type of pointcloud by the MLS framework with an active median 

filter, the resulting map (image 3.4) looks much more promising. The planes are 

smoother and don’t have any sharp peaks. However edges stay sharp in contrast to 

linear filters like the gaussian. 

We can look at a second visualization of the map with surface patches just displayed 

as points and its depths by lines. The advantage of this visualization is more its 

completeness than the benefit of recognizing the environment with surfaces. All 

surface patches and depths are displayed here, while in the original visualization 

surface patches with less than two direct neighboring patches won’t show up. Especially 

outliers won’t be displayed although they are present in the map. Also the 

visualization of depths with polygons is hard and therefore often not complete. The 

image 3.5 shows a comparison of median filtering with this alternative visualization. 

The left map without median filtering shows a lot more outliers than the one on the

9 3.1. Median filter 

Type Variable Description 

double nbRadius Defines the maximum radius, in which neighbors 

of the current point are extracted. 

uint maxNb If the neighborhood inside nbradius contains more 

than maxnb points, we just get the maxnb closest 

ones. 

uint minNb If the neighborhood inside nbradius contains less 

than maxnb points, we assume the current point is 

an outlier an therefore won’t consider it for building 

the map. 

Table 3.1: Median filter parameters 

Figure 3.3: MLS map without median filtering 

right. There are also more and larger dephts produced without median filtering. 

Finally we can double the outliers (20%) to show, that the mapping still looks 

relatively good. The result is shown in the image 3.6. 

Figure 3.4: MLS map with median filtering


Figure 3.5: MLS maps without median filtering on the left versus median filtering 

on the right 

Figure 3.6: Median filtering with double the amount of outliers

11 3.2. Region growing 

3.2 Region growing 

At the start it wasn’t sure if the median filter would perform good enough for 

multiple level pointclouds. Therefore I also implemented a region growing algorithm 

in order to identify planes as different regions. With these regions, it probably could 

be easier reject outliers to the nearest plane. However since the performance of the 

original median filtering was satisfying, I didn’t use the implemented region growing 

any further and therefore saved additionally required computing power. 

Still I will explain the principle of the algorithm in case future users of this framework 

like to use it. However due to visualization problems the algorithm couldn’t 

yet be tested and can’t be shown in an example. The corresponding flowchart is 

shown in image 3.7, a table of parameters in 3.2. 

The region growing algorithm works on the surface map. While iterating through 

all map cells and its containing surface patches we look for patches without a yes 

assigned region. Lets call this iteration the main loop. If such a patch is found, we 

push it onto a stack and also create a new current region. Now the algorithm enters 

a stack-loop where it pops a patch and compares the assigned region of that patch 

with the current region: If the patch wasn’t assigned a region yet (which is the 

case in the first loop iteration), it gets assigned to the current region. Additionally 

we also add all its directly neighboring surface patches in x and y direction within 

a certain height threshold heightThresh to the stack. Like that, we can let the 

region grow into new areas. If the patch has another region assigned, we replace 

the current region with that discovered former region and reassign all patches of 

the current region. This is basically region merging. with the last possibility of the 

popped patch having the same region assigned as the current one, we don’t have 

to do anything. Now we can again pop the next patch from the stack and compare 

again. If the stack is empty, we return into the main-loop and look for patches 

without any assigned region. 


double heightThreshold Neighbors of a patch are extracted within +- 

heightThresh in z-direction 

Table 3.2: Region growing parameters


Figure 3.7: Flowchart region growing

13 3.3. Orphan points 

3.3 Orphan points 

This simple algorithm is an addition to the outlier rejection, that was already done 

on the pointcloud by removing points which had to few neighbors inside a certain radius. 

The orphan point algorithm however is applied to the surface map. We simply 

iterate through all map cells and their surface patches and remove patches, which 

haven’t got any neighbors inside a certain box. The dimensions of this search-box 

are defined by the two parameters vertical and horizontal such that the dimensions 

are 2 · horizontal × 2 · horizontal × 2 · vertical (length x width x height). 

A corresponding flowchart is shown in image 3.8 and the parameters are listed in 

table 3.3. 

Figure 3.8: Flowchart orphan points algorithm 

In the following example, the first pointcloud displayed from the side in image 3.9 is 

processed by the MLS framework without orphan points rejection (in order to show 

the effect of the algorithm, median filtering is inactive in both examples). Of course 

those outlier-points which weren’t merged with other points to surface patches with 

depth are still present in the surface maps. With the orphan points algorithm



double horizontal Half the width of the surface patch search-box, in 

which the algorithm searches for neighbors. 

double vertical Half the width of the surface patch search-box, in 

which the algorithm searches for neighbors. 

int min x Restricts the algorithm to a certain sub map by 

setting a limit in the negative x direction. 

int max x Restricts the algorithm to a certain sub map by 

setting a limit in the positive x direction. 

int min y Restricts the algorithm to a certain sub map by 

setting a limit in the negative y direction. 

int max y Restricts the algorithm to a certain sub map by 

setting a limit in the positive y direction. 

Table 3.3: Parameters orphan points algorithm 

turned on, most of the outliers disappeared in the surface map (image 3.10). The 

search-box dimensions are shown in the illustration of the surface map by a red 

square. Don’t forget about the third dimensions - the surface patches which still 

look like outliers probably have another patch placed behind them. 

Figure 3.9: Pointcloud and surface map structure without orphan points algorithm 

Figure 3.10: Pointcloud and surface map structure with active orphan points algorithm

15 3.4. Pyramid artifacts 

3.4 Pyramid artifacts 

This algorithm is the last tweak I implemented to reject outliers. While testing 

I noticed some pyramid-like artifacts which were under some circumstances still 

present in the map (despite median filtering and the orphan points algorithm). 

Hence with this algorithm I tried to just address this local map issue and not to 

implement another global median filter. A flowchart is shown in image 3.11 and the 

parameters are listed in table 3.4. 

The principle of the pyramid artifact algorithm is quite simple: First we just look 

out for surface patches which have all 4 direct neighbors in ±x and ±y directions. If 

that’s the case, we compare the height difference of the current surface patch and the 

highest neighbor with the height difference of the highest and the lowest neighbor. 

If the first difference is larger than heightFactorThresh times the difference among 

the neighbors, the surface patch height gets adjusted. Currently the height of the 

current patch is thereby set to the same height of the highest neighbor. 

Figure 3.11: Flowchart pyramid artifacts algorithm



double heightFactorThresh The height-difference of a point with its highest 

neighbor has to be at least heightFactorThresh 

times larger than the difference between the highest 

and the lowest neighbor in order get adjusted. 

The neighborhood of a point is defined by its 4 

direct neighbors in ±x and ±y directions. 









Table 3.4: Parameters pyramid artifacts algorithm

17 3.4. Pyramid artifacts 

The following example in image 3.12 shows a surface map with such pyramid artifacts, 

not optimized by the algorithm. With the parameter heightFactorThresh set 

to 1.5 and the pyramid artifacts algorithm active, the resulting map is free of those 

flaws as can be seen in image 3.13. The algorithm really just address those specific 

structures - for example there are no affected edges or corners of the building 

structure. 

Figure 3.12: Surface map without pyramid artifacts algorithm 

Figure 3.13: Surface map with active pyramid artifacts algorithm


3.5 Hole plastering 

While the former four algorithms were all about rejecting outliers, this algorithm 

addresses a different problem. Described in the chapter Introduction to the MLS 

framework on page 3, due to sparseness and visualization difficulties a map can 

contain a lot of gaps and holes. With this algorithm it’s possible to fill these gaps 

in two levels of precision. First the gaps, which are smaller than the quadrotor, can 

be closed unquestionable - the quadrotor couldn’t fly through anyway. We can also 

fill up larger gaps to get a feeling of the map. In order to justify this method, we 

mark all surface patches which where filled in, such that we can still return later 

and get a more detailed view of this region. 

To detect these gaps, the algorithm primarily iterates over all map cells and its surface 

patches. At each surface patch we then check immediate map cells in x direction 

for candidate surface patches. Candidate surface patches lie inside a certain height 

interval from the height of the starting patch. This interval on the other hand is 

calculated using the current gap distance and the two parameters heightThresh and 

heightClearance: Interval boundaries = Height of starting patch ± heightThresh · 

gapDistance + heightClearance. An illustration in image 3.14 shows this search. If 

a surface patch is found inside this interval, it depends on gapDistance if we can 

close this gap. gapDistance has to be larger 1 (otherwise it isn’t a gap at all) and 

smaller equal the maxGap parameter. As soon as we found a gap fulfilling our 

requirements, we can close it by inserting new surface patches along a constant 

gradient. If the algorithm is on its second run to close larger gaps of width max- 

GapEnhanced, we also mark these inserted patches by setting FlagArtificial equal to 

1. Finally, the algorithm also checks for gaps in y direction. The resulting flowchart 

is shown in image 3.15 and the parameters are listed in the table 3.5. 

Figure 3.14: Hole plastering search principle

19 3.5. Hole plastering 

Figure 3.15: Flowchart hole plastering algorithm



double heightThresh Defines the maximum gradient of the with which 

the gap can be closed. 

double heightClearance Defines the minimum vertical clearance of the gap 

to close to other surface patches. 

double maxGap Defines the maximum gap width to be closed. 

double maxGapEnhanced Defines a larger, secondary gap width to be closed. 

However the surface patches, which fill these gaps 

are marked by setting the flag SetFlagArtificial to 

1 









Table 3.5: Parameters hole plastering algorithm

21 3.5. Hole plastering 

To show how the hole plastering algorithm performs, I generated a small pointcloud 

showing a part of a house and a ground. Both ground and house contain a hole of 

the size 0.7 × 0.7. The average distance from point to point is 0.1 (grid size). This 

pointcloud is shown in image 3.16. 

Figure 3.16: Pointcloud for hole plastering tests 

Now we let this pointcloud be processed by our MLS framework with two different 

settings. Everything is identical except for the maxGap parameter: During the first 

run it’s set to 0.4, afterwards it’s increased to 0.7. The cell size of the surface map 

is 0.15 in both runs. The resulting maps can be seen in image 3.17. All inserted 

surface patches are marked with red dots. As one can see in both runs some small 

gaps, where only one surface patch was missing, are closed. In the second run, the 

maxGap parameter also allowed to fill up the large hole in the ground. But what 

about the hole in the wall of the house That’s a limitation of the hole plastering 

algorithm. As soon as several points in the pointcloud end up on top of each other 

in one MLS cell column, the MLS framework converts them into one surface patch 

with a depth. But hole plastering algorithm can only close gaps among surface 

patches, not depths. Therefore in none of the two runs this hole in the wall gets 

closed. This is surely an expandability of the algorithm. 

We can now push the algorithm to it’s limit and once again generate the pointcloud, 

but this time with an average point distance of 0.4 instead of 0.1. Like that we get 

a quite sparse pointcloud with 64 times less the amount of points compared to our 

first examples. The only setting changed for the hole plastering algorithm is the 

slightly increased maxGap parameter, now 0.75. The result in image 3.18 shows 

still a quite complete map, although a lot of surface patches were inserted. The 

shape of the roof however wasn’t reconstructed perfectly, the pointcloud is just too 

sparse to feature such shapes.


Figure 3.17: Left: Holeplastering with maximum gap size of 0.4, right: Holeplastering 

with maximum gap size of 0.7 

Figure 3.18: Holeplastering with maximum gap size of 0.75, sparse pointcloud with 

4 times less points

Chapter 4 

Integration into ROS 

4.1 Module layout 

As already mentioned in the introduction, one of the goals was to integrate the MLS 

framework into a ROS module and making it work for sequential online mapping. 

Like that it’s easy, to add this mapping to the existing quadrotor project, which 

also runs on ROS modules. ROS modules consist of nodes communicating among 

each other or with nodes from other modules. There are basically two ways of 

communicating: First, a node can broadcast messages to so-called topics, which 

in turn can be read by other nodes. And second, there is also a request/reply 

communication by using ROS services. Further information is available on the 

ROS website [3]. 

While in flight, the quadrotors SLAM algorithm extracts and processes feature 

points from the video stream. It also groups them locally and indexes these subpointclouds 

with keyframes. We decided to use these keyframe-groups to split the 

whole pointcloud and publish sub-pointcloud on-the-go to the MLS module. Of 

course the SLAM algorithm will accumulate keyframes independently of the work 

of our MLS module. Due to this unidirectional relation, we decided to use a topic 

for publishing keyframe-pointclouds online to our MLS module. As a message type 

for pointclouds, the common package geometry msgs is used. It supports publishing 

arrays of points (with each having a x, y and z value). 

Before ROS integration the optimized framework was tested using recorded pointclouds. 

We couldn’t do any live testing on the quadrotor, so I always had to rely 

on offline data. With ROS integration I wanted to simulate online, sequential 

mapping and therefore had to build another node which provided the MLS node 

periodically with data. In order to use the recorded offline data, a parser node 

(called PcParser) was implemented, which reads such an input file and continuously 

publishes keyframe pointclouds at certain time intervals. If the number of 

points grouped together to a keyframe pointcloud is too small, one can also set a 

minimum size for published pointclouds. Like that, keyframes are cumulated until 

the minimum point count is reached and then published. You can see the final 

ROS testing setup the image 4.1. More about testing will be explained in the next 

chapter simulation and testing []. 

Our central node, which is called MapBuild, will subscribe to this pointcloud topic 

and sequentially receive pointclouds published by the node PcParser. With every 

new received pointcloud a function processCloud is called to apply optimizations 

to it and finally attach it to a global map, maintained by MapBuild. The detailed 

sequence of action is as follows: 

1. The function processCloud is called as soon as MapBuild receives a new point- 

23

Chapter 4. Integration into ROS 24 

Figure 4.1: Diagram of ROS setup 

cloud from the topic. 

2. In processCloud, the pointcloud gets median filtered first (function MedianizeAll). 

3. A temporary surface map is generated from the pointcloud. 

4. The temporary surface map is added to the global surface map maintained by 

MapBuild. 

5. Orphan points are removed from the map (function RemoveOrphanPoints). 

This function is restricted to the area where the new pointcloud was inserted. 

6. Gaps and holes are filled up in the map with hole plastering (function Hole- 

Plastering). This function is also restricted to an expanded area where the 

new pointcloud was inserted. The expansion of the area is equal to the maximum 

gapsize to be closed. 

7. Pyramid artifacts are removed in the area where the pointcloud was inserted 

(function RemovePyramids). 

4.2 MLS module usage 

The modules are programed in such a way, that the most important parameters can 

be set in a so called launch file. Once the parameters are set, one can initiate the 

ROS core with the these parameters with by typing. 

\verb|roslaunch mlsmaps paramfile.launch| 

Afterwards we just have to start the nodes MapBuild with the command 

\verb|rosrun mlsmaps MapBuild| 

Like that the MLS module is ready to read pointclouds from the topic. The PcParser 

node, which parses recorded test pointclouds from files can then be launched with 

\verb|rosrun mlsmaps PcParser| 

For more information about ROS, visit its website on www.ros.org [3].

Chapter 5 

Simulation and testing 

Testing the MLS framework and the ROS module with pointclouds was essential 

during my thesis and also consumed a lot of time. During the whole time it wasn’t 

possible to perform real tests with a flying quadrotor and true live data. Therefore 

in the beginning I was given three pointclouds which were recorded during former 

flights. Later I additionally generated artificial maps for different scenarios. Prior 

to showing results in the next chapter, I will illustrate the used pointclouds in this 

chapter. 

5.1 Recorded data 

The first pointcloud I used contains a table and a floor (see image 5.1). In order to 

get that much points on indoor, plain surfaces, one had to lay newspapers all over 

the floor and table. Like that SLAM could detect more features and therefore draw 

more points, but there was still a lot of wrong correspondences as one can see in 

the pointcloud image: There are a lot of outliers above table and ground and also 

some individuals even further away and outside the captured image area. Because 

of these circumstances, I used this map in the beginning to test my optimization 

algorithms for outlier rejection. 

Figure 5.1: Table with floor pointcloud 

The second pointcloud was recorded during an EMAV competition. The floor in this 

case was from a gym and therefore it would have been also hard to get any feature 

points. As help, the organizers rolled out two bands for easier feature detection. 

The quadrotor had to fly along these bands and at the end also through a window 

as an obstacle passage. I used this map to optimize the hole plastering algorithm, 

25

Chapter 5. Simulation and testing 26 

since there were a lot of featureless areas between the bands and therefore gaps in 

the pointcloud (see image 5.2). It’s also interesting to see, what would happen to 

the window ledge at the end of the pointcloud. 

Figure 5.2: EMAV competition pointcloud with feature bands and window ledge 

The last and most important pointcloud resulted from a flight near Birmensdorf, 

ZH. Due to safety reasons, the quadrotor flew over a village-like but uninhabited 

environment. The captured area features a lot of landmarks like roads, houses (also 

destroyed ones), a creek and a bridge (see aerial image 5.3 of Google Maps). In 

the pointcloud shown in image 5.4, one can probably recognize partial sparseness - 

especially areas along the creek and the roofs of the houses contain almost no points. 

The lack of roofs are thereby mostly originated from bad video quality: They were 

overexposed and therefore white in the video, such that there was no chance of 

extracting any features. Besides from that points seem to be randomly distributed 

and there are also outliers. Overall it’s difficult for us to recognize landmarks in this 

pointcloud. I spent most of the time testing with this pointcloud, as it is the most 

complex and realistic setup for future sFly mapping goals in city-like environments. 

I will also show most of the test results with this map, as it features a lot of problems 

I address in this thesis.

27 5.1. Recorded data 

Figure 5.3: Birmensdorf aerial view 

Figure 5.4: Birmensdorf pointcloud with road, creek and houses

Chapter 5. Simulation and testing 28 

5.2 Artificial setups 

The three recorded pointclouds I was given were good for testing the optimizations 

for the existing MLS framework with real data. But there is one major drawback: 

None of them contains real multi level surface properties. In the first pointcloud 

with the table and the floor, the table is a little bit overlapping, but not much. In 

the other pointcloud sets there are no multiple levels at all. Therefore I decided 

to add a test mode to the PcParser node. In this test mode, an algorithm should 

assemble pointclouds by combining horizontal and vertical surfaces. The functions 

for generating these horizontal and vertical surfaces (functions addHorizontal and 

addVertical) offer a variety of parameter settings like position, size, orientation, 

grade and the gaussian distribution in horizontal and vertical direction for placing 

the points around its grid points. Latter parameter simulates simple noise from 

SLAM triangulation. Through global class parameters one can also set the grid size 

and the ratio of salt & pepper noise - meaning the chance of placing a point with 

5 times the normal gaussian displacement in z-direction. Like that we can simulate 

salt & pepper noise in z-direction, which was also noticed in the real, recorded 

pointclouds. 

The function TestPointCloud can now assemble different pointcloud with these 

surfaces. Besides a parameter pcType to choose the type of the pointcloud, the 

function features an additional parameter addParam to set one crucial dimensional 

aspect of the pointcloud like the size of a gap or the height of a bridge. pcType can 

be set with the following numbers: 

1. A simple bridge - a horizontal surface situated over a valley of surfaces. addParam 

defines the height of the bridge. 

2. A more realistic bridge which additionally features walls and therefore has 

a more narrow underpassing (image 5.5). addParam defines the height of 

the bridge. This pointcloud was mostly used to test median filtering and 

visualization of an underpassing. 

3. Some ground and a part of the house. This pointcloud features a horizontal 

hole in the ground and a vertical one in the house wall (image 5.6). The 

diameters of the holes can be set with addParam. This map was used for 

testing the hole plastering algorithm and taking a look at the visualization of 

walls. 

4. Three planes, one of them tilted (image 5.7). This pointcloud was also used 

for median filtering tests. 

5. Three planes, one of them tilted. The planes are connected by walls. 

Figure 5.5: Complex bridge pointcloud

29 5.2. Artificial setups 

Figure 5.6: Pointcloud of house and floor with holes 

Figure 5.7: Pointcloud of three planes

Chapter 5. Simulation and testing 30

Chapter 6 

Result discussion 

In this last chapter, I will show the resulting MLS maps of the three given recorded 

pointcloud sets. I will also point out problems I’ve encountered and couldn’t yet 

be solved mostly due to lack of time. Finally a conclusion of my thesis and an 

interesting outlook will be given. 

6.1 Results 

We’ve already seen some partial multi level surface results for each of the implemented 

optimization algorithms. Now the most interesting part is to see how these 

optimizations perform together and transform the three recorded pointclouds into 

MLS maps. All three pointcloud sets were presented in the chapter Simulation and 

Testing on page 25. I’d like to start with the Birmensdorf set. Once again, one 

can take a look at the pointcloud which is illustrated in image 5.4 on page 27. In 

order to compare the improvements later, the MLS map which was created by the 

original MLS framework and is shown again in image 6.1. As you can see, there 

are a lot of randomly arranged polygon patches. There is no chance of someone 

recognizing anything in this map. We have to consider however, that the map contains 

more surface patches as one can see in the MLS map (like mentioned in the 

beginning, surface patches with less than two neighbors can’t be connected to a 

triangle polygon and therefore won’t be visible). The alternative visualization was 

shown in image 2.3, page 5. 

Figure 6.1: Surface map of Birmensdorf set without any optimizations 

If we now let the pointcloud be converted by our ROS MLS module, the result in 

image 6.2 looks better, but not yet satisfying. That’s because of the hole plastering 

algorithm which until now only fills up gaps smaller than out quadrotor. Since all 

measurements from the sFly SLAM algorithm are relative, we have to guess the 

31

Chapter 6. Result discussion 32 

quadrotors width by analyzing the map. In this Birmensdorf set, the quadrotor 

seems to be 0.15 wide - about the same size as the cells in the surface map. With 

maxGap set to 0.15, the hole plastering algorithm will therefore only close gaps of 

one cell size width. Now to still get a good feeling of how the environment looks 

like, the maxGapEnhanced parameter is set to 0.75 to also close gaps which are up 

to 5 cell sizes wide. This final result is shown in image 6.3. 

Figure 6.2: Surface map of Birmensdorf set with partial hole plastering 

Figure 6.3: Surface map of Birmensdorf set with all optimizations, viewed from two 

different perspectives 

In this final result we can see a lot of red points. Those points mark the surface 

patches, which were additionally inserted by the hole plastering algorithm since the 

last shown MLS map output. So all these patches were used to plaster holes and 

gaps which were larger than 0.15 and smaller equal 0.75. In the MLS framework, 

these surface patches are also marked for later use with a flag called FlagArtifical. 

With this marking we can always keep track of areas which were inserted unjustly. 

It’s possible, that those really are closed areas, but they could also be open. And 

based on the fact, that these closed gaps were larger than the quadrotor, it would

33 6.1. Results 

be able to fly through. Therefore we call these marked areas regions of uncertainty. 

The benefit of trying to close these large gaps is obvious: For the first time we 

are able to recognize the environment the quadrotor flew through. The green line 

represents the route of the quadrotor projected to the ground. Blue marks the 

depression of the creek, once interrupted by a bridge. And finally buildings (houses 

and ruins) are marked with orange color. 

As you compare the map with the aerial view shown in image 5.3 on page 27 you 

might notice, that the roofs of the houses are missing (except for the first house on 

the right after the curve). This is due to an overexposed video recording of these 

roof areas. I assume the sun was reflecting on them, such that the whole roofs 

appeared white in the image - impossible for the SLAM to extract features from 

these areas. 

Another pointcloud mentioned in the former chapter was the one recorded during 

the EMAV competition. We first take another look at the given pointcloud in the 

image 6.4 and 6.5. From the first perspective we can clearly see the feature bands 

which helped the SLAM algorithm to extract features on the homogeneous gym 

floor. There are almost no points between these bands and a lot of points were 

triangulated wrong and placed somewhere below the floor (only half of those are 

visible in the image). From the side one can recognize the window ledge on the 

right side. 

Figure 6.4: EMAV competition cloud 

Figure 6.5: EMAV competition cloud from the side 

The resulting MLS map of this recorded flight is displayed in the image 6.6. Once 

again the feature bands can be recognized as regions without red dots. Unfortunately 

there are also two major flaws in the middle and on the right, but considering 

the pointcloud, the result is still quite good. Even the window ledge is visible on 

the right. 

The third and last pointcloud contains a table and a floor. This pointcloud was 

shown in the last chapter in image 5.1 on page 25. The result illustrated in image 6.7


Figure 6.6: Surface map of EMAV competition 

looks fairly good except for a little flaw on the table. The table even overlaps a 

little bit with the floor plane, such that the benefit of a multi level surface map can 

be partially verified. There are also no red markers, meaning that there weren’t 

any gaps larger than the quadrotor (the quadrotor was assumed to be around 3 cell 

sizes wide). 

Figure 6.7: Surface map of table and floor 

6.2 Computational costs 

Besides from the result of the MLS maps another factor is also quite crucial for 

doing online mapping - the computational cost. I did some measurements of the 

computational time with the Birmensdorf pointcloud on a rather old laptop from 

2005. It features an Intel Pentium M 750 1.86 GHz prozessor and 2GB RAM. The 

goal was more to get a feeling, how computational costs relatively change against 

pointcloud size. The absolute result of the measurement is not that significant, 

since laptops are a lot faster nowadays. However one can still get a feeling of what 

seems to be possible with online mapping. 

The results of the test are shown in the diagram 6.8. From 10ms for a 20 point 

pointcloud up to 2 seconds for 400 points, the curve seems to follow an exponential 

curve progression. In order to find out the feasibility of online mapping, we have to 

know the size of the pointclouds published by the SLAM periodically. Lets again 

take the Birmensdorf example. Its pointcloud features about 3100 points. If we 

assume a mapping time of 1 minute for the whole area (which would be quite fast) 

and like to process a pointcloud every second, we would end up with an average 

pointcloud size of about 52 points every second. 52 points consume about 25ms on 

this relatively old laptop, which’s only 2.5% of the computational power. Therefore 

online mapping should be easy feasible, even if we are able to extract more features 

from of the video feed in the future.

35 6.3. Problems 

Figure 6.8: Diagram of computing time depending on pointcloud size 

Computing time against pointcloud size 

Average time in seconds 

2.5 

2 

1.5 

1 

0.5 

0 

0.01 

0.02 

0.03 

0.06 

0.07 

0.11 

0.15 

0.17 

0.21 

0.24 

0.61 

0.43 

20 40 60 80 100 120 140 160 180 200 240 280 320 360 400 

Pointcloud size 

Average time 

1.37 

1.63 

2.06 

6.3 Problems 

There is also some problems I’ve encountered during my thesis. Most of them 

couldn’t be solved due to lack of time. To begin with, I will point out some issues, 

which are visible in the final Birmensdorf MLS map. One big problem concerns the 

sequential map building. In the launch file one has the possibility to set the minimum 

size of published pointclouds. The higher this number, the more keyframes 

have to be published together. Unfortunately the resulting map also changes its 

appearance with such different settings as you can see in image 6.9. Both times all 

the settings were identical except for the minimum size of published pointclouds - 

in the top map this was set to 300 points, in the lower 80 points. Major differences 

are encircled orange. This issue is probably due to the hole plastering algorithm, 

which doesn’t perform the same on map areas as a whole compared to the same 

map area pieced together from two pointclouds. 

Another problem, which also concerns the hole plastering algorithm, is its preference 

of processing direction. In the image 6.10 a gap near the bridge is somehow closed 

wrong for our impression. You can imagine, that if the order of gap search and 

processing direction in the code is swapped - x after y instead of y after x - this gap 

would have been closed differently - more along the bridge. Solving this problem 

is probably quite hard. Maybe it’s possible to try filling gaps in both directions 

and then choose the local result where the algorithm filled a higher gap. The 

consideration here is to prioritize higher situated gaps, because as potential obstacle, 

they are closer to the quadrotor and therefore more important. 

As already showed in the chapter of adaptions and optimizations, the hole plastering 

algorithm only works on surface planes and not on walls. But for walls (surface 

patches with depths) we’re basically addressing the same problem: As soon as 

depths representing a wall are too sparse, the visualization of the MLS framework 

isn’t able to draw polygons, such that even these existing depths won’t be visible 

in the MLS map. If there was a similar algorithm to close gaps in walls, it 

would most certainly improve mapping. I also think there is a lot of room for improvements 

concerning the visualization algorithm of the original MLS framework. 

The correspondence problem for connecting surface patches to polygons is hard but 

probably can be improved further. Especially the generation of wall surfaces is not 

yet satisfying.


Figure 6.9: Differences by just changing the package size of points processed at 

once. Top: 300 points, bottom: 80 points 

Figure 6.10: Differences by just changing the package size of points processed at 

once. Top: 300 points, bottom: 80 points

37 6.4. Conclusion and outlook 

6.4 Conclusion and outlook 

To conclude this thesis one can say that it basically worked to process sparse pointclouds 

from the sFly SLAM into multi level surface maps. We were able to reject 

many of the outliers efficiently and also could countervail the sparseness of the pointclouds 

by using a hole plastering algorithm. Closed gaps larger than the quadrotor 

are additionally marked as regions of uncertainty. Like that we get a feeling of 

the mapped environment and can also keep track of regions, where we have to fly 

over again and get more features. Despite the shown capability of processing MLS 

pointclouds, the framework couldn’t yet be tested with real recorded data featuring 

multiple level surfaces. Hence this is definitely a matter which has to be tested 

next. Furthermore as we saw in the last problem section, the hole plastering algorithm 

isn’t yet perfectly implemented. There are some implications when doing 

sequential online mapping which lead to slightly different MLS maps depending on 

the pointcloud sizes published by the SLAM algorithm. The cost of processing a 

pointcloud increases exponentially with its size, but with the actual rate of newly 

extracted features from the video feed, online mapping is easily feasible. 

As an outlook it can be said that it’s going to be interesting to do further quadrotor 

flight tests in terrain with multiple levels. Close flights to bridges and houses 

could give us more information on how well the MLS framework converts its inputs 

into MLS maps. With the feature of all uncertain regions marked, one could also 

implement an exploration mode for the quadrotor. First there would be a higher 

fly-over and next, the quadrotor could fly closer to the uncertain areas in order to 

get more features and complete the map. Thinking further ahead we could even 

implement adaptive cell sizes. The cell size is a central design criterion for the maps: 

Smaller cell sizes allow more detailed maps. But as a drawback, this also implies 

that less points from the pointcloud will be added per cell and therefore results in 

more empty cells and finally a sparse map. A larger cell size however decreases the 

level of detail while making the map denser. Like that one could get the idea of 

introducing local, adaptive cell sizes, depending on the local amount of points in 

the pointcloud.

Chapter 6. Result discussion 38

Bibliography 

[1] R. Triebel, P. Pfaff, W. Burgard: Multi-Level Surface Maps for Outdoor 

Terrain Mapping and Loop Closing. In Proc. of the IEEE/RSJ Int. Conf. on 

Intelligent Robots and Systems (IROS), 2006. 

[2] R. Zask, M.N. Dailey: Rapid 3D Visualization of Indoor Scenes Using 3D 

Occupancy Grid Isosurfaces, In Proceedings of ECTI-CON, 2009. 

[3] ROS.org community site for hosting ROS-related software. http://www.ros.org. 

39

Multi Level Surface Mapping using Sparse Point Clouds - Student ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?