3D Building and Road Reconstruction - GeoComputing Laboratory

3D Building and Road 

Reconstruction (I) 

Fuan Tsai 

Center for Space and Remote Sensing Research & 

Department of Civil Engineering 

National Central University 

GeoComputing 

Laboratory 

國立中央大學太空及遙測研究中心

2 

Strategy of Modeling 

 

 

Buildings 

By strategies 

Data-driven v.s. model-driven 

By extracted features 

Points, lines, polygons 

By degree of automation 

Roads 

Manual; semi-automatic; fully-automatic 

strategies 

LOD0 

Data-driven; fully-automatic 

LOD1 and higher 

Data- and model-driven; semi-automatic 

國立中央大學太空及遙測研究中心空間運算實驗室 

National Central University Center for Space and Remote Sensing Research GeoComputing Laboratory

3 

Data-driven Building Reconstruction 

Model 

deduction 

Modeling 

Images Edge detection Line tracing 



4 

Model-driven Building Reconstruction 

Model 

Database 

Image 

Model-Image matching 



5 

Point Features 

(Gruen and Wang, 1998) 



6 

Line Features 

Initial Bld Models & 3D line segments 

Split 

Split 

2D Space 

Merge 

3D Space 

Shaping 

(Rau and Chen, 2002) 



7 

Polygon/Polyhedron Features 

(Lang and Förstner, 1996 ) (Gulch et al, 1998) 

(Tseng and Wang, 2002 ) 



8 

Manual 

Human image 

interpretation 

Degree of Automation 

Computerized processing 

User-friendly GUI 

Interactive processing 

QC & QA 

Semi-automatic 

Semi-automatic image 

interpretation 

Computerized processing 

Fully-automatic 

Computerized image 

interpretation and 

processing 



9 

Data and Techniques 

By data sources 

Topo. maps, images, 

LiDAR 

Data fusion 

Topo. maps 

Buildings 

Roads 

LiDAR 

TLS 

Airborne 

Images 

Close-range 

Airborne 

Satellite 



10 

Block-based models 

Single data source 

Airborne images 

Topo. maps 

Airborne LiDAR 

Data fusion 

Topo. map+images 

Topo map+LiDAR 

LiDAR+images 

Building Modeling 

Unconventional methods 

Single-view reconstru 

ction 

Highly detailed 

models 



11 

Stereo Aerial Photos 

3-D View 

Partially visible 

Fully visible 



Building Modeling using Aerial Photographs 

12 

Input data: 3D structure lines 

SPLIT-MERGE-SHAPE (SMS) 

SPLIT-MERGE 

2D topology reconstruction 

For closed objects 

Buildings, lake, farmland, etc 

SHAPE 

Roof Shaping 

Level-of-Detail Building Modeling 

Polyhedral or Prismatic 

Polyhedral 

Prismatic 



SMS-Building Modeling Procedure 

Initial Bld Models & 3D line segments 

Split 

Split 

2D Space 

Merge 

3D Space 

Shaping

14 

NCU 

Examples 

Images and lines 

Bld. Models 



15 

DBM Examples 



16 

Building Modeling using Aerial Images 

Hsin-Chu Science Park 

By-product of large scale vector maps 

3D structure lines (include ridge lines) 

Real height 

Data Provider:CECI 



17 

Bld. Modeling from Topo. Map 

Building outlines and no. of floors (notations) 

Polygon Tracing 

Export Boundaries 

Height Assignment 

2 

1 

Building Polylines 

Building Polygons 

4 

3 

1 

2 

1 

1 

Point-in-polygon Examination 

1 

Topomaps(CAD) 

4 

3 

1 

Building Polygons with 

Stories 

3D Building Models 

Export Stories 

1 

1 

Stories 



18 

Example (Taipei City) 

LOD0 Bld. Models 

2D Outlines 



Building Modeling Using LIDAR 

1.Elev. Segmention 2.Regularization 

... 

... 

Digital Surface Model 

3.Reconstruction 

1 2 

3

20 

Airborne LiDAR Data 

Point Clouds 



Ground Based LIDAR 

21 

© OPTECH 



LIDAR22 

Fusion of Airborne and 

Ground-based LIDAR 

© Optech 



Building Modeling using LIDAR 

23 

Lidar DSM 

Building Boundaries 

Top-View DBM 

Aerial photo (Only for comparison) 



24 

Examples 

LiDAR Contours 

Bld. Models 



25 

Fusion of Aerial Photo and Topo Map 



26 

Example 

Left Right Left Right 

3D Lines DBM 3D Lines DBM 



27 

Roof Structures 

Structure lines 

Left Right Roof structure 



28 

Building Boundaries 

Building boundaries and Structural lines 



Building Modeling using Aerial images 

and Topographic map 

29 

with structural lines 

Block Model 



30 

Fusion of Topo Map and LIDAR 



31 

Example 

LIDAR and Boundary Lines 

DBM 



32 

Example 



33 

Generated DBM 

Generated DBM and LIDAR points 



34 

Fusion of LIDAR and Aerial Photo 

+ + 



35 

Image NDVI Textures lidar 



36 

Building detection 

Segments Building regions Building regions and 

topographic maps 



37 

Example 

Generated 3D DBM 



38 

Single-View Reconstruction 



39 

Vanishing Point Metrology 

What is a vanishing point 

The point (in a perspective image) at 

which parallel lines receding from an 

observer seem to converge. 



Linear perspective (1-3 vanishing points): 

40 

Curvilinear perspective (4 or 5 vanishing points): 

(Pietro Perugino) 

Jan van Eyck, Arnolfini's Wedding 



41 

(Horry et al., 1997) 



42 

(Leibowitz et al., 1999) 



43 

SVR Procedure 

Raw image 

Line extraction 

Feature points 

selection 

Vanishing points 

calculation 

Feature points 

calculation 

Height equalization 

Generate 

building model 

Vanishing points 

refinement 



44 

Automatic Vanishing Point Detection 

Vx (-1492.4, -429.9) 

Vy (1104.8, -480.0) 

Total – 300 points 

Vx – 16 

Vy – 50 

Vz – 20 

Vz (248.7, 1699.5) 



45 

Automatic Vanishing Point Detection 

K-means Manual LS Proposed 

i j i j i j i j 

VPx -1563.7 -456.9 -1711.7 -504.7 -1478.4 -415.9 -1492.4 -429.9 

VPy 1117.1 -490.5 1078.6 -459.3 1106.83 -478.0 1104.8 -480.0 

VPz 259.4 1723.4 256.1 1663.5 246.7 1675.5 248.7 1699.5 

RMSE 0. 598 % 0.673 % 0.605% 0.458 % 



46 

(X, Y) Coordinates 

Get projected images for two main plans, and use reference 

height to measure the scale. 



47 

Elevation (Z) Coordinate 

Single-View perspective metrology: 

P = L ∞ 

∩Br B 

R =VB∩T r P 

D= 

d V , R ∗d T , B 

d V , T ∗d R , B ∗D r 



48 

Height Equalization 

To minimize the errors in order to obtain more accurate models. 

1. Sort according to height values. 

2. Define the threshold of “same-elevation” 

3. Starting from the smallest value, calculate the difference with next 

value; if the difference is within the threshold, then replace all values 

with their average. 

4. Continue the procedure until the difference is larger then the 

threshold--define it as the next elevation and repeat. 



49 

Vanishing Point Fine Tuning 

Vanishing point fine-tuning based on height 

equalization: 

VPy 

VPx 



50 

Example 

Image size : 647*682 Sobel Canny 



51 

VP y 

(-6511 , -3613) 

VP x 

(16363 , -5175) 

VP z 

(3325 , 19978) 



52 

26.23 

8.01 

2.67 

Blue: cyber city model 

Black: our results 

2.88 

19.53 

2.72 



53 

Example 



54 

Example (2) 

VP z 

(-111 , -5054) 

VP y 

(-526 , 453) 

VP x 

(1370 , 319) 



55 

Blue: field surveyed 

Black: our results 

8.72 

18.25 

15.10 

1.50 

0.65 

5.68 

9.07 



56 

RMSE:1.81% 

Fea. Cal. Ref. Error 

Per. 

a 19.11 19.4 -1.52% 

b 19.11 19.4 -1.52% 

a b 

d 

f 

e 

g 

c 5.34 5.35 -0.13% 

d --- 17.4 --- 

e 17.13 17.4 -1.55% 

c 

f 19.98 20.5 -2.52% 

g 19.98 20.5 -2.52% 




National Central University Center for Space and Remote Sensing Research GeoComputing Laboratory 

57

58 

Example (3) 

(Barns by John Ernsting, 2004) 



59 

Reconstructed Model of Example (3) 



60 

Example (4) 

(The Vicarage at Nuenen by Vincent Van Gogh, 1885) 



61 

特徵點 

計算值 

ab 

c d e 

f g 

h 

a 5.12686 

b 4.87984 

c 7.66114 

d 8.00063 

e --- 

h 

f 5.12689 

g 5.27483 

h 1.77354 

Reference height e is 8m 



62 

Reconstructed Model of Example (4) 



63 

TLS Building Modeling 

Fitting 

Registration 

Texturing 

LOD3 model 



64 

Close-Range Image-Based Reconstruction 



65 

Image-Based Reconstruction 



66 




67 




68 

TLS and Close-Range Images 

點雲套合 



69 

Strategies for 3D Building Modeling 

Item LOD1 LOD2 LOD3 

Scope National-wide Urban Area Landmark buildings 

Compare to 

vector maps 

1/5000 topo. map 1/1000 topo map 1/200~1/500 topo map 

Cost 1. Piggyback on 1/1000 

topo mapping tasks → 2.1% 

cost increase. 

2. Carry out independently 

→ 10% of topo mapping 

cost. 

Strategies 1. Store height/elevation 

data during 1/1000 topo 

mapping. 

2. Using existing 

vector/topo maps. 

Piggyback on 1/1000 

topo mapping tasks → 

40% cost increase. 

Treated as an extension 

of 1/1000 mapping tasks. 

High cost (varies by 

building complexity.) 

Implemented as-needed 

and for landmark 

buildings only. 



70 

Data-Driven Road Reconstruction 

Vector Maps 

Road Networks 

Model 

Reconstruction 

LiDAR 

Ribbon Model 



71 

Automatic Road Modeling 

Ribbon Models 

Road Neworking 

Modeling 



72 

Original LiDAR data 

3D Road Modeling 

CASE II 

978 

139 

113 

-140 

CASE I 

90 

CASE III 

國立中央大學太空及遙測研究中心 

61 

空間運算實驗室 


73 

443 

Extracted road points 

124 

-140 

113 

90 

國立中央大學太空及遙測研究中心 

61 

空間運算實驗室 


74 

Examples 




National Central University Center for Space and Remote Sensing Research GeoComputing Laboratory 

75

76 

Highly Detailed Road Modeling 

(OGC CityGML) 



77 

高精緻度道路模型重建 

Mobile Images Stop-and-Go LiDAR Mobile LiDAR 



78 

Examples 

Surface Models 

Elevated Models 



79 

Highly Detailed Road Modeling 

Item Mobile Imaging Static Mobile TLS Mobile TLS 

Data acquisition 

Continuous snap shots 

(e.g., 2 frames/sec.) 

Static scanning 

(stop-and-go) 

(e.g.:Scan rate: 120Hz; Pulse 

rate: 300kHz) 

Continuous scanning 

(e.g.: Scan rate: 200Hz; Pulse 

rate: 500kHz) 

DAR performance high medium high 

Limitations Day-time only No puddles or splashes 

Cannot operate in nonstoppable 

or heavy traffic 

areas (e.g., express ways or 

highways) 

Data pre-processing 

Computation of mage 

parameters 

(triangulation) 

Data registration and 

transformation to projective 

coordinate systems 

No puddles or splashes 

Data registration and 

transformation to projective 

coordinate systems 

Data volume small medium large 

Accuracy factor Image resolution Point cloud density Point cloud density 

Data processing Conjugate points Point cloud processing Point cloud processing

3D Building and Road Reconstruction - GeoComputing Laboratory

Create successful ePaper yourself

Delete template?

Save as template?