maps.live.com ? maps.google.com - Visionday

Consumer Electronics Show, January 2008THE FIRST DIGITAL DECENNIUM1998-2008THE INTERNET NOW ISLOCATION AWAREmaps.live.com ?Every Place, Anywhere, Anytimemaps.google.comMicrosoftMapping LegacyLaunchedMapPoint WebServiceAdded businessmapping & mobileproductsMapPoint BusinessUnit establishedLaunched firstconsumerproducts199519992004Maps & Directions20012007Live Mapsfor Mobile2006 Vexcelacquisition2005 Virtual EarthLaunch2008MapPoint Web Service APIsStreets & Trips 2007 w/GPS deviceMapPoint 2006 w/GPS deviceInnovative Imagery stylesVirtual Earth Mobile1

Industrial Vision Day 2009PHOTOGRAMMETRYTHE HUMAN HABITATTHE INTERNETFranz LeberlGraz University of TechnologyInstitute of Computer Graphics and VisionIntervisibilityTelekom ApplikationTheTelecom UsederGebaude-DatenofBuilding DataAUTOMATICALLY DERIVED 3D MODEL OF GRAZ, AUSTRIA, 20072

Yellow Pages &Local AdvertisementUS$ 60 Billion/YearValidation by Enterprise CustomersExtensive Government Agency Customer BaseValidation by Enterprise CustomersExtensive Corporate Customer BaseLOCATION AWARENESSSATELLITESGEODATA SOURCES?3

KievNew Zealand Aerial MappingRockwell 690 Turbo CommanderSATELLITES18 mGolden GateNew Zealand Aerial MappingRockwell 690 Turbo CommanderGeoEye-1 Launch50 cm PixelYES4

THE EARTH’S LAND MASSTHE EARTH’S LAND MASSThe World in 15 cm per PixelSq km 148 millionTHE EARTH’S LAND MASSThe World in 15 cm per Pixel~ 20 Peta Bytes, 2-DTHE EARTH’S LAND MASSThe World in 15 cm per Pixel~ 20 Peta Bytes, 2-DKiloMegaGigaTeraPetaExaZettaYottaKiloMegaGigaTeraPetaExaZettaYottaDenverWE WANT 3D5

PERSPECTIVESSearchNavigationGamese-CommerceSmart PhonesMixed RealityInternet of ThingsChips on Body………3D MODELS3D-VISIONDIGITAL 3D-V?EXCELLENT AERIAL IMAGERYSMART IMAGE COLLECTIONAUTOMATED TRIANGULATIONDENSE MATCHINGSUPERIOR GEOMETRYLIMITLESS DETAILDIGITAL 3D-V?EXCELLENT AERIAL IMAGERYSMART IMAGE COLLECTIONAUTOMATED TRIANGULATIONDENSE MATCHINGSUPERIOR GEOMETRYLIMITLESS DETAILUltraCAM x6

3 Gigabits per Second565 MB per TriggerOne Image per ~ 1 Second± 2 µm Geometric Accuracy7,000 Gray ValuesMICROSOFT DUE DILIGENCE“What is so hard aboutlarge format digital aerial cameras?”Bill Gates, Jan 2006~ 4 Tera-Bytes per Data UnitUnlimited # of Images in One Aerial SortieExtremeEnvironmentalChallengeArea (Pan):Min 60Max 79207860 Grey Levels12.94 bitMicrosoft/Vexcel Aerial Camera EvolutionUltraCamD‐03 UltraCamX‐06 UltraCamXp‐0811,310Pxl.414217,310 Pxl.7

43444546478

GERMANY3 cmWeser LuftbildCHINAAUSTRIADIAMOND AIRMaiden Flight Fall 2006DIGITAL 3D-V?EXCELLENT AERIAL IMAGERYSMART IMAGE COLLECTIONAUTOMATED TRIANGULATIONDENSE MATCHINGSUPERIOR GEOMETRYLIMITLESS DETAIL9

InputMultiple “Raw” Aerial PhotosFrame 175 Frame 176 Frame 177 Frame 178 Frame 179OutputSingle Ortho PhotoFrame 218 Frame 219 Frame 220 Frame 221 Frame 222Stereo GeometryMulti-Image GeometryTwo Ray IntersectionNo redundancyMulti-IntersectionsImprove StatisticalOptimum10 cm Pixel10 cm Pixel2 Links σ z ~ ±16.0 to ±33.0 [cm]8 Links σ z ~ ± 4.3 to ± 4.6 [cm]Redundancy accuracyRedundancy automation10

Redundancy occlusion avoidanceRedundancy outlier detectionTHE EARTH’S LAND MASSThe World in 15 cm per Pixel~ 20 Peta Bytes, 2-Dtowards 200KiloMegaGigaTeraPetaExaZettaYottaDIGITAL 3D-V?EXCELLENT AERIAL IMAGERYSMART IMAGE COLLECTIONAUTOMATED TRIANGULATIONDENSE MATCHINGSUPERIOR GEOMETRYLIMITLESS DETAILGRAZ 3000 PHOTOS 1.5 TBYTES• Coarse Match• Bundle adjustment- Several thousand images-Few GCPs11

Dragonfly(Child of Seadragon)DIGITAL 3D-V?EXCELLENT AERIAL IMAGERYSMART IMAGE COLLECTIONAUTOMATED TRIANGULATIONDENSE MATCHINGSUPERIOR GEOMETRYLIMITLESS DETAILWinston-Salem DSM to DEMClassificationAUTOMATED BUILDING EXTRACTIONImage classification in combination with densesurface model supports buildings to be extracted12

AUTOMATED BUILDING EXTRACTIONRecent ResultsExtrude / intersect roofpolygons to each otherExtrude completed roofgeometry to theDTM (ie 2.5D)Apartment complexin AtlantaPhototexture from UltraCam imagesAUTOMATED BUILDING EXTRACTIONCombined with DEM and Ortho in Virtual Earth2.5 Dimensional Building, Denver2.5D has its limitations!Multi-ImageTrees: Multi- vs. Two-Image Stereo,2-Image StereoMulti-ImageTrees: Multi- vs. Two-Image Stereo2-Image StereoIndividual TreesNo Trees, just ForestIndividual TreesNo Trees, just Forest13

UltraCam Vegetation Modeling, M. SchardtDIGITAL 3D-V?EXCELLENT AERIAL IMAGERYSMART IMAGE COLLECTIONAUTOMATED TRIANGULATIONDENSE MATCHINGSUPERIOR GEOMETRYLIMITLESS DETAILTwo FlightsNorth/SouthEast/WestDenseSurfaceModel@ 10 cm PixelDigitalSurfaceModelFromNorth/SouthImagesDigityalSurfaceModelFromEast/WestImagesDifferenceDSMNorth/SouthminusEast/WestImagesR.M.S.Differences± 7 cm14

MEDIOCREMATCHERSUPERIORMATCHERDIGITAL 3D-V?EXCELLENT AERIAL IMAGERYSMART IMAGE COLLECTIONAUTOMATED TRIANGULATIONDENSE MATCHINGSUPERIOR GEOMETRYLIMITLESS DETAILNEW IDEASTO SOLVE NEW PROBLEMSWhere are the cars ?15

90% Correct……Substantialredundancy-inspired progressyet to be expected16

THE HUMAN SCALETheVirtual EarthVision"You'll be walking around indowntown London and be able tosee the shops, the stores, see whattheWalktraffic is like.inWalka shopin a ?andnavigate the merchandise. Not inthe flat, 2D interface that we have onthe web today, but in a virtual realitywalkthrough.”Sq km 148 Bill Gates million50th Birthday, London, March 2005StreetmapperLaser Scanning in GrazSTREET SIDE17

Still ImagesMappingObjects18

THE THE EARTH’S HUMAN LAND HABITAT MASSESP: Dynamic Seasonal EnvironmentsThe World in 15 cm per Pixeltowards ~ 200 Street Peta Side Bytes, in 2 cm 2-DInterior in 0.5 cm…and with 10 x redundancyTowards 1 ExabyteDATA EXCESS?USERINVOLVEMENTLASER SCANNINGVS3D VISION19

Mid-1980’s-- Airborne Laser Scanning of DenverMid-2000’sGrazFOCUSAERIAL MAPPINGAERIAL LIDAR AERIAL PHOTOGRAMMETRYLidar„Direct“ 3D measurement3D point cloudMultiple echoesDTM by last echo and filteringMulti-Ray PhotogrammetryOverlapping imagesDSM automatically by aerotriangulation &matchingMulti-Rays usedDTM by ground pixel and filteringAERIAL LIDAR AERIAL PHOTOGRAMMETRY20

AERIAL LIDAR AERIAL PHOTOGRAMMETRYAERIAL LIDAR AERIAL PHOTOGRAMMETRYLidar Point Density1 / 5 / 100 p/sqm5 cm GSD -> 400 p/sqmAERIAL LIDAR AERIAL PHOTOGRAMMETRYAERIAL LIDAR AERIAL PHOTOGRAMMETRYImage10cm GSDPoint density: > 50 points/sqmLidar6 points/sqmLidar:8 points/sqm30 deg. FOVHeight: 750 mSpeed: 60 m/sEff. pulse rate: 193 kHzScan rate: 170 scans/sStrip width: 403m20% overlapEffective strip width 322 mUCXp Photogrammetry25 cm GSDHeight: 4,188 mSpeed: < 141 m/sStrip width: 4,328 meter60% overlapEffective strip width 1,731 mAERIAL LIDAR AERIAL PHOTOGRAMMETRYWHY CHOOSE PHOTOGRAMMETRY?Images required in any eventANDMulti-Rays make fully automated DSM generation possiblePhotogrammetric point density is significantly higherCameras have superior collection efficiencyPhotogrammetry also applicable to street-side images Consistent , automated workflow for aerial, street, insideALS vs 3DVDGPF 1) TESTAREA VAIHINGEN/STUTTGART1)German Society for Photogrammetry and Remote Sensing21

TWO SUBBLOCKSRosswag and Vaihingen-NordTWO SUBBLOCKSRosswag and Vaihingen-NordALS1506 x 1100 POSTINGS3D-VALS3D-V22

~ 40 cmSamplingIntervalALS~ 40 cmSamplingIntervalDirectionOf FlightALSBuilding25 cmPostingsfrom40 cmSamplingInterval25 cm Postings23

3D-VBuilding8 cm GSDALSSAWTOOTHBUILDING3D-VSAWTOOTHBUILDING24

3D-VSurface Noise~ ± 6.5 cmALSRecall:GSD 8 cmALSAccuracyfrom GPS/IMU3D-VFOCUSSTREET SIDECar-Based LIDARvsStreet-Side 3D Vision~ 60 Laser Points/m 2~ 7000 Pixel/m 2New Laser Scanning500 pts per m 225

18 ImagesWidth x Height = 1024 x 1360 Pixel, f = 744 Pixel2870 Triangulated Points (SIFT-Features)26

Point CloudTriplet 1Point CloudTriplet 2Point CloudTriplet 327

Point CloudTriplet 4Point CloudTriplet 9Point CloudTriplet 10Point CloudTriplet 15Point CloudTriplet 1628

GRAPHICS PROCESSING UNIT GPUImage Triplet Match ~ 1 to 3 SecondsGPU-acceleration factor 30 to 1004 GPU-PC Acceleration by up to 400Plane Fitting RMSE = ± 0.9 cm3 COMPLAINTS ABOUT 3D VISION“Redundancy” data overloadCosts of a Terabyte DiskthDeptGPS/IMU insufficient for dense matching Aerialtriangulation (AT) needed Automated workflownon-existentUltraMap SoftwarePixel @ 1.3 cmDense matching CPU-consuming, slow.Perhaps 1 hour/ aerial photo?GPU-Aided Computing29

SIXTEEN 3D VISION ADVANTAGES VS LASER1. 2D-image versus single 3D data point2. Multispectral Image classification3. No occlusions via no-cost high image overlaps4. Error checking via redundant observations5. Large area rigid geometry via AT6. Urban facades available at no cost from the air7. Geometric accuracy by AT vs. reliance on GPS/IMU8. Single workflow, all image-basedSIXTEEN 3D VISION ADVANTAGES VS LASERCont’d9. Real time 3DVision via supercomputer in match box10. Point density pixel size and redundancy11. Internal accuracy measures from redundancy12. Superior data collection efficiency13. Single workflow for aerial, street-side & indoor14. Images document details example street signs15. Scene interpretation needs imagery.16. Automation needs redundancy laser adds littleTHEFUTURE? 1977Karl-Franzens Universität Graz1977 JOINTAVAILABLEDISK SPACE2007Technische Universität Graz80 MEGABYTES€ 100,000OUR (THEN) UNIVERSITYVICE-PRESIDENT THEURL30

2007VICE-PRESIDENT THEURLRetired with 4 GB of dataBUYS DISK SPACE500 GIGABYTE€ 1001977-2007Improvement12,500,000DRIVERS FORINNOVATIONS?Price-Performance Performance of ComputingMoore’s Rule: Doubling each 1.5 Years1 €20092039Price-Performance Performance of ComputingMoore’s Rule: Doubling each 1.5 Years1 Million €2009-203920091 €20391 MILLION?31

“HONEY,PICK UP A 6-PACK OFCOMPUTERS ON THEWAY HOME”Mark Weiser, 199632