実写画像の合成による運転シミュレーション映像の構築 ... - 池内研究室

(MIRU2005) 2005 7 † †† ††† †††† ††††† ††††† ††††† ††††† †††††† †††† 153–8505 4–6–1 Ee–405†† 153–8505 4–6–1 Ee–405††† 153–8505 4–6–1 Ee–405†††† 338–8570 255††††† 247–8505 345†††††† 114–0013 1–20–4E-mail: †{onoshin,ogawara,kagesawa}@cvl.iis.u-tokyo.ac.jp ITS ITSConstruction of Driving-Simulation View by Synthesizing Real Video ImagesShintaro ONO † , Koichi OGAWARA †† , Masataka KAGESAWA ††† , Hiroshi KAWASAKI †††† ,Masaki ONUKI ††††† , Junichi ABEKI ††††† , Toru YANO ††††† , Masami NERIO ††††† , KenHONDA †††††† , and Katsushi IKEUCHI †††† Graduate Shool of Information Science and Technology, The University of Tokyo Ee–405 Institute of Industrial Science, TheUniversity of Tokyo, 4–6–1 Komaba, Meguro-ku, Tokyo 153–8505 JAPAN†† Institute of Industrial Science, The University of Tokyo††† Graduate School of Interdisciplinary Information Studies, The University of Tokyo†††† Faculty of Engineering, Saitama University 255 Shimo-Okubo, Sakura-ku, Saitama-shi 338–8570 Japan††††† Mitsubishi Precision Co., Ltd. 345 Kamimachiya, Kamakura-shi, Kanagawa 247–8505 JAPAN†††††† Chodai Co., Ltd. 1–20–4 Nihombashi-Kakigaracho, Chuo-ku, Tokyo 114–0013 JAPANE-mail: †{onoshin,ogawara,kagesawa}@cvl.iis.u-tokyo.ac.jpAbstract We propose an efficient and effective view generation system for more photo-realistic driving simulationby using image-based rendering for landscape scene, in contrast to conventional product represented by full geometry-basedrendering with poor reality and huge cost for model construction. As for the image dataset, we captureomni-directional video image through the model course, a public road, with multiple video cameras on our dataacquisition vehicle. The capturing run is carried out only once. We assume two flat walls along both side of theroad, and the the view for the landscape scene is realized as dynamic textures on the wall by synthesizing capturedimages. On-street scene is still renderd by geometry base for versatility such as overlaying other vehicles. Thesystem works as a module of “Mixed-Reality Traffic Experiment Space”, an enhanced driving and traffic simulationsystem developed in the collaborative research project “Sustainable ITS”.Key words ITS, Image-based rendering, Real-world image, Arbitrary viewpoint image, Driving simulation

1. (Image-Based Rendering; IBR) (Geometry/Polygon-BasedRendering; GBR) 1 1 ITS [1] (Mixed Reality Traffic ExperimentSpace)2. 3. 4. 5. 6. 7. 2. 2. 1 • Traffic SimulatorTS; [2]• KAKUMO [2, 3]• Driving SimulatorDS; [3]• Image GeneratorIMG; TS DS KAKUMO TS IMG DS

TS 2 2. 2 IMG: Image Generator IMG IMG 1. • • DS 60Hz DS 3 DS DS • • KAKUMO • • DS • 5. 4. 5.3. 3 CV, CG VR ITSFrue [7] Zhao [8]Teller [10] Kotake [9]

[19, 21, 22] [20] [13–16]Katakura, Ohta [11, 12] 4. 4 9 EPI 1 6 1 180ac180a b cOmni-Directional Imageb 5Distortedt1 t2 t3180a1b1c1b2a1 b2 c3Omni-Directional Imagec3180(a) 4 [4, 5] 5 6(b) 2

5. 5. 1 7 [6] 7 t 1 ,t 2 ,t 3 ? 8[6] 8 ΔX D ΔU = h0ΔX (1)Dh 0 9BillboardingUpdateTexturest3t2t3t2180CapturingVehicle9 Cameras(Omni-Directional)180 10Time: t Time: t +1t1CapturingNewViewpoint 7Renderingt1t3t2t1Omni-Directional Image 9 10 DS 60Hz KAKUMO 20Hz 11 11

2 5. 2 DS DS 12 3 • CPU: Intel Pentium4 2.80C GHz• : QuadroFX3000G• : 2GB• OS: Windows XP6. 14 1 15 60Hz 16 12 13 IMG (VisCluster) 14Simulator HostPosition / Headingof the DriverPC1PC2PC3100Base-Tx 13Far View 1(Image)GENLOCKFar View 2(Image)GENLOCKNear View(Geometry)settingRGBZRGBZRGBZCompositorIMG Projector (XGA)(a) 45 (b) 45 15

7. ITS IC IC 17 16 17 IC IC

[1] K. Ikeuchi et.al, “Mixed Reality Traffic Experiment Spaceunder Interactive Traffic Environment for ITS Research”,Proc. 11th World Congress on Intelligent Transport Systemsand Services (ITSWC 2004), Nagoya, Japan, Oct. 2004[2] T. Shiraishi, H. Hanabusa, M. Kuwahara, E. Chung, S.Tanaka, H. Ueno, Y. Ohba, M. Furukawa, K. Honda, K.Maruoka, T. Yamamoto, “Development of a MicroscopicTraffic Simulation Model for Interactive Traffic Environment”,Proc. 11th World Congress on Intelligent TransportSystems and Services (ITSWC 2004), Nagoya, Japan, Oct.2004[3] Y. Suda, M. Onuki, T. Hirasawa, H. Ishikawa, M. Kano, Y.Mashiyama, T. Oda, A. Tagaya, T. Taguchi, Y. Kanki, “Developnemtof Driver Model using Driving Simulator with InteractiveTraffic Environment”, Proc. 11th World Congresson Intelligent Transport Systems and Services (ITSWC2004), Nagoya, Japan, Oct. 2004[4] H. Kawasaki, A. Miyamoto, Y. Ohsawa, S. Ono, K. Ikeuchi,“Multiple Video Camera Calibration using EPI for CityModeling”, Proc. Asian Conference on Computer Vision(ACCV 2004), Jeju Island, Korea, Jan. 2004[5] , , , , EPI 2 ITS , Dec. 2003[6] T. Takahashi, H. Kawasaki, K. Ikeuchi, M. Sakauchi, “Arbitraryview position and direction rendering for large-scalescenes”, Proc. Computer Vision and Pattern Recognition(CVPR 2000), Hilton Head Island, US, Jun. 2000[7] Christian Frueh, Avideh Zakhor “Constructing 3D CityModels by Merging Ground-Based and Airborne Views”,Computer Vision and Pattern Recognition (CVPR), MadisonUS, 2003[8] Huijing Zhao, Ryosuke Shibasaki “Reconstruction of TexturedUrban 3D Model by Fusing Ground-Based LaserRange and CCD Images”, IEICE Transactions on Informationand Systems, Vol.E83–D, No. 7, pp.1429–1440, Jul.2000[9] D. Kotake, T. Endo, F. Pighin, A. Katayama, H. Tamuraand M. Hirose, “Cybercity Walker 2001: Walking Throughand Looking Around a Realistic Cyberspace Reconstructedfrom the Physical World”, 2nd International Symposium onMixed Reality (ISMR2001), pp.205–206, Mar. 2001[10] Seth Teller, Matthew Antone, Zachary Bodnar, MichaelBosse, Satyan Coorg, Manish Jethwa, Neel Master: “Calibrated,Registered Images of an Extended Urban Area”,Proc. IEEE Computer Vision and Pattern Recognition(CVPR), 2001[11] M. Katakura, T. Oguchi, S. Shikata, “Development of MovingScenery Image Experiment System”(in Japanese), AnnualConference of Japan Society of Civil Engineers, pp.57–4, Sep. 2002[12] H. Ohta, M. Katakura, T. Oguchi, S. Shikata, “Effects ofTraffic Safety Devices on Driver’s Behavior at a Curve Section”(inJapanese), Infrastructure Planning (Japan Societyof Civil Engineers) , Nov. 2002[13] Panat Leelavansuk, Hidehisa Yoshida, Masao Nagai, “CooperativeSteering Characteristics of Driver and Lane-KeepingAssistance System” International Journal of ITS Research,Vol. 1, No. 1, pp.17–24, Dec. 2003[14] Hidehisa YOSHIDA, Masashi IGARASHI, Masao NAGAI“A New Motion Cue Algorithm for Improving LongitudinalSensation in Driving Simulator” 6th InternationalConference on Motion and Vibration Control, pp.587–592,Saitama, Japan, Aug. 2002[15] Taichi Shiiba, Yoshihiro Suda, Yusuke Tanabe, MasaakiOnuki “Virtual Running Tests of Automobile with DrivingSimulator” International Journal of ITS Research, Vol.1, No. 1, pp.33–39, Dec. 2003[16] J.H. Kim, Y. Matsui, S. Hayakawwa, T. Suzuki, S. Okuma,N. Tsuchida, “Acquisition and Modeling of Driving Skills byUsing Three Dimensional Driving Simulator”, IEICE Trans.Fundamentals Vol. E88–A, No. 3, pp.770–778, Mar. 2005[17] Sustainable ITS Project, Center for Collaborative Research,The University of Tokyo, http://www.its.ccr.u-tokyo.ac.jp[18] Metropolitan Expressway Public Corporation, TokyoMetropolitan Expressway, http://www.mex.go.jp/english[19] Mitsubishi Precision Co., Ltd., http://www.mpcnet.co.jp/e[20] SEGA DRIVING SIMULATOR, http://sega.jp/simulator[21] Tasknet Inc., ACCESS MASTER, http://tasknet.co.jp/www/simulator/driving-simulator.html[22] Doron Precision Systems, Inc., http://www.doronprecision.com/driving.html