download prezo - Nvidia

GPU Computing Seminar3D and Medical Applications on GPUDr. Kai-Che LiuAssistant Manager3D System Application DivisionElectronics and Opto-electronics Research LaboratoriesIndustrial Technology Research Institute2010/01/22

Content• Introduction- End to End Solution• Applications on GPU Platform- DIBR Rendering- Multi-view Synthesis- Optical Coherence Tomography• Other Researches 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE2

Depth Image Based Rendering and Multi-view Interlacing2DImageDepthImageDisparity Mapping andMulti-view RenderingDis-occlusionFillingMulti-viewInterlacing3DDisplay 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE4

Parallel Design2DImageDepthImageDisparity Mapping andMulti-view RenderingDis-occlusionFillingMulti-viewInterlacing3DDisplay1 2 3 4 5 6 7 8 91-9 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE5

ResultSpec.ResultGPU26x Speed-up554.6 1.8CPU 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE6

3D Display in ITRI• Stereoscopic Display (Glasses Type)- Rendering 2-Views + Interlacing/Page-FlippingMicro-retarderFull/Localized Shutter• Auto-Stereoscopic Display (Without Glasses)- Rendering N-Views + Interlacing3.5 5-view 20 7-view 42 9-view i-Screen 3D 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE7

Multi-view Free Viewpoint SynthesisAObjectsB 1CPUNearbyImagesCE 2 DFNearbycameraCopy data fromGPU --> CPUBuffer GPUViewpoint 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTEA: User interface – OpenGLMove & Rotate & Reset viewpointA1: Initial mesh constructionB: Get nearby Images from Buffer 1CPU -> GPUC: Find nearby cameras for everyvertex NearbyD: Depth reconstruction•Projection•Get patch•SSD measurementViewpoint ImageE: Load next near-by frames andstore in Buffer 2F: Render Left & Right Channels•Render scene - OpenGL•Multi-texture blending•Displaycamerandepthplane8

CPU-GPU FrameworkAB 1NearbyImagesCA: User interface – OpenGLMove & Rotate & Reset viewpointA1: Initial mesh constructionB: Get nearby Images from Buffer 1CPU -> GPUC: Find nearby cameras for everyvertexE 2 DFCPUCopy data fromGPU --> CPUBuffer GPU 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTED: Depth reconstruction•Projection•Get patch•SSD measurementE: Load next near-by frames andstore in Buffer 2F: Render Channel•Render scene - OpenGL•Multi-texture blending•Display9

Data Processing for Free Viewpoint Synthesis•Find nearby camerasfor each vertex:Total threads we set :Width Height* = Vertex_numMesh_size Mesh_size•Depth Reconstruction:Total threads we set :Vertex _ num*num _ Depth _ planesMesh size2510Depth Reconstruction (ms) – GPU171.4217.705.52Total (ms)319.6245.5018.69Fps – GPU3.1222.0053.50Depth Reconstruction (ms) – CPU566.1390.3823.13Total (ms)678.92137.4862.59Fps - CPU1.57.316Depth Reconstruction Speed-up3.3x5.1x4.1xTotal Speed-up2x3x3.3x 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE10

Data Processing for Stereo Viewpoint SynthesisMulti-viewCapturingViewpointSelectionGTX 280LeftEyeRightEyeIBR_LGPUIBR_RReconstructDepth (ms)Render Scene(ms)6.16363.02985.22105.1264Left-eyeChannelRight-eyeChannelCopy Data(ms)3.0885StereoscopicApplicationTotal and Fps28.2496(ms) &35 Fps 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE11

OCT System and Platform‧GeForce GTX280 with 1024MB GDDR3‧ NI 14-Bit 100MS/s Digitizer (PCI-5122)‧ NI 14-Bit 20MHz Arbitrary FunctionGenerator (PCI-5402)‧GPU Processing: 15x Speed-up,2,500,000 points/frame 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE13

Other Researches• Depth Estimation from 2D Video (Video 2Dto 3D Conversion)- Algorithm Design and GPU Speed-up- 3fps to 30fps (10x Speed-up)Application 1:3DTVTransmit2DContentVideo 2D to 3DConversionApplication 2:3D Webcam(MSN, ) 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE15

Other Researches• Stereo to MV Rendering (Stereo Matching)- 3D Movie on Autostereoscopic Display(Without Glasses)- On-going (Currently 10fps on GPU)• 3D Endoscopic Imaging- 3D Reconstruction from Single Lens- On-going (Target: 60fps for HD Input)• Smart OCT System- On-going: Automatic Diagnosis Assisting(Database Registration, Template Matching) 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE16

Kai-Che, LiuTel: 886-3-5917983,Email: lkcjack@itri.org.twITRI EOL 料 流 ITRI CONFIDENTIAL DOCUMENT DO NOT COPY OR DISTRIBUTE17