Deliverable 5.2 - the School of Engineering and Design - Brunel ...

ICT Project 3D VIVANT– Deliverable 5.2Contract no.:248420Search & Retrieval Mechanisms &Tools1 INTRODUCTION1.1 EXECUTIVE SUMMARYThe past decade has witnessed an exponential growth in digital multimedia production andcommunication. If nowadays a huge number of images and thousands hours of video are created eachday by professionals or home users, the establishment of the holoscopic imaging technology,advanced by 3D VIVANT, will lead literally to an explosion in the digital content production. Withsuch an increasing rate of content production, the development of effective and efficient contentbasedretrieval tools becomes of utmost importance.3D VIVANT aims to provide tools that will allow the retrieval of similar objects from holoscopiccontent databases. Given the inherent problems of current text-based search engines relying onsubjective manual or automatic annotation, 3D VIVANT opted for a search-by-example approach.The search and retrieval framework can be used from both the professionals and home users.However, in the hyperlinker scenario only the professional users will have access to the integral videoediting and thus also to the search and retrieval framework. Due to this fact, the emphasis in theGraphical User Interfaces was given with the professional user in mind. For a home user, thedifference in the GUI design would be to have less configuration options and emphasize the ease ofuse and not focus on the advanced control of the framework.The usage of the search and retrieval framework will enable efficient editing of hyperlinked integralvideo and content reuse in different scenarios. Moreover, the tools and methodologies can be used bythe home / amateur user to search inside large multimedia databases.To bridge the gap between conventional 2D and 3D technology with holoscopic imaging, 3DVIVANT develops content-based retrieval mechanisms that support multimodal queries, i.e. the useris able to pose holoscopic content queries and the framework will answer these queries with similarmultimedia content from various modalities, such as: 2D images, full 3D models or range scans withor without texture information.In order to reduce the integration constraints and enable a modular usage of the software systemsproduced in 3D VIVANT, allow easy adaptation of the framework to the Content Editing Terminaland enable the usage of different PCs in editing and pre-processing of the content, the software wasdeveloped as an autonomous system. In this approach the S&R framework takes the original integralvideo sequence and metadata and extracts an XML file to be used in the hyperlinker framework. Theselected approach enables integral video pre-processing in advance and distribution of thecomputational workload on different PCs. The hyperlinker environment is presented in deliverableD5.4[2], however, a short presentation of the usage of S&R framework with hyperlinker is discussedlater in this document.Finally, it is worth noting that the software is based on open source libraries that can be compiled onvarious operating systems and it can be easily customized.4/03/2013 6