Deliverable 5.2 - the School of Engineering and Design - Brunel ...

ICT Project 3D VIVANT– Deliverable 5.2Contract no.:248420Search & Retrieval Mechanisms &Tools3 DATABASE PREPARATIONA fundamental step in the preparation of a multimedia database in order to be searchable is to extractdescriptor vectors that represent each multimedia object. In the case of 3DVIVANT S&R frameworkthis also stands true. This section describes the database structure, the descriptors extraction processand the final indexing structure creation to enable multimedia similarity search. This process is alsoknown as offline process as it is depicted in Figure 5.OfflineContentRepositoryPre-ProcessingLow-level FeatureExtractionDescriptorsBrowsing / Selectingexisting objectFeatureRetrievalFeatureMatchingResultsSubmitting newobjectPre-ProcessingLow-level FeatureExtractionOnlineFigure 5: Dynamic behaviour of the search and retrieval framework.The steps of the offline process are discussed in the following subsections. Section 3.1 refers to thedatabase structure, section 3.2 describes the descriptors extraction process and the user’s options,section 3.3 discusses the codebook generation process and the extracted histograms (bag of words)and finally section 3.5 describes the creation of indexes for fast similarity search inside the database.3.1 DATABASE STRUCTUREIn 3DVIVANT the database is actually a filesystem structure that contains all the necessary data andmetadata. The structure aims to organize the data in a) the raw multimedia files b) the features files c)the bag-of-words files d) and finally the index files. The filesystem structure is presented in Figure 6.In the “data” directory of the filesystem, the raw data are separated based on the modality in order todrive the appropriate descriptor extraction algorithms properly for each modality. Moreover, theproposed structure enables easier reference of the multimedia objects in the retrieval process.The decision to select the S&R database to be a filesystem structure is supported by variousarguments. First, a filesystem database is very easy to create even for a novice user. The possibility4/03/2013 12