Deliverable 5.2 - the School of Engineering and Design - Brunel ...

More documents

Recommendations

Info

ICT Project 3D VIVANT– Deliverable 5.2Contract no.:248420Search & Retrieval Mechanisms &Tools3 DATABASE PREPARATIONA fundamental step in the preparation of a multimedia database in order to be searchable is to extractdescriptor vectors that represent each multimedia object. In the case of 3DVIVANT S&R frameworkthis also stands true. This section describes the database structure, the descriptors extraction processand the final indexing structure creation to enable multimedia similarity search. This process is alsoknown as offline process as it is depicted in Figure 5.OfflineContentRepositoryPre-ProcessingLow-level FeatureExtractionDescriptorsBrowsing / Selectingexisting objectFeatureRetrievalFeatureMatchingResultsSubmitting newobjectPre-ProcessingLow-level FeatureExtractionOnlineFigure 5: Dynamic behaviour of the search and retrieval framework.The steps of the offline process are discussed in the following subsections. Section 3.1 refers to thedatabase structure, section 3.2 describes the descriptors extraction process and the user’s options,section 3.3 discusses the codebook generation process and the extracted histograms (bag of words)and finally section 3.5 describes the creation of indexes for fast similarity search inside the database.3.1 DATABASE STRUCTUREIn 3DVIVANT the database is actually a filesystem structure that contains all the necessary data andmetadata. The structure aims to organize the data in a) the raw multimedia files b) the features files c)the bag-of-words files d) and finally the index files. The filesystem structure is presented in Figure 6.In the “data” directory of the filesystem, the raw data are separated based on the modality in order todrive the appropriate descriptor extraction algorithms properly for each modality. Moreover, theproposed structure enables easier reference of the multimedia objects in the retrieval process.The decision to select the S&R database to be a filesystem structure is supported by variousarguments. First, a filesystem database is very easy to create even for a novice user. The possibility4/03/2013 12
ICT Project 3D VIVANT– Deliverable 5.2Contract no.:248420Search & Retrieval Mechanisms &Toolsfor a user to have direct access to the actual content provides many advantages in a process of a videoediting scenario such as the “hyperlinker scenario”. Moreover, the S&R framework aims to be usedalso by novice/home users that need to search inside their multimedia content, that most of the timeisstored inside a folder on their hard disk drive.Figure 6: Database filesystem structure3.2 DESCRIPTORS EXTRACTIONThe selected descriptor extraction algorithms run through the “data” directory in the appropriatemodality (viewpoint images, depth maps, curvature images, 2D snapshots etc.) and extract onefeatures’ file for each multimedia file. For each supported descriptor there is a directory with the same4/03/2013 13
Page 1 and 2: ICT Project 3D VIVANT- Deliverable
Page 11: ICT Project 3D VIVANT- Deliverable

Deliverable 5.2 - the School of Engineering and Design - Brunel ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?