Deliverable 5.2 - the School of Engineering and Design - Brunel ...

More documents

Recommendations

Info

ICTProjectContract no.:2484203D VIVANT–Deliverable5.2Search& Retrieval Mechanismss &Tools3.3CODEBOOK GENERATION AND BAG-OF-WORDSSince most of the well performing descriptor extraction algorithms extract local features, there aremany feature vectors that correspondtoa single multimedia object. The local features for eachmultimedia object may be hundreds or thousands andthus it is very difficult to use in an efficient wayfor the matchingstep. A common approach to this problem is thebag-of-words (BoW) method. Withthe BoW, a codebook with the most dominant code words (quantized descriptor vectors) is generated.The codebook isgenerated using a k-means algorithmwith a typical k of 5000 to 1000 dimensions andby sampling the local features available in the database.Theneach set of local features is checked against the codebook to generate a histogram ofoccurrences of code words for each features’ file (which correspond to one multimedia object). Thefinalhistogram of the BoWis actually a new descriptor that represents themultimediaobject withonlyone descriptor vector. These final BoW descriptor vectors are stored inthe database under the“words” directories.3.4MANIFOLD LEARNING AND UNIFIED SEARCH SPACEThe next step after the generation of the bag-of-words is the creation of the unified search space as itwas described in details inDeliverable D5.1. Forthis the manifold learning process starts andcombines the different modalities (viewpoints, depth, curvature) to build a unified vector space.Thethatfinal outcome of the process is a single descriptor vector (see Figure 10) per multimedia objectcorrespondss to the unified multimodal search space.Figure 10: the manifold learning process produces a single multimodal descriptor per objectThisfinal singledescriptor vector per multimedia object is thenindexed to prepare the database forsimilarity searchqueries.4/03/201316
ICT Project 3D VIVANT– Deliverable 5.2Contract no.:248420Search & Retrieval Mechanisms &Tools3.5 INDEX CREATIONIt is very common in a search environment to use indexing structures to enable faster similaritysearch. For the 3DVIVANT search and retrieval framework the kd-tree indexing structure (see Figure11) was selected. The kd-tree builds a tree data structure (or a number of trees – kd-tree forest) wherethe most similar descriptor vectors are close together. By using such a structure, the exhaustive linearsearch is avoided and thus the framework answers faster in similarity search queries.List ofSpace partitiona: [v a,1v a,2...v a,n]b: [v b,1v b,2...v b,n]⁞k-D12 34 5 6a b c d e f gFigure 11: kd-tree indexing structureIn the cases that the database is small or there are no external-to-the-database queries the kd-treeindexer may not be used. In that case the engine can pre-compute the search results for each record inthe database and store these results for future queries.4/03/2013 17
Page 1 and 2: ICT Project 3D VIVANT- Deliverable
Page 15: ICT Project 3D VIVANT- Deliverable

Deliverable 5.2 - the School of Engineering and Design - Brunel ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?