Exploiting Class Hierarchies for Knowledge Transfer ... - IEEE Xplore

RAJAN et al.: EXPLOITING CLASS HIERARCHIES FOR KNOWLEDGE TRANSFER IN HYPERSPECTRAL DATA 3411the node were modeled using mixtures of Gaussians, with thenumber of Gaussians corresponding to the number of classesat that node. The initial parameters of the Gaussians wereestimated using the corresponding class data from Area 1. In theE-step of the algorithm, the Gaussians were used to determinethe posterior probabilities of the Area 2 data. The probabilities,thus estimated, were then used to update the parameters ofthe Gaussians (M-step). EM iterations were performed untilthe average change in the posterior probabilities between twoiterations was smaller than a specified threshold [3]. A newFisher feature extractor was also computed for each EM iteration,which is based on the statistics of the metaclasses at thatiteration. The updated extractor was then used to project thedata into the corresponding Fisher space prior to the estimationof the class-conditional pdfs.Analysis of the results showed that while this approachyielded somewhat higher overall classification accuracies thana direct application of the original classifier, the errors weremostly concentrated in a few classes. A closer inspection revealedthat the spectral signatures of these classes had changedsufficiently for them to be grouped differently in the BHChierarchies, if there had been adequate amounts of labeled datafrom Area 2. This suggested that we should have obtainedmultiple trees from Area 1, such that some of them would bemore suitable for the new area.Thus, our second approach was to introduce randomizationinto the structure of the BHC tree. The design space for theBHC offers many possibilities for randomizing the tree structure.In our earlier work [28], we generated randomized BHCtrees by varying factors such as the percentage of the availabletraining data, the number of features selected at each node,class priors, and by randomly switching the class labels fora small percentage of the labeled data points. In this paper,randomized BHC trees were generated by choosing an internalnode of the tree and randomly interchanging the classes drawnfrom its right and left children. The corresponding featureextractors and classifiers at that node (and its children) werethen updated to reflect the perturbation. Note that in the absenceof any labeled data from Area 2, there is no way to evaluatewhich of the randomly generated BHC trees best suits thespatially/temporally different data. Hence, we can only generatean ensemble of classifiers using the training data, hoping thatthe ensemble contains some classifiers that are better suitedto Area 2.The key to the success of an ensemble of classifiers ischoosing the classifiers that make independent errors. If theclassifiers are not independent, the ensemble might actuallyperform worse than the best member of the ensemble. Hence, anumber of measures of diversity have been proposed to choosea good subset of classifiers [29]. Of the ten diversity measuresstudied, the authors recommend the Q av ,theρ av , and the κmeasures for their easy interpretability. They further promotethe Q-diversity measure because of its relationship with themajority vote of an ensemble and its ease of calculation. Hence,we made use of the Q-diversity measure in our earlier study[28]. However, on experimenting with the κ measure [30], wefound that it yielded a comparable, if not better, performance inthe sense of resulting overall classification accuracy than that ofthe Q measure. Further, unlike the Q-diversity measure, the κmeasure does not require access to any labeled data. Hence, inthis paper, the κ-diversity measure, which indicates the degreeof disagreement between a pair of classifiers, was used to ensurethe diversity of our classifier ensemble.The data from Area 2 were labeled using each tree in theclassifier ensemble, and these labels were then used to obtainthe κ measure between each pair of classifiers. The classificationresults of a smaller set of classifiers with the lowest averagepairwise κ measure (i.e., higher diversity) were then combinedvia a simple majority voting.B. Semisupervised CaseIf small amounts of labeled data are available, knowledgetransfer mechanisms can improve classification accuracies, especiallyif they exploit the added information. In this section,we generalize both knowledge transfer methods in order toleverage the labeled data and determine how much labeled dataare required from the spatially separate area before the advantagesof transferring information from the original solution areno longer realized.The ensemble-based approach was modified in two stages.First, after the set of classifiers was pruned to improve thediversity of the ensemble by using the κ-diversity measure, wefurther pruned the remaining set of classifiers to include onlythose which had yielded higher classification accuracies on thelabeled data. A scheme similar to the online weighted majorityalgorithm [31], which assigns all classifiers a weight, was thenused to weight the different classifiers. Prior to learning, theweights of all the classifiers are equal. As each data sampleis presented to the ensemble, a classifier’s weight is subsequentlyreduced multiplicatively, if that example is misclassified.For each new example, the ensemble then returns the classwith the maximum total weighted vote over all the classifiers.Thus, the algorithm used for computing the class label predictedby the BHC ensemble is as follows.Weighted majority vote for BHC ensemble1) Initialize the weights w 1 ,...,w n of all n BHCs to 1.2) For each labeled data point, let y 1 ,...,y n be the set ofclass labels predicted by the BHCs.3) Output class h i if ∀h j ≠ h i , j =1,...,m, where m isthe number of classesn∑n∑w k ≥w k .k=1;y k ==h i k=1;y k ==h j4) On observing the correct class label, if h i is wrong, thenmultiply the weight of each incorrect BHC by 0.5; else ifh i is correct, do not modify the weights.At the end of this learning, the “winnowing property” ofthe weighted majority scheme assigns lower weights to thoseclassifiers with poorer classification accuracies on the incomingdata. Thus, by reducing the contribution of the inaccurateclassifiers to the final decision, the voting scheme ensures thatthe performance of the ensemble is not much worse than thatof the best individual predictor, regardless of the dependencebetween the members of the ensemble [31]. For the semisupervisedimplementation, the EM-based method was modifiedto perform a constrained EM. Here, the E-step only updatesthe posterior probabilities (memberships) for the unlabeled datawhile fixing the memberships of the labeled instances according