Master Thesis - Department of Computer Science

a lower dimensional space using PCA and then LDA is applied to this PCA subspace.
But the removed subspace may contain some useful discriminative information which
is lost in this process.
Chen et al. [25] suggested that the null space of the within-class scatter Sw
contains most discriminative information, and hence they proposed LDA in null space
of Sw called N-LDA. However, when the number of training samples is large, the null
space becomes small resulting in loss of discriminative information outside this null
space. The performance of the null space depends on the dimension of the null
space. Thus any preprocessing that reduces original sample space should be avoided.
Another drawback of this approach is that it involves solving the eigenvalue problem
for a very high dimensional matrix. Yu et al. [138] proposed an algorithm which
basically uses simultaneous diagonalization method [39]. First, the null space of
between-class scatter Sb is removed, assuming that the null space of Sb contains no
discriminative information, and then the method seeks a projection to minimize Sw
in the transformed subspace of Sb. As the rank of Sb is smaller than that of Sw,
removing the null space of Sb may lose entire null space of Sw. So Sw is likely to be of
full rank after this removal [14, 28, 45]. One more drawback is that the whitening of
Sb is redundant in this method. Another novel method, proposed by Huang et al. [45]
uses PCA+Null space approach. Here the core idea is that null space of Sw is useful for
discrimination unlike that of Sb. Since the null space of total scatter matrix St is the
intersection of the null spaces of Sb and Sw, this method applies PCA to remove the
null space of St and then Null space method is used on reduced subspace of St. Hakan
et al. [20] proposed a novel scheme to solve SSS problem called Discriminant common
vectors (DCV) method. Among two algorithms to extract the discriminant common
vectors for representing each person in the training set of face database, one algorithm
uses within-class scatter matrix of the samples in the training set while the other uses
the subspace methods and the Gramm-Schmidt orthogonalization procedure. These
DCVs are used for classification of new faces.
Most of the null space based methods like DCV [20], PCA+Null space approach
[45] explored null space of Sw while others like PCA+LDA [120] and Fisherface [12]
eliminated a part of null space to obtain discriminative features. None of them ex-
72