Master Thesis - Department of Computer Science

PCA Projection
Class 1
Class 2
LDA Projection
Figure 2.3: An example of PCA and LDA projection for a two class problem.
to a lower dimension space using PCA and then LDA is applied to this PCA
subspace.
• DCV (Discriminative Common Vectors Approach): DCV [20] solves
“small sample size problem” of LDA by optimizing a variant of Fisher’s crite-
rion. It searches for the optimal projection vectors in the null space of within-
class scatter Sw (see equation 2.4), satisfying the criterion,
J(Wopt) = arg max
|W T |W
SwW |=0
T SbW | = arg max
|W T |W
SwW |=0
T StW |. (2.7)
So, to find the optimal projection vectors in the null space of Sw, it projects
the face samples onto the null space of Sw to generate common vectors for each
class and then obtain the projection vectors by performing PCA on common
vectors. A new set of vectors, called as discriminative common vectors, are ob-
tained by projecting face samples on the projection vectors. Thus each class is
represented by a single discriminative common vector. Among two algorithms
to extract the discriminant common vectors for representing each person in the
training set of face database, one algorithm uses within-class scatter matrix of
the samples in the training set while the other uses the subspace methods and
the Gram-Schmidt orthogonalization procedures to obtain the discriminative
15