Master Thesis - Department of Computer Science

two vectors can be expressed in the following way:
d(yi, yj) = �yi − yj� 2 . (2.2)
The desired class label for the probe image can be obtained by minimum mem-
bership rule.
L(xt) = arg min
c rc. (2.3)
• LDA (Linear Discriminant Analysis): The objective of LDA is to find the
subspace that best discriminates different face classes by maximizing between-
class scatter, while minimizing the within-class scatter. The eigenvectors cho-
sen by LDA provide the best separation among the class distributions, while
PCA selects eigenvectors which provide best representation of the overall sam-
ple distribution. To illustrate the difference, Fig. 2.3 shows the first projection
vector chosen by PCA and LDA for a two class problem. The eigenvectors for
LDA can be obtained by computing the eigenvectors of S −1
w Sb. Here, Sb and
Sw are the between-class and within-class scatter matrices of training samples
and are defined as:
Sw =
C�
�
i=1 xk∈Ci
(xk − mi)(xk − mi) T , (2.4)
C�
Sb = ni(mi − m)(mi − m)
i=1
T . (2.5)
where mi is the mean face for i th class and ni is the number of training samples
in i th class. LDA subspace is spanned by a set of vectors W, which maximizes
the criterion, J, defined as:
J = tr(Sb)
. (2.6)
tr(Sw)
W can be constructed by the eigenvectors of S −1
w Sb. In most of the image pro-
cessing applications, the number of training samples is usually less than the
dimension of the sample space. This leads to the so-called small-sample-size
(SSS) problem due to the singularity of the within-class scatter matrix. To
overcome SSS problem, the following approaches are attempted: a two stage
PCA+LDA approach [120], Fisherface method [12] and discriminant compo-
nent analysis [143]. In all cases the higher dimension face data is projected
14