Master Thesis - Department of Computer Science

in our case. On the other hand, a between-class scatter matrix is the scatter of the
expected vectors around the mixture mean as:
C�
Sb = Pi(Mi − M0)(Mi − M0)
i=1
T . (4.43)
where M0 represents the expected vector of the mixture distribution and is given by:
C�
M0 = E{x} = PiMi. (4.44)
i=1
A linear transformation from an n-dimensional x to an m-dimensional y (m < n) is
expressed by:
y = A T x. (4.45)
where A is an n × m rectangular matrix and the column vectors are linearly inde-
pendent. The problem of feature extraction for classification is to find the A which
optimizes J. It can be easily proved that J is optimized if A constitutes first C-1
eigenvectors of S −1
w Sb [39].
4.4.2.2 Nonparametric Linear Discriminant Analysis
The number of features, C-1, selected in LDA [39] is suboptimal in Bayes sense.
Therefore, if the estimate of Bayes error in feature space (y space) is much larger
than the one in the original space (x space), the feature extraction process should be
augmented. If tr(S −1
w Sb) is used as a criterion, LDA selects the first (C-1)-dimensional
subspace containing classification information using the scatter of mean vectors, while
the second (n-C+1)-dimensional space containing information due to the covariance-
differences is neglected, where n is the dimensionality of x space. Fig. 4.5 shows
some of the cases where traditional LDA does not work. Therefore, we need to select
additional features from the (n-C+1)-dimensional subspace. The basis of nonpara-
metric LDA [39] is the nonparametric formulation of scatter matrices, using k-nearest
neighbor (kNN) techniques, which measure between-class and within-class scatter on
a local basis, both being generally of full rank. In addition, the nonparametric na-
ture of the scatter matrix inherently leads to extracted features that preserve class
structure important for accurate classification.
88