Master Thesis - Department of Computer Science

project only the class means in range space and apply PCA to obtain W Range
opt . Let V
and ¯ V be the range space and null space of Sw respectively and can be expressed as,
V = span{αk | Swαk �= 0, k = 1, ..., r}, (4.6)
¯V = span{αk | Swαk = 0, k = (r + 1), ..., d}. (4.7)
where r (< d) is the rank of Sw, and {α1, ...., αd} is the orthogonal eigenvector set
of Sw. A typical example of null space ( ¯ V ), range space (V) and eigenspectrum
of a scatter matrix is shown in Figure 4.1. Now the matrices Q = [α1, ..., αr] and
V
V
r d
Figure 4.1: A typical eigenvalue spectrum with its division into null space and range
space.
¯Q = [αr+1, ..., αd] correspond to the basis vectors for range space and null space
respectively. As R d = V ⊕ ¯ V , every face x i j ∈ R d has a unique decomposition of the
form,
x i j = yi j + zi j
where (4.8)
y i j = P x i j = QQ T x i j ∈ V, (4.9)
z i j = ¯ P x i j = ¯ Q ¯ Q T x i j ∈ ¯ V . (4.10)
P and ¯ P are the orthogonal projection operators onto V and ¯ V . As the range space
is much smaller than null space we obtain z i j using yi j ,
z i j = xij − yi j = xij − P xij . (4.11)
Now the vectors in Q can be constructed by selecting the first r eigenvectors of Sw
obtained from eigen-analysis. Let x i Null and x i Range represent the vectors obtained from
75