MACHINE LEARNING TECHNIQUES - LASA

26
2.2 Canonical Correlation Analysis
Adapted from Hardoon, Szedmak & Shawe-Taylor, Neural Computation, 16:12, p. 2639-2664, 2004
Canonical Correlation Analysis (CCA) finds basis vectors for two sets of variables such that the
correlation between the projections of the variables onto these basis vectors is mutually
maximized. CCA is, thus, a generalized version of PCA for two or more multi-dimensional
datasets. Note that the two multidimensional datasets do not have the same dimensions nor live
in the same basis. The may code for completely different types of information.
Take for example the case when you may want to develop a biometric system that can identify
different people based on audio recordings of the persons’ voices and sets of video recordings of
each person’s face when talking. Audio and video recordings are recorded simultaneously and
hence when taken as a pair they may reveal some more information than taking each of these
individually. CCA will aim at extracting the features in each dataset (audio and visual) that
correlate best for each person individually. In each space (audio and video), CCA would find one
or more eigenvectors that maximize correlation across pair of first, second, third eigenvectors in
each basis. In other words, CCA tries to find a linear combination of audio features and a linear
combination of video features that correlate best. Data once projected in each of these separate
sets of eigenvectors would then be most representative of a person and hence can be used to
best discriminate afterwards (e.g. by using the projected data in a classifier afterwards).
While PCA works with a single dataset and maximizes the variance of the projections of the data
onto a set of eigenvectors forming a basis of this dataset, CCA works with a pair of random
vectors determined in each basis separately and maximizes correlation between sets of
projections. In other words, while PCA leads to an eigenvector problem, CCA leads to a
generalized eigenvector problem.
M
i N
i q
Consider a pair of multivariate datasets X = { x ∈ } , Y =∈{ y ∈ }
i
i
M
i i
measure a sample of M instances ( x , y )
{ }
i=
1
° ° of which we
M
= 1 = 1
. CCA consists in determining a set of projection
vectors w and w for X and Y such that the correlation ρ between the projections
x
y
T
T
X'
= w X and Y' = w Y (the canonical variates) is maximized.
x
y
T
{ }
T
T
wxE XY wy wxCxywy
ρ = max corr ( X',Y' ) = max = max
(2.16)
wx,
wy
w X w Y w C w w C w
w , T , T T
x w
T
y wx wy
x y x xx x y yy y
Where Cxy, Cxx,
Cxy
are respectively the inter-set and within sets covariance matrices.
C is N× q, C is N× N and C q×
N.
xy xx xy
Given that the correlation is not affected by rescaling the norm of the vectors w , w , we can set
that:
w C w = w C w = 1
(2.17)
T
x
T
xx x y yy y
x
y
© A.G.Billard 2004 – Last Update March 2011