MACHINE LEARNING TECHNIQUES - LASA
MACHINE LEARNING TECHNIQUES - LASA
MACHINE LEARNING TECHNIQUES - LASA
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
28<br />
2.3 Independent Component Analysis<br />
Adapted from Independent Component Analysis, A. Hyvarinen, J. Karhunen and E. Oja, Wiley<br />
Inter-Sciences. 2001<br />
Similarly to PCA, ICA is a linear transformation that projects the dataset on a sub-manifold, that<br />
best represent the data at hand. While PCA looks for the principal components, ICA looks for the<br />
directions along which statistical dependence of the data is minimal.<br />
ICA is particularly useful in order to decorrelate and denoise data. ICA is very closely related to<br />
the method called blind source separation (BSS) or blind signal separation. The ``source'' is the<br />
original signal, i.e. the independent components, (e.g. the speaker in a cocktail party). ``Blind''<br />
means that neither the mixing matrix nor the independent components are known to start with.<br />
ICA is one method, perhaps the most widely used, for performing blind source separation. It has<br />
been used successfully in a wide range of signal processing applications, e.g. for de-correlating<br />
multiple sound sources or extracting invariant features in image processing.<br />
2.3.1 Illustration of ICA<br />
,<br />
To illustrate the idea of ICA, consider that you have observed four instances of a two-dimensional<br />
distribution S of independent components s 1 and s 2 and that the four instances form the four<br />
corners of a rectangle, as shown in Figure 2-5:<br />
Figure 2-5; Distribution of two independent components<br />
Note that the range of values for these data was chosen so as to make the mean zero and the<br />
variance equal to one, an important constraint of ICA. Now let us mix these two independent<br />
components, using the mixing matrix<br />
⎛⎛x1 ⎞⎞ ⎛⎛s1<br />
⎞⎞<br />
⎜⎜ ⎟⎟= A⋅⎜⎜ ⎟⎟<br />
x s<br />
⎝⎝ 2⎠⎠ ⎝⎝ 2⎠⎠<br />
⎛⎛1 1⎞⎞<br />
A = ⎜⎜ ⎟⎟<br />
⎝⎝1 1⎠⎠<br />
. This gives us two mixed variables, x 1 and x 2 .<br />
© A.G.Billard 2004 – Last Update March 2011