MACHINE LEARNING TECHNIQUES - LASA

29
Figure 2-6: Mixture of variables
Note that the random variables x 1 and x 2 are not independent any more, see Figure 2-6; an easy
way to see this is to consider, whether it is possible to predict the value of one of them, say x 2 ,
from the value of the other. Clearly it is the case, as they line up along a line of slope 1.
The problem of estimating the data model of ICA is now to estimate the mixing matrix A using
only information contained in the mixtures x 1 and x 2 . In the above example, it is easy to estimate
A by simply solving the inverse for each of the four data points, assuming that we have at least
four points. The problem is less trivial, once we consider a mixture of two arbitrary continuous
distributions.
Let us now assume that s 1 and s 2 were generated by the following uniform distribution:
( )
p s
i
⎧⎧ 1
⎪⎪ if si
≤ 3
= ⎨⎨2 3
(2.20)
⎪⎪⎩⎩ 0 otherwise
Figure 2-7: joint distributions of S1 and S2
© A.G.Billard 2004 – Last Update March 2011