MACHINE LEARNING TECHNIQUES - LASA

71
4.4.2 Multi-Gaussian Case
It is easy to extend the previous result to a mixture of Gaussians, where p( x,
y)
is given by
Equation (4.18). Recall that each joint density pk
( y, x)
, k = 1,.. K can be decomposed
as p ( y, x) = p ( y|
x) ⋅ p ( x)
(Bayes’ Theorem), then the joint density of the complete mixture
k k k
is given by:
K
k = 1
k k k k
( ) ( )
p( y, x) = α
k
⋅pk y| x, µ ( x)
, ∑%
∑ ⋅pk x| µ
X, ∑ XX
−
( x) ( ) ( x ) and %
( )
% (4.25)
1 −1
k k k k k k k k k k
% .
with µ = µ +∑ ∑ −µ
∑ =∑ −∑ ∑ ∑
Y YX XX X YX YX XX XY
The marginal density of x is given by:
k
K
k k k k
( | µ , ∑ ) =
k( | µ , ∑ )
p x p x
∑ (4.26)
Replacing (4.26) in (4.25), we can express the conditional probability of y given x :
Where
w
k
=
K
j=
1
k
j
k
( x| k k
, ∑ )
j( x| j j
, ∑ )
α p µ
∑
α p µ
k=
1
K
∑ k k
(4.27)
k = 1
( | ) = ( ) ⋅ ( | )
p y x w x p y x
.
Equation (4.27) forms the core of the GMR model. One can then compute explicitly the
expectation and variance on the above conditional, similarly to what we did for the probabilistic
regression.
( Y YX XX X )
K
K
−1
k k k k k
{ ( | )} =
k( ) ⋅ µ ( ) =
k( ) ⋅ µ +∑ ( ∑ ) ( −µ
)
∑
E p y x w x x w x x
∑
% (4.28)
k= 1 k=
1
The expectation is thus a non-linear combination of the expectation of each local component. In
effect, the regression signal from GMR is the result of a non-linear weighting of local linear
regressions.
K
{ ( )} k( ) ( )
∑
2 K
k k k
(( µ )
%
( ) ) ∑ k( ) µ ( )
k= 1 k=
1
( )
2 2
var p y|
x = w x ⋅ % x + ∑ − w x ⋅ % x (4.29)
The variance of GMR is no longer a simple function increasing with the amplitude of the input x
(as in probabilistic regression). Rather it is modulated by the variance of each component locally
and hence carries across a notion of local variance.
A schematic of these variables is shown in Figure 4-3.
© A.G.Billard 2004 – Last Update March 2011