MACHINE LEARNING TECHNIQUES - LASA

70
When applying Bayes’ theorem, we get:
z
1
=
∫
∫
( , ,...,
N )
( , ,..., ) ⋅
z ⋅p z z z ⋅dz
1 1 2 1
p z z z dz
1 2 N 1
(4.20)
In contrast to most regression techniques, GMR also allows to make prediction on multi-
y z , z , z x= z z , then we can
dimensional output in one swipe. So, if we set = { } and { }
compute:
1 2 3
{ ( )} ( 1 2 3 4 5 )
{ }
4 ,..., N
yˆ = E p y| x = E p z , z , z | z , z ,..., z
(4.21)
N
4.4.1 One Gaussian Case
Let us first consider first the case where the joint density p( z) p( y,
x)
= follows a single
Gaussian distribution (and not a mixture). Using the fact that, if the joint distribution of two
variables is Gaussian then each conditional distribution is also Gaussian, see Figure 9-2 for an
illustration, we can compute the conditional density ( | )
where µ Y
and
X
p y x which is given by:
1 −1
( y YX XX x YX YX XX XY)
−
( | ) µ ( ) ( µ ),
( )
p y x = N +∑ ∑ x− ∑ −∑ ∑ ∑ (4.22)
µ are the means of the variable y and x respectively and have thus the
Y X
dimensions of y and x , which we shall call N , N . ∑ XX
variable x has
dimension
dimension ,
N
Y X X Y
N × N N × N respectively.
the covariance matrix of the input
X X
× N and the cross-covariances matrices
YX ,
∑ ∑ have
In predictive regression, we are interested in computing the expectation and uncertainty of the
predictive regressive model at a given query point. When the conditional density is Gaussian, this
*
is easy to obtain and we thus have for a query point x , the estimated output:
−1
{ ( )} µ ( ) ( µ )
* *
yˆ E p y| x
Y YX XX
x
X
= = +∑ ∑ − (4.23)
XY
with associated uncertainty:
−
( p( y x)
) ( ) 1
var | =∑YX −∑YX ∑XX ∑ (4.24)
XY
© A.G.Billard 2004 – Last Update March 2011