MACHINE LEARNING TECHNIQUES - LASA

114
i=
1
1
( K )
−1 ⎡⎡⎡⎡ 1
% ⎤⎤ 0 ............. 0
⎤⎤⎡⎡y%
⎤⎤
⎢⎢⎢⎢⎣⎣ ⎥⎥⎦⎦ ⎥⎥⎢⎢ ⎥⎥
1 ⎢⎢ .
α ~ ...............................
⎥⎥⎢⎢ ⎥⎥
m ⎢⎢
i
.
K%
⎥⎥⎢⎢ ⎥⎥
−1
∏ ⎢⎢
m
0 .................. ⎡⎡
1
( K%
⎤⎤⎥⎥⎢⎢ ⎥⎥
) m
i=
⎢⎢
⎣⎣
⎢⎢⎣⎣
⎥⎥⎦⎦
⎥⎥⎢⎢ ⎦⎦⎣⎣y
% ⎥⎥⎦⎦
1
−1
l l l
α = ⎡⎡K% ⎤⎤ y , l = 1... m
m ⎣⎣ ⎦⎦
%
i
K%
∏
(5.90)
Where
l
l
y% is composed of the output value associated to the datapoints X .
For each cluster
l
l
C , one can then compute the centroid { l l
µ µ x , µ y }
= of the cluster and a
measure of the dispersion of the datapoints associated to this cluster around the centroid, given
T
{ }
j l l l l
by the covariance matrix
xx E ( X µ x I)( X µ x I)
Σ = − − of the matrix
l
X of datapoints
l
associated to the cluster. Further, for each set of datapoints X , one can use the associated set
of output value
T
( )( ) .
y% l
T
l l l l l
nd compute the crosscovariance matrix Σ = ( %)
−µ y −µ
x
yx
y I X I
If we further assume that each of local kernel Matrix is approximatively a measure of the
−1 T
−1
l
covariance, i.e. ~
l l l l
⎡⎡K % ⎤⎤ ⎡⎡( X −µ xI) ( X −µ
xI)
⎤⎤ and ( ) T
l ( l l
µ ) x
⎣⎣ ⎦⎦ ⎢⎢⎣⎣
⎥⎥⎦⎦
Replacing in Equation (5.88) yields:
k x*, X ~ X I x*
T
− .
m
1
−1
l l l
f* =
( *, )
m ∑ ⎡⎡K% y k x X
i
⎣⎣
⎤⎤ ⎦⎦
%
l=
1
K%
∑
i=
1
(5.91)
Observe that our prediction is now a non-linear combination of m linear projections of the
−1
l l
datapoints through % %. If the number of cluster m is composed of a single datapoint, we
⎡⎡
⎣⎣
K ⎤⎤
⎦⎦
y
obtain a degenerate Gaussian Mixture Model with a unitary covariance matrix for each Gaussian.
Similarly when the clusters are disjoints, the prediction f * can be expressed as the product of
the conditional on each cluster separately and the equivalence with GMM is immediate. In the
more general case, when the clusters contain an arbitrary number of datapoints and are not
disjoints, the full Gaussian Process takes into account influence from all datapoints. In a GMM the
interactions across all datapoints are conveyed in part through the weighting of the effect of each
Gaussian. Note also that the centers of the Gaussians are chosen so as to best balance the
effect of the different datapoints through E-M.
© A.G.Billard 2004 – Last Update March 2011