MACHINE LEARNING TECHNIQUES - LASA

112
T
In this case the covariance of the prior becomes: ( )
is a matrix whose columns
i
x through (5.84).
2
( ε) ( ) σ
yy = cov f X + = K X , X + I , where y
i
y , i=1….M, correspond to the projection of the associated training point
As done previously, we can express the joint distribution of the prior y (now including noise in the estimate of
prior on the training datapoints) and the testing points given through f*:
2
( , ) + σ I ( *, )
( ) ( )
⎡⎡y
⎤⎤ ⎛⎛ ⎡⎡K X X K X X ⎤⎤⎞⎞
⎢⎢ N 0,
f *
⎥⎥ ⎜⎜ ⎢⎢
⎥⎥⎟⎟
⎣⎣ ⎦⎦ ⎜⎜ ⎢⎢K X, X * K X*, X * ⎥⎥⎟⎟
⎝⎝ ⎣⎣
⎦⎦⎠⎠
: (5.85)
Again, one can compute the conditional distribution of f* given the pair of training datapoints X,
the testing datapoints X* and the noisy prior y.
( ( ))
f*| X*, X, y : N f*,cov f *
{ } ( ) ⎡⎡ ( )
2
−1
f* = E f*| X*, X, y = K X*, X K X,
X + σ I⎤⎤
y
2
−1
( f ) = K( X X ) − K( X X) ⎡⎡K( X X) + σ I⎤⎤
K( X X )
cov * *, * *, ⎣⎣ , ⎦⎦ , *
⎣⎣
⎦⎦
(5.86)
We are usually interested in computing solely the response of the model to one query point x *.
In this case, the estimate of the associated output y * is given by the following:
T
{ } ( ) ⎡⎡ ( )
2
−1
y*~ f* = E f*| x*, X, y = k x*, X K X,
X + σ I⎤⎤
y
⎣⎣
⎦⎦
(5.87)
i
( ) ( )
k x*, X is the vector of covariance k x*, x between the query point and the
i
M training data points x , i = 1... M.
Since all the training pairs ( )
i i
x , y , i 1... M
= are given, these can be treated as parameters to
the system and hence the prediction on y * from Equation (5.87) can be expressed as a linear
i
combination of kernel functions k( x*, x ):
M
i
{ } ∑αi
( )
y*~ f* = E f*| x*, X, y = k x*,
x
( )
i=
1
2
−1
K X X σ I⎤⎤
y
with α = ⎡⎡
⎣⎣ , +
⎦⎦
(5.88)
We have M kernel functions for each of the M training points
x i .
© A.G.Billard 2004 – Last Update March 2011