MACHINE LEARNING TECHNIQUES - LASA

118
Non-linear case
The non-linear case of classification using Gaussian Process proceeds similarly to Gaussian
Process Regression. Instead of putting a prior on the weight as in the linear case, we put a prior
f x , such that we have
on the latent function ( )
1
p( y =+ 1| x) = (5.94)
f ( x)
1 + e −
This has for effect to ensure that the output is bounded between 0 and +1 and can hence be
interpreted as a probability (as in the linear case), see Figure Figure 5-20
Figure 5-20: Example of an arbitrary prior function f(x) (here composed of the superposition of two
Gaussian). Applyinhg the sigmoid function on f(x) flattens the function, while normalizing between 0 and +1.
One now can build an estimate of the class label
*
y for a query point
*
x by computing the
posterior distribution of the function f ( x ) applied on our query point. If we make this a
distribution that is a function of the training datapoints, we have ( )
posterior distribution we want to compute is given by:
( ) ( )
( ) ( ( ) )
p y * | x * , X, Y sigmoid f x * p f x * | x * , X,
Y df
*
( | *
, , )
p f x x X Y and the
= ∫
(5.95)
The integral on the righthandside compute all values on our prior on f ( )
x . While in GPR there
was an analytical solution, in the classification case, the integral is usually analytically intractable.
To solve this, one must use either an analytic approximations of integrals, or solutions based on
Monte Carlo sampling.
© A.G.Billard 2004 – Last Update March 2011