MACHINE LEARNING TECHNIQUES - LASA
MACHINE LEARNING TECHNIQUES - LASA
MACHINE LEARNING TECHNIQUES - LASA
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
3<br />
4. 4 Regression Techniques ........................................................................................ 64<br />
4.1 Linear Regression ......................................................................................................... 64<br />
4.2 Partial Least Square Methods ...................................................................................... 64<br />
4.3 Probabilistic Regression .............................................................................................. 66<br />
4.4 Gaussian Mixture Regression ..................................................................................... 69<br />
4.4.1 One Gaussian Case ................................................................................................ 70<br />
4.4.2 Multi-Gaussian Case ............................................................................................... 71<br />
5. 5 Kernel Methods ...................................................................................................... 73<br />
5.1 The kernel trick ............................................................................................................. 73<br />
5.2 Which kernel, when? .................................................................................................... 75<br />
5.3 Kernel PCA .................................................................................................................... 76<br />
5.4 Kernel CCA .................................................................................................................... 81<br />
5.5 Kernel ICA ...................................................................................................................... 84<br />
5.6 Kernel K-Means ............................................................................................................. 88<br />
5.7 Support Vector Machines ............................................................................................. 90<br />
5.7.1 Support Vector Machine for Linearly Separable Datasets ....................................... 92<br />
5.7.2 Support Vector Machine for Non-linearly Separable Datasets ................................ 96<br />
5.7.3 Non-Linear Support Vector Machines ...................................................................... 97<br />
5.7.4 n-SVM ...................................................................................................................... 98<br />
5.8 Support Vector Regression ......................................................................................... 99<br />
5.8.1 n-SVR .................................................................................................................... 106<br />
5.9 Gaussian Process Regression .................................................................................. 109<br />
5.9.1 What is a Gaussian Process .................................................................................. 109<br />
5.9.2 Equivalence of Gaussian Process Regression and Gaussian Mixture Regression<br />
113<br />
5.9.3 Curse of dimensionality, choice of hyperparameters ............................................. 115<br />
5.10 Gaussian Process Classification .............................................................................. 116<br />
6. 6 Artificial Neural Networks ................................................................................... 120<br />
6.1 Applications of ANN ................................................................................................... 120<br />
6.2 Biological motivation .................................................................................................. 120<br />
6.2.1 The Brain as an Information Processing System ................................................... 120<br />
6.2.2 Neural Networks in the Brain ................................................................................. 121<br />
6.2.3 Neurons and Synapses ......................................................................................... 122<br />
6.2.4 Synaptic Learning .................................................................................................. 122<br />
6.2.5 Summary ............................................................................................................... 123<br />
6.3 Perceptron ................................................................................................................... 124<br />
6.3.1 Learning rule for the Perceptron ............................................................................ 126<br />
6.3.2 Information Theory and the Neuron ....................................................................... 127<br />
6.4 The Backpropagation Learning Rule ........................................................................ 129<br />
6.4.1 The Adaline ............................................................................................................ 130<br />
6.4.2 The Backpropagation Network .............................................................................. 131<br />
6.4.3 The Backpropagation Algorithm ............................................................................ 132<br />
6.5 Willshaw net ................................................................................................................ 133<br />
6.6 Hebbian Learning ........................................................................................................ 134<br />
© A.G.Billard 2004 – Last Update March 2011