advanced-algorithmic-trading

Chapter 16
Supervised Learning
Supervised Learning is the most widely utilised form of machine learning, since it performs two
very general learning tasks across a multitude of domains. These tasks are classification and
regression.
Classification problems involve estimating membership of a set of features into a particular
categorical group. One example might be determining whether a patient is likely to possess
a disease or not (binary class membership) from MRI scan data. A commonly-cited financial
example is predicting whether an asset will rise or fall in price the following day, based on N
days of asset price history.
Regression problems involve estimating a real-valued response from a set of features. One
example would be estimating the sales volume of a product based on separate budget allocations
to different forms of advertising, such as TV, radio or internet ads. A financial example would
be to predict the actual value of an asset in the following day from its past price history, not just
whether it has risen or fallen.
In this book techniques from both aspects of supervised learning will be considered.
16.1 Mathematical Framework
There are two approaches used to provide a mathematical formalism to machine learning.
The first approach formulates the problem in terms of function estimation. The "true"
response variable y (categorical or real-valued) is modelled as a function f = f(x). The predictors
(or features) x ∈ R p is a p-dimensional vector containing feature measurements. In addition to
f there is additional noise ɛ containing information not available within x. ɛ is often assumed to
be normally distributed with mean zero and variance σ 2 :
y = f(x) + ɛ (16.1)
The goal of machine learning under a function estimation approach is to make an estimate
of y, denoted by ŷ by attempting to find a function ˆf which "best" approximates f. Once this ˆf
has been obtained it is straightforward to estimate any new ŷ given a new predictor vector x test .
In some instances however it is harder to choose between values when classifying or regressing,
as there may be some ambiguity from the feature vectors. This motivates the second approach
225