advanced-algorithmic-trading

232
Figure 17.1:
(2012)[71]
Plot of p(y | x, θ) against y and x, influenced from a similar plot in Murphy
the feature increase.
17.2.1 Basis Function Expansion
One benefit of a probabilistic interpretation is the ease of modelling non-linear relationships by
replacing the feature vector x with a transformation function φ(x):
p(y | x, θ) = (y | β T φ(x), σ 2 ) (17.3)
For x = (1, x 1 , x 2 , x 3 ), say, it is possible to create a φ that includes higher order terms such
as cross-terms, e.g.:
φ(x) = (1, x 1 , x 2 1, x 2 , x 2 2, x 1 x 2 , x 3 , x 2 3, x 1 x 3 , . . .) (17.4)
A key point is that while this function is not linear in the features x it is still linear in the
parameters β. Hence it is still called a linear regression.
Such a modification using a transformation function φ is known as a basis function expansion.
These transformations can be used to generalise linear regression to many non-linear
models. One such approach is given in the chapter on Support Vector Machines using the "kernel
trick".