advanced-algorithmic-trading

203
team at OpenAI spend significant time looking at such problems. They have released an opensource
toolkit to allow straightforward testing of new RL agents known as the OpenAI Gym[28].
Unfortunately RL, along with MDP and POMDP, are not within the scope of this book.
Note that in this book continuous-time Markov processes are not considered. In quantitative
trading the time unit is often given via ticks or bars of historical asset data. However, if the objective
is to price derivatives contracts then the continuous-time machinery of stochastic calculus
would be utilised.
14.1.1 Markov Model Mathematical Specification
This section as well as that on the Hidden Markov Model Mathematical Specification will closely
follow the notation and model specification of Murphy (2012)[71].
In quantitative finance the analysis of a time series is often of primary interest. As has been
discussed in previous chapters such a time series generally consists of a sequence of T discrete
observations X 1 , . . . , X T . An important assumption about Markov Chain models is that at any
time t, the observation X t captures all of the necessary information required to make predictions
about future states. This assumption will be utilised in the following specification.
Formulating the Markov Chain into a probabilistic framework allows the joint density function
for the probability of seeing the observations to be written as:
p(X 1:T ) = p(X 1 )p(X 2 | X 1 )p(X 3 | X 2 ) . . . (14.1)
T∏
= p(X 1 ) p(X t | X t−1 ) (14.2)
t=2
This states that the probability of seeing sequences of observations is given by the probability
of the initial observation multiplied T − 1 times by the conditional probability of seeing the
subsequent observation, given that the previous observation has occurred. It will be assumed in
this chapter that the latter term known as the transition function p(X t | X t−1 ) will itself be timeindependent.
In addition since the market regime models considered in this book will consist
of small, discrete numbers of regimes the type of model under consideration is a Discrete-State
Markov Chain (DSMC).
If there are K separate possible states (regimes) for the model to be in at any time t then
the transition function can be written as a transition matrix that describes the probability of
transitioning from state j to state i at any time-step t. Mathematically the elements of the
transition matrix A are given by:
A ij = p(X t = j | X t−1 = i) (14.3)
As an example it is possible to consider a simple two-state Markov Chain Model. Figure
14.1 represents the numbered states as circles while the arcs represent the probability of jumping
from state to state:
Notice that the probabilities sum to unity for each state, i.e. α + (1 − α) = 1. The transition
matrix A for this system is a 2 × 2 matrix given by: