advanced-algorithmic-trading

Chapter 14
Hidden Markov Models
A consistent challenge for quantitative traders is the frequent behaviour modification of financial
markets, often abruptly, due to changing periods of government policy, regulatory environment
and other macroeconomic effects. Such periods are known as market regimes. Detecting such
changes is a common, albeit difficult, process undertaken by quantitative market participants.
These various regimes lead to adjustments of asset returns via shifts in their means, variances,
autocorrelation and covariances. This impacts the effectiveness of time series methods that rely
on stationarity. In particular it can lead to dynamically-varying correlation, excess kurtosis ("fat
tails"), heteroskedasticity (volatility clustering) and skewed returns.
There is a clear need to effectively detect these regimes. This aids optimal deployment of
quantitative trading strategies and tuning the parameters within them. The modeling task then
becomes an attempt to identify when a new regime has occurred adjusting strategy deployment,
risk management and position sizing criteria accordingly.
A principal method for carrying out regime detection is to use a statistical time series technique
known as a Hidden Markov Model[5]. These models are well-suited to the task since they
involve inference on "hidden" generative processes via "noisy" indirect observations correlated
to these processes. In this instance the hidden, or latent, process is the underlying regime state,
while the asset returns are the indirect noisy observations that are influenced by these states.
This chapter will discuss the mathematical theory behind Hidden Markov
Models (HMM) and how they can be applied to the problem of regime
detection for quantitative trading purposes.
The discussion will begin by introducing the concept of a Markov Model[8] and their associated
categorisation, which depends upon the level of autonomy in the system as well as how
much information about the system is observed. The discussion will then focus specifically on
the architecture of HMM as an autonomous process with partially observable information.
As in the previous chapter on State Space Models and the Kalman Filter, the inference
concepts of filtering, smoothing and prediction will be outlined. Specific algorithms such as the
Forward Algorithm[14] and Viterbi Algorithm[18] exist to solve these inference problems, but
their derivations are beyond the scope of this book.
In this chapter the HMM will be applied to the S&P500 returns series to detect regimes. In
a later chapter these detection overlays will be added to quantitative trading strategies via a
201