advanced-algorithmic-trading

99
Definition 10.2.1. Strictly Stationary Series. A time series model, {x t }, is strictly stationary if
the joint statistical distribution of the elements x t1 , . . . , x tn is the same as that of x t1+m, . . . , x tn+m,
∀t i , m.
One can think of this definition as simply that the distribution of the time series is unchanged
for any abritrary shift in time.
In particular, the mean and the variance are constant in time for a strictly stationary series
and the autocovariance between x t and x s (say) depends only on the absolute difference of t and
s, |t − s|.
We will be revisiting strictly stationary series in future chapters.
10.3 Akaike Information Criterion
I mentioned in previous chapters that we would eventually need to consider how to choose between
separate "best" models. This is true not only of time series analysis, but also of machine learning
and, more broadly, statistics in general.
The two main methods we will use, for the time being, are the Akaike Information Criterion
(AIC) and the Bayesian Information Criterion (BIC).
We will briefly consider the AIC, as it will be used in the next section when we come to
discuss the ARMA model.
AIC is essentially a tool to aid in model selection. That is, if we have a selection of statistical
models (including time series), then the AIC estimates the "quality" of each model, relative to
the others that we have available.
It is based on information theory, which is a highly interesting, deep topic that unfortunately
is beyond the scope of this book. It attempts to balance the complexity of the model, which in
this case means the number of parameters, with how well it fits the data. Here is a definition:
Definition 10.3.1. Akaike Information Criterion. If we take the likelihood function for a statistical
model, which has k parameters, and L maximises the likelihood, then the Akaike Information
Criterion is given by:
AIC = −2log(L) + 2k (10.1)
The preferred model, from a selection of models, has the minimum AIC of the group. You
can see that the AIC grows as the number of parameters, k, increases, but is reduced if the
negative log-likelihood increases. Essentially it penalises models that are overfit.
We are going to be creating AR, MA and ARMA models of varying orders and one way to
choose the "best" model fit for a particular dataset is to use the AIC.
10.4 Autoregressive (AR) Models of order p
The first model to be considered is the Autoregressive model of order p, often shortened to
AR(p).