advanced-algorithmic-trading

29
x = np.linspace(0, 1, 100)
params = [
(0.5, 0.5),
(1, 1),
(4, 3),
(2, 5),
(6, 6)
]
for p in params:
y = beta.pdf(x, p[0], p[1])
plt.plot(x, y, label="$\\alpha=%s$, $\\beta=%s$" % p)
plt.xlabel("$\\theta$, Fairness")
plt.ylabel("Density")
plt.legend(title="Parameters")
plt.show()
Essentially, as α becomes larger the bulk of the probability distribution moves towards one
(a coin biased to come up heads more often), whereas an increase in β moves the distribution
towards zero (a coin biased to come up tails more often).
However, if both α and β increase then the distribution begins to narrow. If α and β increase
equally, then the distribution will peak over θ = 0.5, which occurs when the coin is fair.
Why have we chosen the beta function as our prior? There are a couple of reasons:
• Support - It is defined on the interval [0, 1], which is the same interval that θ exists over.
• Flexibility - It possesses two shape parameters known as α and β, which give it significant
flexibility. This flexibility provides us with a lot of choice in how we model our beliefs.
However, perhaps the most important reason for choosing a beta distribution is because it is
a conjugate prior for the Bernoulli distribution.
Conjugate Priors
In Bayes’ rule above we can see that the posterior distribution is proportional to the product of
the prior distribution and the likelihood function:
P (θ|D) ∝ P (D|θ)P (θ) (3.11)
A conjugate prior is a choice of prior distribution that when coupled with a specific type
of likelihood function provides a posterior distribution that is of the same family as the prior
distribution.
The prior and posterior both have the same probability distribution family, but with differing
parameters.
Conjugate priors are extremely convenient from a calculation point of view as they provide
closed-form expressions for the posterior thus negating any complex numerical integration.
In our case if we use a Bernoulli likelihood function and a beta distribution as the choice of
our prior it immediately follows that the posterior will also be a beta distribution.