MACHINE LEARNING TECHNIQUES - LASA

160
7 Markov-Based Models
Except for ICA, recurrent neural networks and continuous Hopfield network, these lectures notes
have covered primarily methods to encode static datasets, i.e. data that do not vary in time. Being
able to predict patterns that vary in time is fundamental to many applications. Such patterns are
often referred to as time-series.
In this chapter, we cover two models to encode time-series based on the hypothesis of a firstorder
Markov Process, which we describe next.
7.1 Markov Process
A Markov Process is by definition discrete and assumes that the evolution of a system can be
described by a finite set of states. If
N particular values { s ,... 1
s
N}
, i.e. x { s ,... 1
sN}
x is the state of the system at time, then x can take only
t
∈ .
A first-order markov process is a process by which the future state of the system can be
determined solely by knowing the current state of the system. In other words, if one has made T
consecutive observation of the state x of the system: { t} t 1
the system at the next time step
T 1
current state, i.e.:
T
X x =
= , the probability that the state of
x +
take any of the N possible value is determined solely by the
( t+ 1 j
|
t, t− 1,...., 0) ( t+
1 j
|
t)
p x = s x x x = p x = s x
(7.1)
When modeling time-series, to assume that a process description is first-order markov is
advantageous in that it simplifies greatly computation. Instead of computing the complete
x , x ,...., x
conditional on all previously observed states , one must solely compute the
t t−
1 0
conditional on the last state. We will see how this is exploited in two different techniques widely
used in machine learning next.
© A.G.Billard 2004 – Last Update March 2011