MACHINE LEARNING TECHNIQUES - LASA

166
Finally, the emission probabilities in the discrete case are given by computing the expected
number of times one will be in each particular state and that the observation took a particular
value v k
, i.e. ot
= v :
k
bˆ v
tot
= k
( )
{: v }
i k T
= ∑ ∑ =
t 1
γ () i
t
γ () i
t
(7.7)
In the continuous case, one builds a continuous estimate of the b ˆi , e.g. by building a GMM for
each state.
The ^ symbol in each of the above designates the new estimate of the values for each of the
HMM parameter.
7.2.3 Determining the number of states
A crucial parameter to HMM (and similarly to GMM) is the number of states K. Increasing K
increases the number of parameters and so computation. Of course, increasing the number of
states allows a finer description of the density and hence increases the likelihood of the model
given the data. This increase may however be negligeable in contrast to the increase in
computation time due to the increase of parameters. One must then find a tradeoff between
increasing the number of parameters and improving the likelihood of the model. This is
encapsulated in the Aikaike Information Criterion (AIC):
where L is the likelihood of the model.
AIC= − 2ln L+ 2K
(7.8)
This criterion was however lately replaced by the Bayesian Information Criterion that also weights
the fact that the estimate of the number of parameters depends on the number of data points.
Sparsity in the data may require a lot of parameters to encapsulate non-linearities in these. As the
number of datapoints M increases, so does the influence of the number of parameters.
BIC =− 2lnL + K ln( M )
(7.9)
An alternative to the above two criteria is the Deviation Information Criterion (DIC)
{ ( )} { }
( ) ( ) ( )
DIC = E D K - D E K with D K 2 ln p X | K
=− (7.10)
Again the smaller the DIC, the better the model fits the data. DIC favors a good fit, while
penalizing models by measuring the effective number of parameters required for a similar level of
fit.
© A.G.Billard 2004 – Last Update March 2011