MACHINE LEARNING TECHNIQUES - LASA

164
Figure 7-3: The Forward Procedure used in the Baum-Welch algorithm for estimating the HMM parameters
The forward procedure is thus iterative with an initialization step, an induction step (typical of
dynamic programming) and termination step, see Figure 7-3 right.
It is easy to see that the same principle can be extended backwards and, thus, that one can infer
the probability of being in a particular state by starting from the last state and building one’s way
back in time. This is referred to as the backward procedure and is illustrated in Figure 7-4.
Figure 7-4: The backward procedure in the Baum-Welch algorithm for estimating the HMM parameters
While solving with Equation (7.2) would take on the order of 2T*N T computation steps, the
forward or backward procedures reduce this computation greatly and take instead on the order of
N 2 T computation steps each. One may wonder what the advantage would be to do the
computation either forwards or backwards. There is no direct advantage to do either. These two
procedures are however crucial to the estimation of the parameters of the HMM. It is easy to see
that, if one can compute the probability to be in state i at time t with the forward procedure and
the probability to be in state j at time t+1 with the backward procedure, then one can combine
© A.G.Billard 2004 – Last Update March 2011