Segmentation of heterogeneous document images : an ... - Tel

probability p(y|x) is maximized according to the trained CRF model.
Finding the optimal label configuration in a linear-chain conditional random
field or graphical models with tree-like structure is a straightforward process
using the Viterbi algorithm. However, in two-dimensional CRF, no exact
method exists for label decoding. Nearly all methods that perform decoding in
two-dimensional CRFs are approximation methods that iteratively perform inference
and change label configuration until they reach to a predefined number
of iterations or until they converge. Examples of such methods are Monte Carlo
methods [3], Loopy Belief Propagation [64], variational methods [43]. While
the success of these methods is proven in some application like Turbo decoding
[67], in the domain of computer vision [31], the precise conditions under which
these methods will converge are still not well understood. Furthermore, since
we work on high-resolution document images with typically 300 dpi, methods
that demand high number of iterations to converge, are impractical.
tel-00912566, version 1 - 2 Dec 2013
Another simple to implement solution is Iterated Conditional Models (ICM).
ICM is an iterative method proposed by Besag in 1986 [9]. Instead of maximizing
the probability as a whole, the method tries to maximize the conditional
probability of each site by considering only its neighbors. At each iteration, the
algorithm chooses sites from left to right and top to bottom and estimates the
probability for all possible label configurations for the site in question. Then it
picks the label that maximizes the local probability. Finding the global maximum
is not guaranteed, but the method converges to a local maximum. Moreover,
if instead of using all the neighbors of a site, we restrict the label of the
site to be dependent merely on the sites that we have already visited in one
iteration, then it takes just one iteration for the ICM to converge. Therefore,
ICM is the method that we choose for label decoding.
4.6 Training (parameter/weight estimation)
In this section, we discuss how to estimate the parameters λ j of a conditional
random field. In general, conditional random fields may be trained with latent
variables or for structure learning. However, we are provided with fully labeled
data, which is the simplest case for training.
Maximum likelihood is the foundation for training CRFs. Weights are chosen
such that the training data has the highest probability under the model. The
conditional log-likelihood of a set of training sites (s s , y s ) using λ as parameters
is given by:
l λ = ∑ ( F
)
∑
λ k f k (y i,i∈N(s) , y s , x i,i∈N(s) , x s ) − log Z(x s , λ) .
s∈S
k=1
where S is the total number of sites in training dataset and F is the total
number of feature functions. Differentiating the log-likelihood function with
respect to parameter λ k is given by:
68