Segmentation of heterogeneous document images : an ... - Tel

Table 3.2: AVERAGE PERFORMANCE OF SEVERAL CLASSIFIERS FOR
TEXT GRAPHICS SEPARATION
Classifier Text recall Graphics recall Text precision Graphics precision Accuracy Comments
LogitBoost 93.08 93.03 92.43 92.38 92.73% 500 weak learner
Decision Tree (gini-index) 95.33 88.19 90.47 94.14 92.94%+/-0.66
GentleBoost 93.32 90.63 90.92 93.12 91.97
GiniBoost 93.24 90.01 90.34 93.01 91.62
AutoMLP 95.32 80.54 90.7 89.64 90.38%+/-0.45 51 neurons
Linear C-SVC 98.21 57.06 81.99 94.12 84.45%+/-0.33 SV: 6470, C = 1.0
Decision Tree (accuracy) 96.65 60.69 83.03 90.1 84.62%+/-2.61
LDA 98.99 46.86 78.76 95.89 81.56%+/-0.48
Nave Bayes 98.67 43.26 77.58 94.22 80.14%+/-0.77
Linear C-SVC 99.81 37.68 76.12 99 79.03%+/-0.31 SV: 7942, C = 0.0
Decision Tree (gain-ratio) 99.82 15.43 70.12 97.68 71.57%+/-6.34
Based on our evaluation, LogitBoost classifier is selected.
tel-00912566, version 1 - 2 Dec 2013
3.5 LogitBoost for classification
Boosting is a machine learning algorithm for performing supervised learning.
It iteratively combines the performance of many weak classifiers to produce
a final strong classifier. A weak classifier is only required to be better than
chance, however the combination of them, results in a strong classifier which often
outperforms most strong classifiers such as Neural Networks and SVMs [37].
We use boosted decision trees which are popular weak classifiers used in
boosting scheme. Our boosted model is based on a large number of training
examples (x i , y i ) with x i ∈ R K and y i ∈ {−1, +1}. x i is a vector with K elements
(K = 15 is the number of our features). The desired two-class output is
encoded as −1 and +1.
Various boosting methods are known such as AdaBoost [35], LogitBoost and
GentleBoost [36]. Regarding the overall structure, all of them are similar. Initially
each instance of data in our training dataset is assigned the same weight as
the other data points. Then a weak classifier is trained on the weighted training
data. A weight indicates the importance of a data point for the classification.
In each iteration the weights of misclassified data are increased and the weights
of each correctly classified sample are decreased. As a result the new classifier
focuses on the examples which have eluded correct classification in previous iterations.
LogitBoost is very similar to the original AdaBoost algorithm. If one considers
AdaBoost as a generalized additive model and then applies the cost functional
of logistic regression, LogitBoost algorithm can be derived. The implementation
of LogitBoost comes from OpenCV machine learning library.
LogitBoost algorithm is as follows:
1. Given N instances (x i , y i ) with x i ∈ R K , y i ∈ {−1, +1}
2. Start with weights w i = 1/N, i = 1, ..., N, F (x) = 0 and probability estimates
p(x i ) = 1 2
46