unified detection and recognition for reading text in scene images

More documents

Recommendations

Info

2. DISCRIMINATIVE MARKOV FIELDS . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.1 Basic Model Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.1.1 Probability for Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.1.2 Model Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.2 Model Training. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.2.1 Model Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.2.1.1 Parameter Decoupling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.2.1.2 Piecewise Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.2.2 Model Priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.3 Model Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.3.1 Prediction Strategies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.3.2 Belief Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3. TEXT AND SIGN DETECTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.2 Markov Field for Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.2.1 Detection Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.2.2 Model Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.3 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 3.3.1 Feature Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.3.2 Feature Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.4.1 Experimental Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.4.2 Experimental Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.4.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.4.4 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 3.5 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.5.1 Learned Layout Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.5.2 Multi-Scale Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.5.3 Spatial Context without Boundary Cases . . . . . . . . . . . . . . . . . . . . . 50 3.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 4. UNIFYING INFORMATION FOR READING . . . . . . . . . . . . . . . . . . . 51 4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 4.2 Markov Models for Recognition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4.2.1 Appearance Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 4.2.2 Language Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.2.3 Similarity Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.2.4 Lexicon Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4.2.4.1 Lexicon Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.2.4.2 Sparse Belief Propagation . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.3.1 Experimental Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.3.2 Experimental Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 x
4.3.2.1 Model Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 4.3.2.2 Model Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.3.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.3.3.1 Unified Similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.3.3.2 Lexicon-Based Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.3.4 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 4.3.4.1 Similarity Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 4.3.4.2 Lexicon Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 4.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 5. UNIFYING DETECTION AND RECOGNITION . . . . . . . . . . . . . . . . 77 5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 5.2 Simple Models for Detection and Recognition . . . . . . . . . . . . . . . . . . . . . . . 79 5.2.1 Model Formulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 5.2.2 Feature Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.3 Detection and Recognition Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 5.4.1 Experimental Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5.4.2 Experimental Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 5.4.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.4.4 Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5.5 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 5.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 6. THE ROBUST READER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 6.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 6.2 Semi-Markov Model for Segmentation and Recognition . . . . . . . . . . . . . . 101 6.2.1 Segmentation and Recognition Model . . . . . . . . . . . . . . . . . . . . . . . 101 6.2.1.1 Character Appearance . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 6.2.1.2 Character Bigrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6.2.1.3 Character Overlap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6.2.1.4 Character Gap. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6.2.1.5 Lexicon Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6.2.2 Model Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6.2.2.1 Lexicon-Free Parsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.2.2.2 Lexicon-Based Parsing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.2.2.3 Computational Complexity . . . . . . . . . . . . . . . . . . . . . . . . 107 6.2.2.4 Sparse Lexicon-Based Parsing. . . . . . . . . . . . . . . . . . . . . . 108 6.3 Features and Pre-Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6.3.1 Image Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6.3.2 Pre-Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 6.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.4.1 Experimental Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.4.2 Experimental Procedure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 xi
Page 1 and 2: UNIFIED DETECTION AND RECOGNITION F
Page 3 and 4: UNIFIED DETECTION AND RECOGNITION F
Page 5 and 6: ACKNOWLEDGEMENTS As soon as one beg
Page 7 and 8: ABSTRACT UNIFIED DETECTION AND RECO
Page 9: CONTENTS Page ACKNOWLEDGEMENTS . .
Page 13 and 14: LIST OF TABLES Table Page 1.1 Diffi
Page 15 and 16: 3.11 Visual comparison of local and
Page 17 and 18: CHAPTER 1 INTRODUCTION The first at
Page 19 and 20: Figure 1.2. Images for document pag
Page 21 and 22: T he P h oto S p e cialists Input O
Page 23 and 24: Figure 1.4. Small text in an image
Page 25 and 26: section, we review some of the appr
Page 27 and 28: Kusachi et al. [62] have a multi-re
Page 29 and 30: 1.3.3 Adaptive Recognition Several
Page 31 and 32: ather than integrated systems. In a
Page 33 and 34: Background Faces Allen Andrew Keith
Page 35 and 36: CHAPTER 2 DISCRIMINATIVE MARKOV FIE
Page 37 and 38: If we have a family of sets C = {C
Page 39 and 40: ̂θ ≡ arg max p (θ | D, I) (2.6
Page 41 and 42: functions, i.e., θ = [ θ A θ ] B
Page 43 and 44: P L (θ; α) = −α ‖θ‖ 1 (2.
Page 45 and 46: we drop the dependence of the messa
Page 47 and 48: CHAPTER 3 TEXT AND SIGN DETECTION B
Page 49 and 50: For example, since neighboring regi
Page 51 and 52: Figure 3.3. Decomposition of images
Page 53 and 54: 3.3.1 Feature Overview Rather than
Page 55 and 56: • Raw pixel statistics (e.g., ran
Page 57 and 58: 3.4.2 Experimental Procedure For ou
Page 59 and 60: Figure 3.7. Example contextual dete
Page 61 and 62:
Table 3.1. Sign detection results w
Page 63 and 64:
Table 3.2. Sign detection results a
Page 65 and 66:
In conclusion, the “default” pr
Page 67 and 68:
CHAPTER 4 UNIFYING INFORMATION FOR
Page 69 and 70:
will outline the details of our mod
Page 71 and 72:
y 1 01 01 01 0 0 01 1 1 0 0 01 1 1
Page 73 and 74:
4.2.2 Language Model Properties of
Page 75 and 76:
5 Basis Functions Learned Function
Page 77 and 78:
node for the factor C, while w B is
Page 79 and 80:
Figure 4.5. Examples of sign evalua
Page 81 and 82:
Lexicon The lexicon we use is deriv
Page 83 and 84:
40 40 Number of Queries 30 20 10 Nu
Page 85 and 86:
and could be modeled directly if we
Page 87 and 88:
Figure 4.9. Examples from the sign
Page 89 and 90:
Model Correct Free checking 31 BOLT
Page 91 and 92:
4.3.4.2 Lexicon Model Here we discu
Page 93 and 94:
CHAPTER 5 UNIFYING DETECTION AND RE
Page 95 and 96:
character and these were directly u
Page 97 and 98:
L C ( θ C ; F, D ) ≡ ∑ k log p
Page 99 and 100:
For the independent method, each ca
Page 101 and 102:
Wavelet Transform Scale 0.3 0.25 0.
Page 103 and 104:
Scale −0.02 0 0.02 0.04 0.06 0.08
Page 105 and 106:
Figure 5.6. Sample images used in e
Page 107 and 108:
1 Avg. Categ. Accuracy 0.95 0.9 0.8
Page 109 and 110:
Relative Improvement 0.5 0.4 0.3 0.
Page 111 and 112:
Recognition Gain 1 0.8 0.6 0.4 0.2
Page 113 and 114:
of further special recognition. Hav
Page 115 and 116:
CHAPTER 6 THE ROBUST READER In the
Page 117 and 118:
6.2 Semi-Markov Model for Segmentat
Page 119 and 120:
6.2.1.2 Character Bigrams As in ear
Page 121 and 122:
exponential of the corresponding Ma
Page 123 and 124:
is signalled, and this string may e
Page 125 and 126:
and k. This joint space over i and
Page 127 and 128:
each location, all the rectified fi
Page 129 and 130:
This procedure is outlined visually
Page 131 and 132:
(e.g., cs, Bk, Kr, Nb, pd, Tl), we
Page 133 and 134:
18.8 18.6 18.4 KL N−Best Ratio Er
Page 135 and 136:
σ Image Output Binarized OmniPage
Page 137 and 138:
F I B E IA A R T C E N T E R (a) F
Page 139 and 140:
6.4.5 End-to-End Demonstration In t
Page 141 and 142:
T he P h oto S p e cialists The Pro
Page 143 and 144:
T he P hoto S p ecialists L L O Y D
Page 145 and 146:
Table 6.2. Comparison of recognitio
Page 147 and 148:
mercial document recognition softwa
Page 149 and 150:
In addition to the models, we have
Page 151 and 152:
[12] Blum, Avrim, and Langley, Pat.
Page 153 and 154:
[37] Geman, Stuart, and Geman, Dona
Page 155 and 156:
[63] Lafferty, John, McCallum, Andr
Page 157 and 158:
[89] Pal, Chris, Sutton, Charles, a
Page 159 and 160:
[114] Torralba, Antonio, Murphy, Ke
Page 161:
[139] Zhou, Yaqian, Wenb, Fuliang,
show all

unified detection and recognition for reading text in scene images

Create successful ePaper yourself

Delete template?

Save as template?