unified detection and recognition for reading text in scene images

More documents

Recommendations

Info

5.8 Average recognition accuracy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.9 Relative improvement of the factored over independent method . . . . . . . 93 5.10 Relative gains of features for different tasks . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.11 Scatterplot of normalized feature gains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.1 Examples signs that make prior word segmentation difficult . . . . . . . . . . 100 6.2 Broken characters from a standard segmentation algorithm . . . . . . . . . . . 100 6.3 Example text string segmentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 6.4 Dynamic programming variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.5 Partial finite state word graph. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 6.6 Rectification of even and odd steerable pyramid filter outputs . . . . . . . . 110 6.7 Pre-processing for recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 6.8 Synthetic font training data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 6.9 Comparison of beam search strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 6.10 Comparison of recognition methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 6.11 Example input and binarized images with OmniPage output . . . . . . . . . 118 6.12 Example recognition comparison at lower resolutions . . . . . . . . . . . . . . . . 119 6.13 Examples of failed segmentation and/or recognition . . . . . . . . . . . . . . . . . 121 6.14 Example text detection probabilities and candidate regions . . . . . . . . . . . 124 6.15 Recognition of candidate regions detected at multiple scales . . . . . . . . . . 125 6.16 Example scene images used for qualitative comparison. . . . . . . . . . . . . . . 126 6.17 Example output of the end-to-end system . . . . . . . . . . . . . . . . . . . . . . . . . . 127 6.18 Example detected text regions from OmniPage . . . . . . . . . . . . . . . . . . . . . 128 xvi
CHAPTER 1 INTRODUCTION The first attempt to build machines that could read occurred nearly two-hundred years ago [102]. Since then, computers have barely reached the reading level accuracy of a second-grade child [99]. Many facilities are involved in human reading [97], and similarly, there has been a vast amount of research on computational methods for text recognition. The bulk of text recognition research over the last fifty years has focused on scanned documents and faxes, but interest in indexing text in video and scene images has grown rapidly in the last decade. Detecting and recognizing text in scenes involves some of the same issues as document processing, but there are also several new problems to face. Recent research in these areas has shown some promise, but there is still much work to be done. The problem of developing a computational model for reading is broad, encompassing many facets of information. There are any number of typefaces, and they may be encountered at many legible sizes. Text is usually printed on a surface, and humans do not generally require that the surface be fronto-parallel to their eyes in order to read it. Additionally, lighting conditions may vary widely and designers use many colors for text and background. All of these factors conspire to make the robust reading problem very challenging. Many processes are undoubtedly at work when humans read. Reading text in arbitrary images likely goes hand in hand with scene interpretation and object recognition. Humans might (subconsciously) ask: Where do I expect to see text? Does text at a particular size and location contradict a sensible scene interpretation? Recognizing these factors will allow us to take some necessary steps toward a unified view of scene interpretation, even if such propositions are not directly modeled. One of the central limitations of many current approaches is that they operate primarily in a feed-forward, bottom-up, pipelined architecture that isolates the many tasks and information involved in reading. The result is a system that commits errors from which it cannot recover and has components that lack access to relevant information. We propose a system that in its design, training, and operation is more integrated. First, we present a contextual model for text detection (Chapter 3). Although we improve detection results by taking a broader view of the input scene than the typical sliding window approach, the model is still illiterate. That is, it can find what it thinks looks like text, but is unable to interpret it. Next, we focus on the recognition task, introducing a model that integrates several information sources, including 1
Page 1 and 2: UNIFIED DETECTION AND RECOGNITION F
Page 3 and 4: UNIFIED DETECTION AND RECOGNITION F
Page 5 and 6: ACKNOWLEDGEMENTS As soon as one beg
Page 7 and 8: ABSTRACT UNIFIED DETECTION AND RECO
Page 9 and 10: CONTENTS Page ACKNOWLEDGEMENTS . .
Page 11 and 12: 4.3.2.1 Model Training . . . . . .
Page 13 and 14: LIST OF TABLES Table Page 1.1 Diffi
Page 15: 3.11 Visual comparison of local and
Page 19 and 20: Figure 1.2. Images for document pag
Page 21 and 22: T he P h oto S p e cialists Input O
Page 23 and 24: Figure 1.4. Small text in an image
Page 25 and 26: section, we review some of the appr
Page 27 and 28: Kusachi et al. [62] have a multi-re
Page 29 and 30: 1.3.3 Adaptive Recognition Several
Page 31 and 32: ather than integrated systems. In a
Page 33 and 34: Background Faces Allen Andrew Keith
Page 35 and 36: CHAPTER 2 DISCRIMINATIVE MARKOV FIE
Page 37 and 38: If we have a family of sets C = {C
Page 39 and 40: ̂θ ≡ arg max p (θ | D, I) (2.6
Page 41 and 42: functions, i.e., θ = [ θ A θ ] B
Page 43 and 44: P L (θ; α) = −α ‖θ‖ 1 (2.
Page 45 and 46: we drop the dependence of the messa
Page 47 and 48: CHAPTER 3 TEXT AND SIGN DETECTION B
Page 49 and 50: For example, since neighboring regi
Page 51 and 52: Figure 3.3. Decomposition of images
Page 53 and 54: 3.3.1 Feature Overview Rather than
Page 55 and 56: • Raw pixel statistics (e.g., ran
Page 57 and 58: 3.4.2 Experimental Procedure For ou
Page 59 and 60: Figure 3.7. Example contextual dete
Page 61 and 62: Table 3.1. Sign detection results w
Page 63 and 64: Table 3.2. Sign detection results a
Page 65 and 66: In conclusion, the “default” pr
Page 67 and 68:
CHAPTER 4 UNIFYING INFORMATION FOR
Page 69 and 70:
will outline the details of our mod
Page 71 and 72:
y 1 01 01 01 0 0 01 1 1 0 0 01 1 1
Page 73 and 74:
4.2.2 Language Model Properties of
Page 75 and 76:
5 Basis Functions Learned Function
Page 77 and 78:
node for the factor C, while w B is
Page 79 and 80:
Figure 4.5. Examples of sign evalua
Page 81 and 82:
Lexicon The lexicon we use is deriv
Page 83 and 84:
40 40 Number of Queries 30 20 10 Nu
Page 85 and 86:
and could be modeled directly if we
Page 87 and 88:
Figure 4.9. Examples from the sign
Page 89 and 90:
Model Correct Free checking 31 BOLT
Page 91 and 92:
4.3.4.2 Lexicon Model Here we discu
Page 93 and 94:
CHAPTER 5 UNIFYING DETECTION AND RE
Page 95 and 96:
character and these were directly u
Page 97 and 98:
L C ( θ C ; F, D ) ≡ ∑ k log p
Page 99 and 100:
For the independent method, each ca
Page 101 and 102:
Wavelet Transform Scale 0.3 0.25 0.
Page 103 and 104:
Scale −0.02 0 0.02 0.04 0.06 0.08
Page 105 and 106:
Figure 5.6. Sample images used in e
Page 107 and 108:
1 Avg. Categ. Accuracy 0.95 0.9 0.8
Page 109 and 110:
Relative Improvement 0.5 0.4 0.3 0.
Page 111 and 112:
Recognition Gain 1 0.8 0.6 0.4 0.2
Page 113 and 114:
of further special recognition. Hav
Page 115 and 116:
CHAPTER 6 THE ROBUST READER In the
Page 117 and 118:
6.2 Semi-Markov Model for Segmentat
Page 119 and 120:
6.2.1.2 Character Bigrams As in ear
Page 121 and 122:
exponential of the corresponding Ma
Page 123 and 124:
is signalled, and this string may e
Page 125 and 126:
and k. This joint space over i and
Page 127 and 128:
each location, all the rectified fi
Page 129 and 130:
This procedure is outlined visually
Page 131 and 132:
(e.g., cs, Bk, Kr, Nb, pd, Tl), we
Page 133 and 134:
18.8 18.6 18.4 KL N−Best Ratio Er
Page 135 and 136:
σ Image Output Binarized OmniPage
Page 137 and 138:
F I B E IA A R T C E N T E R (a) F
Page 139 and 140:
6.4.5 End-to-End Demonstration In t
Page 141 and 142:
T he P h oto S p e cialists The Pro
Page 143 and 144:
T he P hoto S p ecialists L L O Y D
Page 145 and 146:
Table 6.2. Comparison of recognitio
Page 147 and 148:
mercial document recognition softwa
Page 149 and 150:
In addition to the models, we have
Page 151 and 152:
[12] Blum, Avrim, and Langley, Pat.
Page 153 and 154:
[37] Geman, Stuart, and Geman, Dona
Page 155 and 156:
[63] Lafferty, John, McCallum, Andr
Page 157 and 158:
[89] Pal, Chris, Sutton, Charles, a
Page 159 and 160:
[114] Torralba, Antonio, Murphy, Ke
Page 161:
[139] Zhou, Yaqian, Wenb, Fuliang,
show all

unified detection and recognition for reading text in scene images

Create successful ePaper yourself

Delete template?

Save as template?