unified detection and recognition for reading text in scene images

More documents

Recommendations

Info

to provide thoughtful critiques—born from his decades of experience—of my often wandering ideas. Professor Erik G. Learned-Miller’s arrival at UMass could not have been timed better. I am immensely grateful for his insistence on doing work I could take pride in. His scientific and career mentorship have been truly valuable, and I will surely miss our frequent intellectual sparring and sharpening of each other’s ideas. Professor Edward M. Riseman also had a hand in shaping ideas early in my graduate career. To Professor David Jensen I am indebted for a great deal of practical career advice. His personal mentorship helped me navigate the waters of late-stage graduate work, and his insightful course on research methods offered me lessons I hope I will hold for an entire career. Many others at UMass have contributed to the efforts of this thesis in ways large and small, direct and indirect. I wish to thank: Charles Sutton for countless theoretical and technical discussions on topics often more detailed than he had probably bargained for; Aron Culotta for oodles of technical references; Trevor Strohman for many exciting, deep, and unexpectedly long conversations about nearly every topic under the sun; Vidit Jain for being a reliable sounding board and mathematical assistant; Andy Fast and Anne-Marie Strohman for sharpening manuscript prose; Piyanuch (Pla) Silapachote and Marwan A. Mattar for collaboration on the VIDI project and acquiring data used in this thesis; Professor Joseph Horowitz, George Bissias, Richard Weiss, David Walker Duhon, and Chris Pal for opportunities to collaborate and share ideas on interesting projects; Sarah Osentoski for giggling, commiseration, and Nebraska football games; and, all the tea totallers, coffee czars, bagel barons, donut dukes, and graduate representatives that make the UMass Computer Science Department the very special community that it is. This work also would not have been possible without the support of several funding agencies. Various portions have been supported by the National Science Foundation under the NSF grants IIS-0100851, IIS-0326249 and IIS-0546666, and the National Institutes of Health under NIH grant 1R21EY018398-01. Additional support was provided by Central Intelligence Agency and the National Security Agency. Time outside the department has been at least as important as my time inside. The people of the First Baptist Church of Amherst have truly been a family to me. I have been blessed time and again by them. To the many generations of the “somethings” growth group I humbly owe my thanks. You have fed me spiritually, sustained me emotionally, and nourished me physically. From Valley View, to Sugarloaf, to Blingo, to the Fireside, I can hardly think of a single thing that has meant so much over six years as Tuesday nights. And to my wonderful bride Jennifer Nicole I am so thankful for constant encouragement and an inspirational devotion. Finally, the greatest acknowledgement is for a debt redeemed by Jesus Christ. By His grace and mercy can I strive to do all things to the glory of God. 146:8 . 9:25 vi
ABSTRACT UNIFIED DETECTION AND RECOGNITION FOR READING TEXT IN SCENE IMAGES MAY 2008 JEROD J. WEINMAN B.Sc., ROSE-HULMAN INSTITUTE OF TECHNOLOGY M.Sc., UNIVERSITY OF MASSACHUSETTS AMHERST Ph.D., UNIVERSITY OF MASSACHUSETTS AMHERST Directed by: Professor Allen R. Hanson and Professor Erik G. Learned-Miller Although an automated reader for the blind first appeared nearly two-hundred years ago, computers can currently “read” document text about as well as a sevenyear-old. Scene text recognition brings many new challenges. A central limitation of current approaches is a feed-forward, bottom-up, pipelined architecture that isolates the many tasks and information involved in reading. The result is a system that commits errors from which it cannot recover and has components that lack access to relevant information. We propose a system for scene text reading that in its design, training, and operation is more integrated. First, we present a simple contextual model for text detection that is ignorant of any recognition. Through the use of special features and data context, this model performs well on the detection task, but limitations remain due to the lack of interpretation. We then introduce a recognition model that integrates several information sources, including font consistency and a lexicon, and compare it to approaches using pipelined architectures with similar information. Next we examine a more unified detection and recognition framework where features are selected based on the joint task of detection and recognition, rather than each task individually. This approach yields better results with fewer features. Finally, we demonstrate a model that incorporates segmentation and recognition at both the character and word levels. Text with difficult layouts and low resolution are more accurately recognized by this integrated approach. By more tightly coupling several aspects of detection and recognition, we hope to establish a new unified way of approaching the problem that vii
Page 1 and 2: UNIFIED DETECTION AND RECOGNITION F
Page 3 and 4: UNIFIED DETECTION AND RECOGNITION F
Page 5: ACKNOWLEDGEMENTS As soon as one beg
Page 9 and 10: CONTENTS Page ACKNOWLEDGEMENTS . .
Page 11 and 12: 4.3.2.1 Model Training . . . . . .
Page 13 and 14: LIST OF TABLES Table Page 1.1 Diffi
Page 15 and 16: 3.11 Visual comparison of local and
Page 17 and 18: CHAPTER 1 INTRODUCTION The first at
Page 19 and 20: Figure 1.2. Images for document pag
Page 21 and 22: T he P h oto S p e cialists Input O
Page 23 and 24: Figure 1.4. Small text in an image
Page 25 and 26: section, we review some of the appr
Page 27 and 28: Kusachi et al. [62] have a multi-re
Page 29 and 30: 1.3.3 Adaptive Recognition Several
Page 31 and 32: ather than integrated systems. In a
Page 33 and 34: Background Faces Allen Andrew Keith
Page 35 and 36: CHAPTER 2 DISCRIMINATIVE MARKOV FIE
Page 37 and 38: If we have a family of sets C = {C
Page 39 and 40: ̂θ ≡ arg max p (θ | D, I) (2.6
Page 41 and 42: functions, i.e., θ = [ θ A θ ] B
Page 43 and 44: P L (θ; α) = −α ‖θ‖ 1 (2.
Page 45 and 46: we drop the dependence of the messa
Page 47 and 48: CHAPTER 3 TEXT AND SIGN DETECTION B
Page 49 and 50: For example, since neighboring regi
Page 51 and 52: Figure 3.3. Decomposition of images
Page 53 and 54: 3.3.1 Feature Overview Rather than
Page 55 and 56: • Raw pixel statistics (e.g., ran
Page 57 and 58:
3.4.2 Experimental Procedure For ou
Page 59 and 60:
Figure 3.7. Example contextual dete
Page 61 and 62:
Table 3.1. Sign detection results w
Page 63 and 64:
Table 3.2. Sign detection results a
Page 65 and 66:
In conclusion, the “default” pr
Page 67 and 68:
CHAPTER 4 UNIFYING INFORMATION FOR
Page 69 and 70:
will outline the details of our mod
Page 71 and 72:
y 1 01 01 01 0 0 01 1 1 0 0 01 1 1
Page 73 and 74:
4.2.2 Language Model Properties of
Page 75 and 76:
5 Basis Functions Learned Function
Page 77 and 78:
node for the factor C, while w B is
Page 79 and 80:
Figure 4.5. Examples of sign evalua
Page 81 and 82:
Lexicon The lexicon we use is deriv
Page 83 and 84:
40 40 Number of Queries 30 20 10 Nu
Page 85 and 86:
and could be modeled directly if we
Page 87 and 88:
Figure 4.9. Examples from the sign
Page 89 and 90:
Model Correct Free checking 31 BOLT
Page 91 and 92:
4.3.4.2 Lexicon Model Here we discu
Page 93 and 94:
CHAPTER 5 UNIFYING DETECTION AND RE
Page 95 and 96:
character and these were directly u
Page 97 and 98:
L C ( θ C ; F, D ) ≡ ∑ k log p
Page 99 and 100:
For the independent method, each ca
Page 101 and 102:
Wavelet Transform Scale 0.3 0.25 0.
Page 103 and 104:
Scale −0.02 0 0.02 0.04 0.06 0.08
Page 105 and 106:
Figure 5.6. Sample images used in e
Page 107 and 108:
1 Avg. Categ. Accuracy 0.95 0.9 0.8
Page 109 and 110:
Relative Improvement 0.5 0.4 0.3 0.
Page 111 and 112:
Recognition Gain 1 0.8 0.6 0.4 0.2
Page 113 and 114:
of further special recognition. Hav
Page 115 and 116:
CHAPTER 6 THE ROBUST READER In the
Page 117 and 118:
6.2 Semi-Markov Model for Segmentat
Page 119 and 120:
6.2.1.2 Character Bigrams As in ear
Page 121 and 122:
exponential of the corresponding Ma
Page 123 and 124:
is signalled, and this string may e
Page 125 and 126:
and k. This joint space over i and
Page 127 and 128:
each location, all the rectified fi
Page 129 and 130:
This procedure is outlined visually
Page 131 and 132:
(e.g., cs, Bk, Kr, Nb, pd, Tl), we
Page 133 and 134:
18.8 18.6 18.4 KL N−Best Ratio Er
Page 135 and 136:
σ Image Output Binarized OmniPage
Page 137 and 138:
F I B E IA A R T C E N T E R (a) F
Page 139 and 140:
6.4.5 End-to-End Demonstration In t
Page 141 and 142:
T he P h oto S p e cialists The Pro
Page 143 and 144:
T he P hoto S p ecialists L L O Y D
Page 145 and 146:
Table 6.2. Comparison of recognitio
Page 147 and 148:
mercial document recognition softwa
Page 149 and 150:
In addition to the models, we have
Page 151 and 152:
[12] Blum, Avrim, and Langley, Pat.
Page 153 and 154:
[37] Geman, Stuart, and Geman, Dona
Page 155 and 156:
[63] Lafferty, John, McCallum, Andr
Page 157 and 158:
[89] Pal, Chris, Sutton, Charles, a
Page 159 and 160:
[114] Torralba, Antonio, Murphy, Ke
Page 161:
[139] Zhou, Yaqian, Wenb, Fuliang,
show all

unified detection and recognition for reading text in scene images

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?