unified detection and recognition for reading text in scene images

Figure 1.1. Figure from the patent for the first practical OCR system [105].
font consistency, linguistic properties, and a lexicon (Chapter 4). This unified reading
model outperforms approaches using similar information in the more traditional
pipelined architecture.
Detection and recognition are not processes that happen in isolation, but that instead
require feedback and communication for optimal performance. Toward this end,
we introduce a template-based local feature model that can be utilized for detection
or recognition (Chapter 5). We achieve better results with fewer features by selecting
features and training the model on the joint task of detection and recognition,
rather than for each sub-task independently. Finally, we unify character and word
segmentation with recognition, integrating these processes in a single model that can
evaluate and compare interpretations with maximal awareness (Chapter 6).
Next we describe the problem of scene text reading and contrast it with the probem
of document processing. In the remainder of this chapter we review some of the broad
literature relevant to the tasks of document processing, text detection, and character
recognition. Chapter 2 will then review the common, underlying framework for the
probability models used in the rest of the thesis.
1.1 Scene Text
There is a long history of research and development of automatic readers. The first
practical system was a patent filed in 1951 by David H. Shepard [105]. This device
(see Figure 1.1) converted typewritten documents into punch cards in a magazine
subscription department. Of course, hardware devices and software techniques have
improved a great deal in the fifty intervening years.
Several differences exist between reading text in documents and in scene images.
A few examples are shown in Figure 1.2. The first primary difference is in the problem
of locating the text to be recognized. In a standard one or two column document,
almost no detection must be done. Lines of text are easy to identify and simple
heuristics are usually sufficient to prepare the input for recognition. In more complex
documents such as newspapers and magazines, there is an added difficulty of distin-
2