unified detection and recognition for reading text in scene images

ectification, as do Gao et al. [34]. Still, both of these methods rely on hand-coded
rules.
In contrast, Zhang and Chang [137] use triplet cliques on image segments to
flexibly handle the linearity of text in a probabilistic model with learned parameters.
Similarly, Shen and Coughlan [104] classify vertical and horizontal stroke features
with learnable compatibilities in a probabilistic model.
Probabilistic systems for layout analysis are more flexible and require less manual
parameter tuning than heuristic methods. As such, they should be able to directly
handle text that is not strictly horizontal or vertical, if they are trained on such data.
What these two approaches lack is feedback between the feature computation (i.e.,
segmentation [137], stroke detection [104]) and the classification of the features as text
or non-text. Thus, top-down interpretation cannot influence the lower-level features.
To address this issue, we propose joint feature selection for detection and recognition
in Chapter 6 .
1.2.5 Summary
Many of the approaches to text detection and segmentation above may be classified
as texture-based, since they use various features and statistics of intensities and
gradients[53, 132, 24, 67, 135, 113]. Some make use of edge features or morphological
operator outputs [36, 34, 35]. Others combine some of these features [131], e.g., at
different stages in a pipelined classifier [21] or to handle different sizes of text [29].
By contrast, some others methods are region-based [137] or use higher-level features
as input to a classifier [104].
With a few exceptions, most of these systems are divorced from the recognition
process. As such, many do only a rough initial or windowed detection [53, 24, 21, 135],
while others may track text regions across video frames [67, 131]. Some use heuristics
for combining text regions and/or pruning non-text regions [132, 36, 34, 35, 113],
while very few have learned or learnable layout analyses [137, 104].
Prior to recognition, many of the systems binarize text regions [132, 36, 131, 21,
137, 113], while one isolates characters (via binarization) but subsequently classifies
features of the grayscale image [35].
By learning the spatial properties of text to improve detection and bridging the
gap between detection and recognition during training, the methods we present in
this thesis overcome many of the limitations present in earlier approaches to text
detection.
Next we discuss some of the relevant approaches to recognizing text in document
processing and more recent camera- and video-based systems.
1.3 Text Recognition
A great deal of prior knowledge can be brought to bear on the text recognition
problem, from the essentials like the general appearance of characters, to more
complex factors such as language and lexicon awareness. Many of these have been
employed in one form or another, but typically they are not fully integrated. In this
8