unified detection and recognition for reading text in scene images

Figure 1.5. Uninformed segmentation can make recognition difficult. Top: Input
image. Left: Probability of foreground under a Gaussian mixture model (k = 3, HSV
color space). Right: Binarization by threshold (p (foreground|k = 3, image) ≥ 1 2 ).
One other closely related work is that of Tu et al. [118]. Here, discriminative
models are used as bottom-up proposal distributions in a large generative model.
Discriminative text detectors are used, and then a generative model must be able to
explain the pixel data as a contour describing a known character. They claim the
advantage of the generative model is that it ensures consistent regions. However,
spatially coupled discriminative models can overcome this [61, 124]. While their
bottom-up detectors seem to do a reasonable job of detecting text to subsequently be
explained, the generative models for characters are independent of one another. In
other words, there is no coupling of information based on where a character should
appear (i.e., next to other characters), nor how it should appear (similar to nearby
characters). The models in this thesis will demonstrate the utility of such information
for interpreting scene text.
We can broadly categorize the systems for robust character recognition along the
following dimensions:
• Binarization [87, 80, 113] versus grayscale feature extraction [23, 62, 52, 118]
• Separate detection/segmentation [80, 113, 23] versus integrated detection/segmentation
and recognition [87, 62, 52, 118]
• Language independent [87, 23, 62, 118] versus language post-processing [80, 113]
versus integrated language models [52].
Although some OCR systems have a fully integrated language model [5, 14, 52], the
systems thus far designed for reading scene text either use no language information
[87, 22, 62] or, at best, post-processing [80, 113]. Moreover, as the most difficult
scene text suffers from the difficulties listed in Table 1.1 on page 6, binarization and
segmentation will become increasingly difficult without regard for interpretation (see
Figure 1.5). The effectiveness of integrating language with simultaneous segmentation
and recognition on grayscale—not binary—images was demonstrated in the camerabased
OCR system of Jacobs et al. [52]. Research on noisy and degraded documents
has recognized this fact for some time. Indeed, this may explain some of the success
of text detection and recognition systems that use commercial OCR systems (e.g.,
[132, 131, 21]).
12