unified detection and recognition for reading text in scene images

with the detection stage altogether. One important facet being overlooked by all
of these methods is the opportunity to improve testing performance by unifying the
training of detection and recognition stages.
Later in the thesis we will examine how to tie detection and recognition together
during learning as well as the operation of the system. This will rely on sharing
intermediate levels of the classifier—i.e., features. In the remainder of this section,
we review some related feature selection strategies as they apply to computer vision
tasks and some recent work related to sharing features for different tasks.
1.4.1 Feature Selection
Several general frameworks exist for selecting features. The two most basic are
greedy forward and backward schemes. Forward schemes incrementally add features
to a model based on some criterion of feature utility. Examples of this include work
by Viola and Jones [121], who use single-feature linear classifiers as weak learners in a
boosting framework, adding features with the lowest weighted error to the ensemble.
A similar forward method by Berger et al. [9] involves adding only those candidates
that most increase the likelihood of a probabilistic model. Backward schemes, by
contrast, selectively prune features from a model. The l 1 or Laplacian prior [128]
for neural networks, maximum entropy models, logistic regression, etc. belongs to
this category. In this scheme, features are effectively eliminated from a model during
training by fixing their corresponding weights to zero. Many other variants for selecting
a subset of features are possible; see Blum and Langley [12] for a more thorough
review.
Feature types and selection strategies for visual tasks have varied widely. The
Viola and Jones object detector [121] employs outputs of simple image difference
features, which are similar to wavelets. There are many possible filters and only a
few are discriminative, so a selection process is required primarily for computational
efficiency. Other methods use image fragments or patches as feature descriptors.
These patches may be taken directly from the image [119, 1] or an intermediate
wavelet-based representation [103, 83].
1.4.2 Feature Sharing
One of the most prominent examples of shared features in the general machine
learning literature is Caruana’s work on multitask learning [19]. The central idea is
to use a shared representation while learning tasks in parallel.
In particular, the high-dimensional patch-based image features mentioned above
are often densely sampled and vector quantized to create a discrete codebook representation.
Winn et al. [130] iteratively merge code words that do not contribute to
discrimination overall, while Jurie et al. [56] create a clustering algorithm designed to
cover the space with code words for better discrimination. Bernstein and Amit [10]
cluster patches in a generative model for characters. Alternatively, LeCun et al. [65]
learn (rather than select for) a discriminative intermediate feature representation.
However, all of these methods are focused on one type of recognition, either catego-
16