Human Detection in Video over Large Viewpoint Changes

More documents

Recommendations

Info

1248 G. Duan, H. Ai, and S. Lao Fig. 1: Human detection in video over large viewpoint changes. Samples of three typical viewpoints and corresponding scenes are given. a large variation in human appearance and motion over wide viewpoint changes? 3) How to deal with the changes in video frame rate or abrupt motion if using motion features on several consecutive frames? Viola and Jones [1] first made use of appearance and motion information in object detection, where they trained AdaBoosted classifiers with Harr features on two consecutive frames. Later Jones and Snow [2] extended this work by proposing appearance filter, difference filter and shifted difference filter on 10 consecutive frames and using predefined several categories of samples. The approaches in [1] [2] can solve the 1 st problem, but still face the challenge of the 3 rd problem. The approach in [2] can handle the 2 nd problem to some extent but as even human himself sometimes cannot tell which predefined category a moving object belongs to and thus its application will be limited, while the approach in [1] trains detectors by mixing all positives together. Dalal et al. [3] combined HOG descriptors and some motionbased descriptors together to detect humans with possibly moving cameras and backgrounds. Wojek et al. [4] proposed to combine multiple and complementary feature types and incorporate motion information for human detection, which coped with moving camera and clustered background well and achieved promising results on humans with a common frontal viewpoint. In this paper, our aim is to design a novel feature to take advantages of both appearance and motion information, and to propose an efficient learning algorithm to learn a practical detector of rational structure even when the samples are tremendously diverse for handling the difficulties mentioned above in one framework. The rest of this paper is organized as follows. Related work is introduced in Sec. 2. The proposed feature (I 2 CF ), the co-cluster algorithm (EMC-Boost) and the sampling strategy (MVS) are given in Sec. 3, Sec. 4 and Sec. 5 respectively and they are integrated to handle human detection in video in Sec. 6. Some experiments and conclusions are given in Sec. 7 and the last section respectively. 2 Related work In literature, human detection in video can be divided roughly into four categories. 1) Detection in static images as [5] [6] [7]. APCF [5], HOG [6] and Edgelet [7] are defined on appearance only. APCF compares colors or gradient orientations of two squares in images that can describe the invariance of color
Human Detection in Video over Large Viewpoint Changes 1249 and gradient of an object to some extent. HOG computes oriented gradient distribution in a rectangle image window. An edgelet is a short segment of line or curve, which is predefined based on prior knowledge. 2) Detection over videos as [1] [2]. Both of them are already mentioned in the previous section. 3) Object tracking as [8] [9]. Some need manual initializations as in [8], and some are with the aid of detection as in [9]. 4) Detecting events or human behaviors. 3D volumetric features [10] are designed for event detection, which can be 3D harr like features. ST-patch [11] is used for detecting behaviors. Inspired by those works, we propose Intra-frame and Inter-frame Comparison Features (I 2 CF s) to combine appearance and motion information. Due to the large variation in human appearance over wide viewpoint changes, it is impossible to train a usable detector by taking the sample space as a whole. The solution is, divide and conquer, to cluster the sample space into some subspaces during training. A subspace can be dealt with as one class and the difficulty exists mainly in clustering the sample space. An efficient way is to cluster sample space automatically like in [12] [13] [14]. Clustered Boosting Tree (CBT) [14] splits sample space automatically by the already learned discriminative features during training process for pedestrian detection. Mixture of Experts [12] (MoE) jointly learns multiple classifiers and data partitions. It emphasizes much on local experts and is suitable when input data can be naturally divided into homogeneous subsets, which is even impossible for a fixed viewpoint of human as shown in Fig. 1. MC-Boost [13] co-clusters images and visual features by simultaneously learning image clusters and boosting classifiers. Risk map, defined on pixel level distance between samples, is also used to reduce the searching space of the weak classifiers in [13]. To solve our problem, we propose an Enhanced Multiple Clusters Boost (EMC-Boost) algorithm to co-cluster the sample space and discriminative features automatically which combines the benefits of Cascade [15], CBT [14] and MC-Boost [13]. The selection of EMC-Boost instead of MC-Boost is discussed in Sec. 7. Our contributions are summarized in four folds: 1) Intra-frame and Interframe Comparison Features (I 2 CF s) are proposed to combine appearance and motion information for human detection in video over large viewpoint changes; 2) An Enhanced Multiple Clusters Boost (EMC-Boost) algorithm is proposed to co-cluster the sample space and discriminative features automatically; 3) A Multiple Video Sampling (MVS) strategy is used to make our approach robust to human motion and video frame rate changes; 4) A two-stage tree structure detector is presented to fully mine the discriminative features of the appearance and motion information. The experiments in challenging real-world scenes show that our approach is robust to human motion and frame rate changes. 3 Intra-frame and Inter-frame Comparison Features 3.1 Granular space Our proposed discriminative feature is defined in Granular space [16]. A granule is a square window patch defined in grey images, which is defined as a triplet
Page 1: Human Detection in Video over Large
Page 5 and 6: Human Detection in Video over Large
Page 11 and 12: Reca l Reca l Reca l Reca l Reca l

Human Detection in Video over Large Viewpoint Changes

Create successful ePaper yourself

Delete template?

Save as template?