Human Detection in Video over Large Viewpoint Changes

More documents

Recommendations

Info

1256 G. Duan, H. Ai, and S. Lao and a ground-truth box is larger than 50% of their union, we consider it to be a successful detection. Only one detection per annotation is counted as correct. For simplicity, the three typical viewpoints mentioned in Fig. 1 are represented as Horizontal Viewpoint (HV), Slant Viewpoint (SV) and Vertical Viewpoint (VV) in turn from left to right. Datasets. The datasets used in the experiments are ETHZ dataset [20], PETS2007 dataset [21] and our own collected dataset. ETHZ dataset provides four video sequences, Seq.#0∼Seq.#3 (640×480 pixels at 15 fps). This dataset whose viewpoint is near HV is recorded using a pair of cameras, and we only use the images provided by the left camera. PETS2007 dataset contains 9 sequences S00∼S08 (720×576 pixels at 30 fps) and each sequence has 4 fixed cameras and we choose the 3 rd camera whose viewpoint is near SV. There are 3 scenarios in PETS2007, with increasing scene complexity, loitering (S01 and S02), attended luggage removal (S03, S04, S05 and S06) and unattended luggage (S07 and S08). In the experiments, we use S01, S03, S05 and S06. In addition, we have collected several sequences by hand-held DV cameras: 2 sequences near HV (853×480 pixels at 30 fps), and 2 sequences near SV (1280×720 pixels at 30 fps) and 8 sequences near VV (2 sequences are 1280×720 pixels at 30 fps and the others are 704×576 pixels at 30 fps). Training and testing datasets. S01, S03, S05 and S06 of PETS2007 and our own dataset are labeled every five frames manually for training and testing, while ETHZ dataset provides the groundtruth already. The training datasets contain Seq.#0 of ETHZ, S01, S03 and S06 of PETS2007, 2 sequences (near HV), 2 sequences (near SV) and 6 sequences (near VV) of ours. The testing datasets contain Seq.#1, Seq.#2 and Seq.#3 of ETHZ, S05 of PETS2007, and 2 sequences of ours (near VV). Note that the groundtruths of the internal unlabeled frames in the testing datasets are achieved through interpolation. The properties of the testing datasets may have impacts on all detectors, like camera motion states (fixed or moving), illumination conditions (slightly or significantly light changes), etc. Details about the testing datasets are summarized in Table 3. Training detectors. We have labeled 11768 different humans in total and obtain 47072 positives after MVS, where the number of positives near HV, SV Table 3: Some details about the testing datasets. Description Seq. 1 Seq. 2 S05 Seq.#1 Seq.#2 Seq.#3 Source ours ours PETS2007 ETHZ ETHZ ETHZ Camera Fixed Fixed Fixed Moving Moving Moving Light changes Slightly Slightly Slightly Slightly Slightly Significantly Frame rate 30fps 30fps 30fps 15fps 15fps 15fps Size 704×576 704×576 720×576 640×480 640×480 640×480 Frames 2420 1781 4500 999 450 354 Annotations 591 1927 17067 5193 2359 1828
Reca l Reca l Reca l Reca l Reca l Reca l Human Detection in Video over Large Viewpoint Changes 1257 0.95 Test Sequence 1 (ours, 2420 frames, 591 annotations) 0.9 Test Sequence 2 (ours, 1718 frames, 1927 annotations) 0.9 0.85 0.85 0.8 0.8 0.75 0.75 0.7 0.05 0.1 0.15 0.2 0.25 0.3 FPPI 0.75 Intra-frame CF + Boost Intra-frame CF + EMC-Boost I 2 CF + EMC-Boost S05 (PETS2007, 4500 frames, 17067 annotations) 0.7 Intra-frame CF + Boost Intra-frame CF + EMC-Boost I 2 CF + EMC-Boost 0.65 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 FPPI 0.8 Seq. #1 (ETHZ, 999 frames, 5193 annotations) 0.7 0.65 0.75 0.7 0.65 0.6 0.6 0.55 0.55 0.5 0.5 0.45 Ess et al. Schwartz et al. 0.45 Intra-frame CF + Boost Intra-frame CF + EMC-Boost I 2 CF + EMC-Boost 0.4 0.35 Wojek et al.(HOG, IMHwd and HIKSVM) Intra-frame CF + Boost Intra-frame CF + EMC-Boost I 2 CF + EMC-Boost 0.4 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 FPPI 0.75 0.7 0.65 0.6 Seq. #2 (ETHZ, 450 frames, 2359 annotations) 0 1 2 3 4 5 6 FPPI Seq. #3 (ETHZ, 354 frames, 1828 annotations) 0.9 0.8 0.7 0.55 0.6 0.5 0.45 Ess et al. Schwartz et al. 0.4 Wojek et al.(HOG, IMHwd and HIKSVM) Intra-frame CF + Boost 0.35 Intra-frame CF + EMC-Boost I 2 CF + EMC-Boost 0 0.5 1 1.5 2 2.5 3 3.5 4 FPPI 0.5 0.4 0.3 Ess et al. Schwartz et al. Wojek et al.(HOG, IMHwd and HIKSVM) Intra-frame CF + Boost Intra-frame CF + EMC-Boost I 2 CF + EMC-Boost 0.2 0 0.5 1 1.5 2 2.5 3 3.5 4 FPPI Fig. 6: Evaluation of our approach and some results. and VV are 18976, 19248 and 8848 respectively. The size of positives is normalized to 58 × 58. Some positives are shown in Fig. 4. We train a detector based on EMC-Boost selecting I 2 CF as features. Implementation details. We cluster the sample space into 2 clusters in the 1 st stage and cluster the two sub spaces into 2 and 3 clusters separately in the 2 nd stage as illustrated in Fig. 5 (b). When do we start and stop MC/SC? When the false positive rate is less than 10 −2 (10 −4 ) in the 1 st (2 nd ) stage, we start MC and then start SC after learning by MC. Before describing when to stop MC or SC, we first define transferred samples. A sample is called transferred if it belongs to another cluster after current round boosting. We stop MC (SC) when the number of transferred samples is less than 10% (2%) of the total number of samples. Evaluation. To compare with our approach (denoted as I 2 CF +EMC-Boost), two other detectors are trained: one is to adopt Intra-frame CF learned by a general Boost algorithm like [5] [15] (denoted as Intra-frame CF+Boost) and the other one is to adopt Intra-frame CF learned by EMC-Boost (denoted as Intraframe CF+EMC-Boost). Note that due to the large amount of Inter-frame CFs, the large amount of positives and memory limited, it is impractical to learn a detector of Inter-frame CF by Boost or EMC-Boost. We compare our approach with Intra-frame CF + Boost and Intra-frame CF + EMC-Boost approaches on PETS2007 dataset and our own collected videos, and also with [4] [20] [22] on ETHZ dataset. We give the ROC curves and some results in Fig. 6. In general, our proposed approach which integrates appearance and motion information is superior to Intra-frame CF+Boost and Intra-frame CF+EMC-Boost approaches which only use appearance information. From another viewpoint, this experiment also indicates that incorporating motion information improves detection significantly as [4].
Page 1 and 2: Human Detection in Video over Large
Page 9: Human Detection in Video over Large

Human Detection in Video over Large Viewpoint Changes

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?