Human Detection in Video over Large Viewpoint Changes

1256 G. Duan, H. Ai, and S. Lao
and a ground-truth box is larger than 50% of their union, we consider it to be
a successful detection. Only one detection per annotation is counted as correct.
For simplicity, the three typical viewpoints mentioned in Fig. 1 are represented
as Horizontal Viewpoint (HV), Slant Viewpoint (SV) and Vertical Viewpoint
(VV) in turn from left to right.
Datasets. The datasets used in the experiments are ETHZ dataset [20],
PETS2007 dataset [21] and our own collected dataset. ETHZ dataset provides
four video sequences, Seq.#0∼Seq.#3 (640×480 pixels at 15 fps). This dataset
whose viewpoint is near HV is recorded using a pair of cameras, and we only use
the images provided by the left camera. PETS2007 dataset contains 9 sequences
S00∼S08 (720×576 pixels at 30 fps) and each sequence has 4 fixed cameras and
we choose the 3 rd camera whose viewpoint is near SV. There are 3 scenarios in
PETS2007, with increasing scene complexity, loitering (S01 and S02), attended
luggage removal (S03, S04, S05 and S06) and unattended luggage (S07 and S08).
In the experiments, we use S01, S03, S05 and S06. In addition, we have collected
several sequences by hand-held DV cameras: 2 sequences near HV (853×480
pixels at 30 fps), and 2 sequences near SV (1280×720 pixels at 30 fps) and 8
sequences near VV (2 sequences are 1280×720 pixels at 30 fps and the others
are 704×576 pixels at 30 fps).
Training and testing datasets. S01, S03, S05 and S06 of PETS2007 and
our own dataset are labeled every five frames manually for training and testing,
while ETHZ dataset provides the groundtruth already. The training datasets
contain Seq.#0 of ETHZ, S01, S03 and S06 of PETS2007, 2 sequences (near HV),
2 sequences (near SV) and 6 sequences (near VV) of ours. The testing datasets
contain Seq.#1, Seq.#2 and Seq.#3 of ETHZ, S05 of PETS2007, and 2 sequences
of ours (near VV). Note that the groundtruths of the internal unlabeled frames
in the testing datasets are achieved through interpolation. The properties of the
testing datasets may have impacts on all detectors, like camera motion states
(fixed or moving), illumination conditions (slightly or significantly light changes),
etc. Details about the testing datasets are summarized in Table 3.
Training detectors. We have labeled 11768 different humans in total and
obtain 47072 positives after MVS, where the number of positives near HV, SV
Table 3: Some details about the testing datasets.
Description Seq. 1 Seq. 2 S05 Seq.#1 Seq.#2 Seq.#3
Source ours ours PETS2007 ETHZ ETHZ ETHZ
Camera Fixed Fixed Fixed Moving Moving Moving
Light changes Slightly Slightly Slightly Slightly Slightly Significantly
Frame rate 30fps 30fps 30fps 15fps 15fps 15fps
Size 704×576 704×576 720×576 640×480 640×480 640×480
Frames 2420 1781 4500 999 450 354
Annotations 591 1927 17067 5193 2359 1828