PedX: Benchmark Dataset for Metric 3-D Pose Estimation of Pedestrians in Complex Urban Intersections

This letter presents a novel dataset titled PedX, a large-scale multimodal collection of pedestrians at complex urban intersections. PedX consists of more than 5 000 pairs of high-resolution (12MP) stereo images and LiDAR data along with providing two-dimensional (2-D) image labels and 3-D labels of pedestrians in a global coordinate frame. Data were captured at three four-way stop intersections with heavy pedestrian-vehicle interaction. We also present a 3-D model fitting algorithm for automatic labeling harnessing constraints across different modalities and novel shape and temporal priors. All annotated 3-D pedestrians are localized into the real-world metric space, and the generated 3-D models are validated using a motion capture system configured in a controlled outdoor environment to simulate pedestrians in urban intersections. We also show that the manual 2-D image labels can be replaced by state-of-the-art automated labeling approaches, thereby facilitating automatic generation of large scale datasets.