What question did this study set out to answer?

The research aims to enhance the performance of PTZNet for real-world applications by addressing data scarcity issues.

January 22, 20260 citationsOpen Access

Optimizing PTZNet for Real-World Deployment - Batch-Level Mixing to Compensate for Data Scarcity in Pose and Overlap Regression

Key Points

The research aims to enhance the performance of PTZNet for real-world applications by addressing data scarcity issues.
Developed and evaluated a Siamese deep learning model, PTZNet, for joint PTZ rotation and FoV overlap regression.
Investigated Batch-Level Mixing (BLM) to integrate synthetic and real-world images during training.
Measured performance using 3D rotation error and spherical Intersection over Union (IoU) for overlap.
Batch-Level Mixing significantly reduced errors in predicting rotation and overlap compared to training on either synthetic or real-world data alone.
The model maintained improved performance across varying amounts of real data available.
Synthetic images effectively compensated for the lack of real-world data, enhancing predictive accuracy.

Abstract

In urban planning and traffic management, Pan, Tilt and Zoom (PTZ) cameras monitor streets, intersections, and public areas. Precisely estimating how a single PTZ camera moves between images, and how much their Fields of View (FoVs) overlap, is challenging due to scene variability, lens distortion, dynamic motion, and limited or unavailable camera parameters. Collecting large, diverse real-world datasets with precise metadata is costly and logistically complex. This thesis investigates whether our Siamese deep learning model, PTZNet, can infer relative PTZ rotations and FoV overlap with competitive performance caused by camera movement between image pairs, while relying primarily on virtually created synthetic images and mitigating the Synthetic-To-Real (S2R) domain gap, which is the discrepancy in feature distribution between real-world and synthetic data. PTZNet jointly regresses camera rotation and FoV overlap. Performance is measured with a 3D rotation error, the spherical Intersection over Union (IoU) for overlap, and a combined relative error capturing joint accuracy. A key element is the Batch-Level Mixing (BLM) method, which strategically integrates synthetic and limited real-world images within training batches to improve generalization to real-world scenes. Our experiments show that mixing synthetic images with a small amount of real-world data yields substantially lower errors in predicting rotation and overlap than training on either domain alone. This improvement persists across varying real-data amounts: When real-world image collection is costly, training primarily on synthetic images and tuning with a small targeted set of real-world samples effectively compensates for limited real-world data and improves inference. The man contributions of this work include (1) an analysis of the S2R domain gap for virtually created PTZ data and its effect on inference, (2) the development of the Batch-Level Mixing (BLM) data training strategy that outperforms Dataset-Level Mixing (DLM) in robustness to domain shift, (3) quantitative evidence that synthetic images improve relative PTZ rotation and overlap prediction accuracy when real-world images are scarce, and (4) the introduction of PTZNet, a Siamese architecture for joint PTZ rotation and FoV-overlap regression trained on mix of synthetic and real-world data.

问 AI

Bookmark

View Full Paper