Robust RGBD Tracking via Weighted Convolution Operators

Discriminative Correlation Filter (DCF) based trackers achieve superior performance with continuous conceptual improvement in the tracking field. By now, traditional DCF based trackers suppose that the tracked target is rigid and could be represented by an axis-aligned rectangular box well. Those trackers suffer from object deformation, irregular object shape and partial occlusion where the target bounding box is filled with both target and background pixels. Recently, the depth information captured by the depth sensors offer complementary information to RGB data. Generally, depth information highlights the foreground target from the background, which mitigates the backgournd effect in the bounding box. In this paper, we propose to learn Weighted Convolution Operators (WCO) for robust RGBD tracking. First, WCO integrate deep features extracted from the RGB channels and hand-craft features extracted from the depth channel to enhance target representation. Second, a weight map is jointly derived from the depth and color information to highlight the foreground area. Each value on the weight map demonstrates the possibility of this pixel pertaining to the foreground area. Last, WCO is optimized with the Preconditioned Congugate Gradient (PCG) Method during correlation filter training. Our proposed WCO tracker achieves the top performance on the Princetion Tracking Benchmark (PTB), which demonstrates the validity of our RGBD tracking framework.