Revisiting direct methods for monocular model-based 3D object tracking: A hierarchical optimization

Monocular model-based 3D object tracking has steadily progressed and has continued to draw interest across diverse applications. In particular, region-based methods have recently become the predominant approach, consistently demonstrating state-of-the-art performance. In this work, we explore a new direction in monocular model-based 3D object tracking by revisiting direct methods and introducing a novel energy function and optimization strategy that achieve significant performance improvements. Specifically, the energy function is formulated as a multi-constraint cost function that effectively combines photometric and geometric constraints in a complementary manner. The optimization strategy is designed as a hierarchical framework with stage-tailored energy functions and is further enhanced by an efficient second-order approximation of the photometric cost function. To validate our method, we conduct comprehensive performance evaluations and analyses, including detailed comparative and ablation studies. Experimental results demonstrate that our method is highly competitive with current top-performing methods on challenging objects and scenes.