Camera- and IMU-based pose tracking for augmented reality

In this paper, we propose an algorithm for tracking mobile devices (such as smartphones, tablets, or smartglasses) in a known environment for augmented reality applications. For this purpose, we interpret the environment as an extended object with a known shape, and design likelihoods for different types of image features, using association models from extended object tracking. Based on these likelihoods, and together with sensor information of the inertial measurement unit of the mobile device, we design a recursive Bayesian tracking algorithm. We present results of our first prototype and discuss the lessons we learned from its implementation. In particular, we set up a “pick-by-vision” scenario, where the location of objects in a shelf is to be highlighted in a camera image. Our experiments confirm that the proposed tracking approach achieves accurate and robust tracking results even in scenarios with fast motion.