Axiomatic particle filtering for goal-directed robotic manipulation

Manipulation tasks involving sequential pick-and-place actions in human environments remains an open problem for robotics. Central to this problem is the inability for robots to perceive in cluttered environments, where objects are physically touching, stacked, or occluded from the view. Such physical interactions currently prevent robots from distinguishing individual objects such that goal-directed reasoning over sequences of pick-and-place actions can be performed. Addressing this problem, we introduce the Axiomatic Particle Filter (APF) as a method for axiomatic state estimation to simultaneously perceive objects in clutter and perform sequential reasoning for manipulation. The APF estimates state as a scene graph, consisting of symbolic spatial relations between objects in the robot's world. Assuming known object geometries, the APF is able to infer a distribution over possible scene graphs of the robot's world and produce the maximally likely state estimate of each object's pose and spatial relationships between objects. We present experimental results using the APF to infer scene graphs from depth images of scenes with objects that are touching, stacked, and occluded.