Physics-based hand interaction with virtual objects

We propose an intuitive user interaction system with virtual objects by directly using user's hands through a physics simulation. This system aims to provide a more direct way to manipulate and interact with virtual objects and to induce more realistic reactions of the virtual objects in the interaction than the existing methods. The key contribution of this paper is to model the deformations of the user's hands in two ways: global deformation by the hand poses and the local deformation by contacts with objects. By the two deformation models, objects can be manipulated with better stability and robustness without any predefinition of interaction methods depending on the object shapes, by solely using a physics simulation. The experiment shows that the proposed method enables a user to intuitively manipulate virtual object accurately.