Picking up an indicated object in a complex environment

This paper presents a grasping system for picking up an indicated object in a complex real-world environment using a parallel jaw gripper. The proposed grasping scheme comprises the following three main steps: (1) A user indicates a target object and provides the system with a task instruction on how to grasp it, (2) the system acquires geometric information about the target object and constructs a 3D environment model around the target by stereo vision using the information obtained from the task instruction, and (3) the system finds a grasp point based on grasp evaluation using the acquired information. As an example of the scheme, we examined the picking up of a cylindrical object by grasping at the brim. An important and advantageous feature of this scheme is that the user can easily instruct the robot on how to perform the object-picking task through simple clicking operations, and the robot can execute the task without exact models of the target object and the environment being available in advance.