Hybrid Visual Predictive Control for Single and Multi-Agent Systems

Visual servoing (VS) is a technique which utilizes information acquired from a visual sensor to control the motion of a robot. Despite the advantages of the classic VS techniques in controlling of the robots- specifically in unstructured environments- they suffer from some system’s shortcomings such as the lack of an effective mechanism for handling the existing constraints within the control procedure. To alleviate these issues, among the several approaches proposed in the literature, the receding horizon or model predictive control (MPC) technique is an appropriate candidate to be employed by the system. Any combination of the model predictive control and visual servoing is called visual predictive control (VPC) which is categorized as the classic and hybrid methods. The classic VPC schemes are developed based on the classic VS approaches and the hybrid ones employ the hybrid VS schemes to be integrated by MPC. On the other hand, to guarantee the convergence of the system in presence of the noises and uncertainties, the proposed VPC schemes are embedded in internal model control (IMC) framework. This work contributes by utilizing the available VPC scheme in control of different robots, such as 3 DOF catheter as a continuum robot and a 6 DOF manipulator from Denso robotics as a robot arm, and by developing two hybrid predictive controllers, conjugated VPC and depth-based VPC, with higher constraint handling capabilities. By utilizing the proposed VPC methods, some of the available limitations could be resolved in the single-robot control level. In addition, to enable the system to perform more sophisticated tasks under difficult conditions such as unexpected occlusions, a multi-agent system controlled by the mentioned VPC methods is employed. Under these conditions, by utilizing the data sharing ability among neighboring agents inside the network, the obstructed one could be provided with enough information to complete the task. At each stage, the performance of the proposed algorithm is verified through numerous simulations and experiments to show their capability in handling existing constraints while accounting for system uncertainties.