Constrained Actor-Critic Reinforcement Learning for Deformable Object Visual Shape Servoing

A constrained adaptive actor-critic (AAC)-based policy iteration controller for vision-based shape servoing (VSS) of a deformable object is presented in this article. A finite point-based representation of the deformable object is used, and the deformation Jacobian matrix is approximated using Fourier-series basis functions. An integral concurrent learning (ICL)-based parameter update law is designed to update the deformation model parameters. The high-dimensional state representation of the object requires a large number of parameters to learn the deformation model, value function, and the optimal policy. To address this, a principal component analysis (PCA)-based dimensionality reduction method is used to compute a low-dimensional representation of the object, which is then used to design the constrained AAC controller. A barrier transform (BT) method is used to transform part of the state into a constrained state, thereby ensuring that the physical constraints are satisfied. The proposed controller is extensively tested in simulations across several desired configurations, and through Monte Carlo runs on an ABB IRB120 robot platform.