EP-DDPG: A deep reinforcement learning framework for visual-tactile fusion in grasping control of humanoid robotic hand

Dexterous grasping requires multi-modal sensory integration, yet combining visual and tactile perception for five-fingered humanoid hands in unstructured environments remains challenging. This study proposes a deep reinforcement learning (DRL) framework that integrates visual and tactile perceptions for five-digit humanoid robotic grasping. The novel framework, EP-DDPG, is based on a deep deterministic policy gradient (DDPG) algorithm and incorporates entropy regularization applied to both the reward function and the policy update, as well as a prioritized experience replay mechanism (EP). This novel framework is supposed to leverage the synergy of visual and tactile perceptions to enhance autonomous learning and grasping control. Results confirmed that in simulation, the novel EP-DDPG framework achieved grasping success rates up to 94.5% (training) and 89.4% (testing), respectively, exceeding the other popular DRL methods such as the DDPG, soft actor-critic (SAC), proximal policy optimization (PPO), and twin delayed deep deterministic policy gradient (TD3), by an average margin of approximately 27.3% and 28.4%, respectively. Additionally, the EP-DDPG demonstrated superior generalization over the DDPG, SAC, PPO and TD3 methods during the testing process in simulation. The real-world experiments showed that humanoid robotic grasping with the EP-DDPG framework achieved success rates of 93.9% on the training set and 84.8% on the testing set. By leveraging complementary visual and tactile inputs, the proposed EP-DDPG achieved higher grasping success rates and demonstrated improved performance on previously unseen objects compared with the baseline DRL methods. This novel framework may promote the sensorimotor integration for humanoid robotic hand and contribute to diverse grasping tasks towards unstructured scenarios.