PCK-RL: A Unified Framework for Vision-Language Model and Reinforcement Learning-Based Robotic Manipulation

Recent advances in the vision-language model (VLM) have demonstrated impressive capabilities in aligning high-level semantic information with visual environments, offering significant help for task planning in robotic manipulation. However, existing methods often fail to convert semantic tasks into precise robotic actions, especially in complex 3-D environments where keypoint localization and interaction dynamics are critical. To address this challenge, this article proposes principal component keypoint-reinforcement learning (PCK-RL), a multistage framework that bridges semantic reasoning and low-level control through semantic subplan decomposition, object keypoint localization, and example trajectory generation. Principal component analysis (PCA) is introduced to normalize the inputs and outputs of the VLM, constructing novel intermediate representations to enhance model performance. Additionally, a hybrid robotic action alignment method rapidly exploring random tree star-reinforcement learning (RRT*-RL) is introduced. It refines zero-shot reference trajectories by incorporating reinforcement learning (RL) with a closed-loop environmental feedback mechanism, which is crucial for long-horizon tasks. This article validates PCK-RL through extensive experiments in both simulated and real-world robotic environments, demonstrating its effectiveness and robustness in executing long-horizon tasks and manipulating diverse objects.