Robotic Assembly of Interlocking Blocks for Construction Based on Large Language Models

Using robots to automate construction processes can enhance productivity and safety and reduce cost. However, the construction robots haven't been widely accepted primarily due to two challenges: lack of data and computational resources to train robots for performing specific construction tasks, and lack of the capability for robots to deal with new tasks. This paper introduces a framework leveraging large language models (LLMs) to enable robots to perform assembly tasks based on in situ worker instructions without extensive prior robot training. By using predefined prompts tailored to specific tasks, the LLM can interpret worker instructions and generate corresponding action sequences and robot functions without additional training. Hence, this LLM-based approach can effectively reduce training costs and facilitate robot adaptability for new tasks and environments through simple prompt modification. To validate this framework, several interlocking block assembly tasks performed by LLM-based robots were evaluated. It is found simple instructions were accurately executed by the robots, but future research is needed to improve the model to handle complex instructions from humans. Overall, this study demonstrates the potential of LLM-based robots in performing assembly tasks in construction with human collaboration.