Development of an educational robotic platform with dual control interfaces and intelligent computer vision-based interaction

This paper presents the development of a low-cost, AI-enabled educational robotic platform designed to enhance hands-on STEM learning. The system consists of a 4-degree-of-freedom (DOF) robotic arm constructed from 3D-printed components, controlled by an ESP32-based circuit and integrated with both local and Web-based interfaces for flexible operation. Two control interfaces were developed: a local desktop GUI built with PyQt5 and a remote Web-based interface using the MQTT protocol. Both interfaces allow users to manually control robot joint angles, adjust movement step sizes, reset to default positions, and monitor real-time joint states. These interfaces provide intuitive interaction, enabling students to understand motion control in robotics. To integrate artificial intelligence and computer vision, three modules were implemented: face tracking, hand gesture control, and object detection. Face tracking translates facial position and size into 3D coordinates for real-time robot movement using inverse kinematics. Hand gesture recognition uses MediaPipe to interpret finger poses and execute corresponding robot actions. The object detection module employs a YOLOv12 model trained on classroom objects (pens, erasers, markers) to perform autonomous pick-and-place tasks on a simulated conveyor system. Experimental results validate the system's effectiveness in real-time tracking, gesture interpretation, and object manipulation. Real-time plots of joint angles and workspace coordinates illustrate the system's responsive behavior. This integrated platform enables students to explore AI, vision, and robotics in a unified environment. Through interactive exercises, they gain practical experience in programming, AI model development, user interface design, and robotic control, bridging theoretical knowledge and real-world applications.