Hybrid camera and LiDAR sensor for payload transfer control on RTGC using the Lyapunov-Kinetic energy approach under varying light levels

This paper presents an advanced approach for automating Rubber-Tired Gantry Crane (RTGC) systems, integrating a hybrid sensor and optimized control methods to enhance container handling in diverse port environments. The proposed hybrid sensor combines LiDAR and camera data to create depth images, utilizing YOLOv11 for accurate container detection while reducing the computation demands typically associated with LiDAR data. This integration provides reliable sensing capabilities even in low-light or variable lighting conditions. Additionally, a Lyapunov-Kinetic Energy-based control system is introduced to optimize container positioning and sway angle reduction, with parameters refined using Particle Swarm Optimization (PSO), Flower Pollination Algorithm (FPA), and Bayesian based optimization, Gaussian Process (GP) and GP with Markov Chain Monte Carlo (GP-MCMC). To address system nonlinearities, particularly deadzone effects, a Type-2 Fuzzy Precompensator is employed, enhancing control stability and accuracy. Experimental results on a laboratory-scale RTGC prototype demonstrate that the combination of hybrid sensing, optimized Lyapunov control, and deadzone mitigation delivers strong automation performance under varying lighting conditions. The best optimization result was achieved using Gaussian Process optimization, with a mean squared error (MSE) of 0.007 for the steady-state position and 0.005 for the sway angle. Additional tests were conducted under variations in setpoint, rope length, and payload mass. The results show robust performance under these uncertainties, with MSE values for each state remaining close to those previously reported.