Vision based steering control using convolutional neural networks with latency analysis and microcontroller based actuation

This work presents a real-time vision-based steering control framework in which CNN-based steering inference is performed on a host PC and the resulting commands are transmitted to an Arduino UNO for microcontroller-based servo actuation. Using the SullyChen driving dataset for training, the proposed model predicts continuous steering angles from monocular images and transmits the corresponding commands to a servo motor via serial communication. The system achieves a validation mean absolute error (MAE) of 0.093 and a root mean square error (RMSE) of 0.131, outperforming a shallow CNN baseline while providing a favourable trade-off between predictive accuracy and real-time inference speed when compared with the NVIDIA PilotNet architecture. The implemented pipeline has an estimated cumulative processing and actuation latency budget of approximately 150 ms per frame (6–7 frames per second (FPS)), supporting the intended low-speed steering proof-of-concept. While the current frame-wise approach may exhibit minor fluctuations during rapid directional transitions, the system provides a practical foundation for further development of affordable vision-based steering systems, with future improvements directed toward temporal modelling and fully embedded edge deployment. External video evaluations were used primarily to assess visual-domain generalization, as ground-truth steering annotations were unavailable.