Visual Language Model for Autonomous Site Survey

The energy sector's reliance on manual, in-person inspections of machinery is increasingly challenged by limitations in personnel availability, travel restrictions, and safety concerns. This paper presents the development and evaluation of a Visual Language Model (VLM) integrated into a mobile application to automate weekly visual inspections of machinery in client energy plants. The primary objective is to enhance the efficiency and accuracy of inspection reports while reducing the workload on advisory teams. The scope of the work includes extracting and preprocessing images, textual descriptions, technical arrangement data from past reports. These data are used to train a robust VLM capable of analyzing both visual and textual information. The VLM is then integrated into a mobile application that operates autonomously, providing real-time inspection reports. Initial testing in controlled environments and gradual deployment to real-world scenarios have shown promising results, with the model successfully analyzing images to generate accurate inspection reports. The mobile application facilitates seamless data collection and sharing, enabling remote experts to review and validate the results promptly. Observations indicate that the autonomous inspection process significantly reduces the time and resources required for site assessments. The conclusions drawn suggest that the VLM can effectively offload routine inspection tasks, allowing advisory teams to focus on more complex issues. This approach has the potential to increase the volume of inspections conducted, enhance the overall efficiency of advisory services, and ultimately benefit both internal teams and client operations. The novel use of a VLM in this context addresses the limitations of traditional inspection methods, making the process faster, more efficient, and less dependent on personnel availability and travel restrictions.