Rapid adaptation in photovoltaic defect detection: Integrating CLIP with YOLOv8n for efficient learning

Detecting faults in photovoltaic (PV) systems is crucial for maintaining the effectiveness and dependability of PV energy systems. Conventional approaches to train object identification models, including YOLO (You Only Look Once), usually necessitate extensive datasets in order to attain optimal performance. This presents a substantial obstacle when there is only a little amount of data accessible. In order to tackle this issue, pre-trained models are frequently employed due to their ability to capture robust and widely applicable characteristics, which in turn enhance the learning process during fine-tuning. This work introduces a novel method for detecting defects in PV systems. One of The latest versions of YOLO, known as YOLOv8n, integrates deeper and more intricate layers, resulting in enhanced speed and accuracy. The method combines CLIP (Contrastive Language-Image Pre-Training) embeddings with YOLOv8n to improve the accuracy and efficiency of defect identification. Our proposed approach utilizes CLIP by OpenAI to harness rich multimodal embeddings that improve the contextual comprehension of visual input. Our technique enhances data efficiency, flexibility, and robustness by initializing YOLOv8's layers with CLIP embeddings. This flexible approach enables the model to achieve good performance even when there are less training samples, efficiently acquiring distinguishing information from the small dataset. The YOLOv8nCLIP model, when integrated, exhibits substantial enhancements in performance compared to current approaches. The achieved metrics are as follows: a precision of 95. 74 %, a recall of 96. 51 %, and a mean average precision of 98. 5 %. The results revealed that the detection of PV problems using the current model surpasses the accuracy and reliability to the state of the arts methods. By including CLIP embeddings, the model can achieve high performance even with a limited number of training samples, quickly adapt to new tasks, and demonstrate improved resilience during stress testing.