Identifying Rice Field Weeds from Unmanned Aerial Vehicle Remote Sensing Imagery Using Deep Learning

Abstract Background Rice field weed object detection can provide key information on weed species and locations for precise spraying, which is of great significance in actual agricultural production. However, facing the complex and changing real farm environments, traditional object detection methods still have difficulties in identifying small-sized, occluded and densely distributed weed instances. To address these problems, this paper proposes a multi-scale feature enhanced DETR network, named MS-DETR. By adding multi-scale feature extraction branches on top of DETR, this model fully utilizes the information from different semantic feature layers to improve recognition capability for rice field weeds in real-world scenarios. Methods Introducing multi-scale feature layers on the basis of the DETR model, we conduct a differentiated design for different semantic feature layers. The high-level semantic feature layer adopts Transformer structure to extract contextual information between barnyard grass and rice plants. The low-level semantic feature layer uses CNN structure to extract local detail features of barnyard grass. Introducing multi-scale feature layers inevitably leads to increased model computation, thus lowering model inference speed. Therefore, we employ a new type of Pconv (Partial convolution) to replace traditional standard convolutions in the model, so as to reduce memory access time and computational redundancy. Results On our constructed rice field weed dataset, compared with the original DETR model, our proposed MS-DETR model improved average recognition accuracy of rice field weeds by 2.8%, reaching 0.792. The MS-DETR model size is 40.8M with inference time of 0.0081 seconds. Compared with three classical DETR models (Deformable DETR, Anchor DETR and DAB-DETR), the MS-DETR model respectively improved average precision by 2.1%, 4.9% and 2.4%. Discussion This model has advantages such as high recognition accuracy and fast recognition speed. It is capable of accurately identifying rice field weeds in complex real-world scenarios, thus providing key technical support for precision spraying and management of variable-rate spraying systems.