RGB image-based detection of nutrient deficiencies in barley by deep learning methods.

Two or four spring barley genotypes, respectively, were cultivated under single-nutrient-deficient conditions of essential mineral nutrients in four experimental systems: (i) a hydroponic system in greenhouse (HyPo), (ii) a pot experiment in soil in greenhouse (SoPo), (iii) a Mini-Plot experiment under field-like conditions (MiPlo), and (iv) a long-term fertilizer field experiment (Diko). Shoot phenotypes were recorded using a Huawei P20 smartphone camera, thus generating large-scale datasets of annotated RGB images that were analyzed with Convolutional Neural Network (CNN)-based deep learning (DL) models. Plant samples were collected to determine the nutritional status of the plant. For an initial assessment of the suitability of the experimental systems, we present plant morphological data and nutrient concentrations as well as five comprehensive RGB datasets raised from the conducted experiments and respective CNN model prediction accuracies by experiment and treatment. Investigated target nutrients of deficiency treatments were: N, P and K across all experiments, in MiPlo additionally S and B, in SoPo and HyPo, furthermore, Mn, Zn, Mo and Cu, and only in Hypo Ca, Mg and Fe. Target nutrient concentrations in plants grown under deficiency treatments were reduced across all experiments, but did not always result in a reduced biomass. In particular, even though S, B and Mo deficiency treatments could be detected by the CNN model, the evaluated plant parameters hardly indicated the respective deficiencies. The achieved average prediction accuracies from RGB datasets including all investigated genotypes and nutrient treatments per experiment were 38.52% in HyPo, 61.78% in SoPo, 70.3% in MiPlo and 94.5% in Diko, respectively. The achieved average prediction accuracies from standardized RGB datasets, including only those genotypes and nutrient treatments (N, P and K) that were investigated across all experiments, were 48.8% in HyPo, 88.8% in SoPo, 85.5% in MiPlo and 82.4% in Diko, respectively, indicating that soil-based experiments are more suitable for acquisition of RGB datasets for CNN-based nutrient deficiency detection than experiments in nutrient solutions. The results further highlight the potential of an RGB-based deep learning approach for an element-specific non-invasive on-site and real-time diagnosis of nutrient deficiencies in crop plants.