On-the-go table grape ripeness estimation via proximal snapshot hyperspectral imaging

The monitoring of grapes for ripeness estimation is a practice that enables fruit harvesting at the optimal time. Hyperspectral Imaging (HSI) represents a non-destructive and high-throughput alternative to traditional laboratory analyses. Current literature approaches perform hyperspectral measurements using line scan sensors or low-resolution static snapshot cameras, which hinder a fast per-bunch ripeness characterization. We propose a framework for on-the-go collection and processing of proximal snapshot hyperspectral images to estimate single bunch ripeness parameters. Focusing on table grapes ( Vitis vinifera L. cv. Red Globe), we collected images under natural illumination with a hyperspectral camera (500–900 nm) mounted on a moving vehicle in an experimental block sited in Piacenza, Italy. We investigated images collected in August and September 2021 representing two ripening stages. The composition of the imaged grape bunches was determined through laboratory chemical analyses to predict Total Soluble Solids (TSS) and anthocyanin concentration. The images were pre-processed via multimodal image registration to correct the unalignment of bands due to the vehicle motion, and the single bunches were automatically identified on false RGB images through a Mask Region-Convolutional Neural Network (Mask R-CNN) instance segmentation network. The mean spectra of the bunches were used as input features of a Partial Least Squares Regression (PLSR) model to predict the chemical parameters at single bunch and whole vine scales. The regression model of TSS had an R 2 (10-fold nested cross-validation) of 0.75 and 0.85 on a per-bunch and per-vine basis, respectively. The regression model of anthocyanin had an R 2 of 0.68 and 0.49 on a per-bunch and per-vine basis, respectively. The results suggest the potential of using snapshot hyperspectral images for high-throughput analysis of a per-bunch grape ripeness estimation. The method described in this study could give valuable information to improve grape ripening monitoring and management of harvest operations and even allow for precise and automated robotic harvesting. • On-the-go snapshot HSI was used to predict TSS and anthocyanin concentration in grape bunches. • Multimodal registration has effectively corrected band misalignment due to vehicle movement. • Mask R-CNN was used to identify single bunches on false RGB hyperspectral images. • TSS predictions outperformed those of anthocyanins. • Savitzky–Golay outperformed Standard Normal Variate and Multiplicative Scatter Correction.