Drone-based photogrammetry combined with deep learning to estimate hail size distributions and melting of hail on the ground

Abstract. Hail is a major threat associated with severe thunderstorms, and estimating the hail size is important for issuing warnings to the public. For the validation of existing operational, radar-derived hail estimates, ground-based observations are necessary. Automatic hail sensors, for example within the Swiss Hail Network, record the kinetic energy of hailstones to estimate the hail sizes. Due to the small size of the observational area of these sensors (0.2 m2), the full hail size distribution (HSD) cannot be retrieved. To address this issue, we apply a state-of-the-art custom trained deep learning object detection model to drone-based aerial photogrammetric data to identify hailstones and estimate the HSD. Photogrammetric data of hail on the ground were collected for one supercell thunderstorm crossing central Switzerland from southwest to northeast in the afternoon of 20 June 2021. The hail swath of this intense right-moving supercell was intercepted a few minutes after the passage at a soccer field near Entlebuch (canton of Lucerne, Switzerland) and aerial images were taken by a commercial DJI drone, equipped with a 45-megapixel full-frame camera system. The resulting images have a ground sampling distance (GSD) of 1.5 mm per pixel, defined by the focal length of 35 mm of the camera and a flight altitude of 12 m above the ground. A 2-dimensional orthomosaic model of the survey area (750.4 m2) is created based on 116 captured images during the first drone mapping flight. Hail is then detected using a region-based convolutional neural network (Mask R-CNN). We first characterize the hail sizes based on the individual hail segmentation masks resulting from the model detections and investigate the performance using manual hail annotations by experts to generate validation and test data sets. The final HSD, composed of 18 207 hailstones, is compared with nearby automatic hail sensor observations, the operational weather-radar-based hail product MESHS (Maximum Expected Severe Hail Size) and crowdsourced hail reports. Based on the retrieved data set, a statistical assessment of sampling errors of hail sensors is carried out. Furthermore, five repetitions of the drone-based photogrammetry mission within 18.65 min facilitate investigations into the hail-melting process on the ground.