Uncertainty of SYNOP Cloud Classifications by Human Observers

Human cloud observations are by design to some extent subjective since they are based on the observer's visual perception of the sky and features therein. In this work, we assess the uncertainty of operational SYNOP cloud observations in three different settings: (i) seven observers at the same station but at different times, (ii) observations conducted at the same time but at different stations about 20 km apart, and (iii) five volunteer meteorologists classifying clouds from the same set of ground‐based camera images. Investigations indicate the presence of substantial systematic differences for different observers and also at different stations, which cannot be explained purely by meteorological circumstances. When evaluated on a complex cloud classification scheme, where for each instance up to three out of 30 cloud types have to be chosen, inter‐observer agreement on image‐based classifications reached MCC scores up to 0.33. However, combining observations from different participants to a single report and using a classification scheme of reduced complexity strongly increases MCC scores up to 0.67. Results suggest that human cloud observations have to be used with caution if they are used for example, to evaluate new automatized cloud classification methods.