Methodology for Identifying Mesoscale Weather Patterns from High-dimensional Climate Datasets

Abstract We develop a new methodology to solve the problem of identifying and selecting Mesoscale Weather Patterns (MWPs) from high-dimensional spatio-temporal climate datasets.This problem is important and topical and has many implications for decision-makers across multiple sectors, such as urban design, urban climate, agriculture, transportation, energy, disaster management and more. This problem involves selecting a small subset of data (specific days) from the original large dataset (decades long), such that it captures the essential information and characteristics of the large dataset, while minimizing redundancy. This is useful as it makes the dataset more manageable for processing, analysis and insight gathering without degrading the overall quality of the information content.We develop a novel algorithm that is based on advanced machine learning and optimization techniques, and consists of two stages: 1) Spatial Dimensionality Reduction (SDR) to reduce the number of spatial cells analyzed while preserving as much relevant information as possible; 2) Representative Subset Selection (RSS) to find a small subset of days in the dataset that captures the essential patterns, relationships and information present in the full dataset - these are the Mesoscale Weather Patterns.We demonstrate our methodology by applying it to a spatio-temporal dataset of atmospheric observations, the ERA5 dataset, in Singapore. The MWPs offer valuable insights into the region's diverse weather conditions and help researchers, climatologists and policymakers comprehend the complex interactions between atmospheric elements.