STGym: A Modular Benchmark for Spatio-Temporal Networks With a Survey and Case Study on Traffic Forecasting

The rapid advancement of spatio-temporal domain has led to a surge of novel models. These models can typically be decomposed into different modules, such as various types of graph neural networks and temporal networks. Notably, many of these models share identical or similar modules. However, the existing literature often relies on fragmented and self-constructed experimental frameworks. This fragmentation hinders a comprehensive understanding of model interrelationships and makes fair comparisons difficult due to inconsistent training and evaluation processes. To address these issues, we introduce Spatio-Temporal Gym (STGym), an innovative modular benchmark that provides a platform for exploring various spatio-temporal models and supports research for developers. The modular design of STGym facilitates an in-depth analysis of model components and promotes the seamless adoption and extension of existing methods. By standardizing the training and evaluation processes, STGym ensures reproducibility and scalability, enabling fair comparisons across different models. In this paper, we use traffic forecasting, a popular research topic in the spatio-temporal domain, as a case to demonstrate the capabilities of the STGym. Our detailed survey systematically utilizes the modular framework of STGym to organize key modules into various models, thereby facilitating deeper insights into their structures and mechanisms. We also evaluate 18 models on six widely used traffic forecasting datasets and analyze critical hyperparameters to reveal their impact on performance. This study provides valuable resources and insights for developers and researchers. The source codes are available at https://github.com/JW-Shen/STGym.