Time Series Forecasting of Air Pollutant PM2.5 Using Transformer Architecture

Transformer architectures are widely used, especially in computer vision and natural language processing. Transformers have been used recently in a number of time-series analysis applications. An overview of the Transformer architecture and its uses in time-series analysis is given in the literature review. To improve performance, the Transformer's primary parts-the encoder/decoder, multi-head, positional encoding, and self-attention mechanism-have been updated. To implement time-series analysis, a few improvements to the original transformer architecture were adopted. Additionally, the optimal hyperparameters values for overcoming the difficulty of successfully training Transformers for time-series analysis are provided in this work. The effectiveness of the Transformer model in forecasting PM2.5 concentrations is examined in this paper. The dataset is pre-processed as a first step. In order to minimize the input parameters while taking into account their statistical significance, multi-collinearity among the independent variables is found using a Variance Inflation Factor (VIF). The proposed model have been trained to forecast PM2.5 concentrations up to one day ahead of time.