A Transformer-based Fermi/GBM Background Predictor

As one of the primary scientific payloads on board the Fermi Gamma-ray Space Telescope, the Gamma-ray Burst Monitor (GBM) is designed to detect and study gamma-ray bursts as well as other high-energy transient events, including solar flares, magnetar bursts, and terrestrial gamma-ray flashes. A major challenge in GBM observations is the highly complex and variable background radiation, which can obscure astrophysical events and degrade subsequent analysis. Reliable background modeling is therefore crucial for reliable signal extraction and for enhancing the overall scientific performance of the instrument. We present a deep neural network approach for background spectral modeling of the GBM detector. Built on the transformer framework, our model uses intragroup multilayer perceptron encoding and intergroup multihead self-attention to capture complex dependencies among diverse physical variables. A learnable gating mechanism adaptively fuses the global context with feature-group information, enabling the model to simultaneously exploit global context and local physical correlations and thereby improve background estimation. Experimental results show that the model predicts light curves comparable to and in some cases better than those from orbit-level revisit analyses. For spectral reconstruction, the model predicts across 118 energy channels every 4.096 s, with more than 98% of channels showing standardized residuals lying within the ±3 σ range.