Robust and Efficient 3D First-Arrival Traveltime Reconstruction using Swin Transformer and Masked Autoencoder

Abstract First-arrival traveltimes provide crucial information about subsurface velocities but can be missing due to environmental noise, equipment malfunction, or acquisition limitations. While spatial interpolation methods like kriging and inverse distance weighting have been used to address these issues, they struggle with the non-linearity and complexity inherent in 3D traveltime data. Recent deep learning approaches, such as Fully Convolutional Network (FCN), have shown promise in addressing these limitations but still face challenges in capturing global data variations, especially in 3-D cases. In this study, we adopt a deep learning framework combining a Swin Transformer-based Masked Autoencoder (MAE) with a Swin-Unet architecture for 3D traveltime reconstruction, validating its capability to capture both local and global data relationships. The Swin Transformer’s hierarchical self-attention mechanism is leveraged to learn diverse scales of representation, while MAE-based pretraining enhances the encoder’s ability to capture the intrinsic patterns in traveltime data by reconstructing masked portions. Blind tests show up to 80% reduction in root mean square error and 76% reduction in mean absolute error compared to existing FCN-based U-Net models. The results highlight the proposed method’s performance and potential for improving 3D traveltime reconstruction in complex geological settings.