Exploring the 3D Gravity Inversion Problem Using a Compact Hybrid Transformer‐Based Deep Learning Model

The three‐dimensional (3D) gravity inversion problem is the process of delineating the volumetric mass distributions from the surface gravity anomalies. Recently, many innovative Convolutional Neural Network (CNN) based algorithms have found some success in reduction of computation costs and delineation of sharper boundaries of causative sources. Despite this success, most deep learning models require an extensive set of data pairs to achieve satisfactory performance and significant computational costs during training. We propose a new hybrid transformer model which is significantly smaller than its U‐Net based counterparts. The hybrid transformer model consists of CNN layers for the initial layers to capture the local features in the ground gravity data. The convolutional tokens thus generated from the CNN layers are fed to a transformer encoder for global context modeling. For the transformer part, several modifications were made to the vanilla transformer architecture, such as rotary positional embeddings and T‐Fixup initialization, to speed up training. Apart from the regular cuboidal and dipping prisms, other geologically relevant subsurface structures were generated through diffeomorphic deformation of ellipsoids. The deep learning model thus obtained was one of the smallest models with only 1 million parameters, which could be trained and tested on a modest computational resource in a reasonable time. The model was found to exhibit good depth stability and noise suppression properties. The trained model could be easily fine‐tuned through focused data sets for specific use case scenarios. The approach was further tested on real data of the San Nicolas deposit in Mexico with good agreement.