SAR-3DTR: A Novel Feature Hybrid Transformer Network for End-to-End 3-D Target Reconstruction From SAR Images

Recovering the 3-D representation of anisotropic targets from synthetic aperture radar (SAR) images is an intractable difficulty in the field of SAR target image interpretation. Recently, deep learning-based approaches have made significant progress in classical 2-D vision tasks, such as SAR target recognition and detection. However, due to the lack of spatial and depth information in SAR images, as well as the more complex task modeling and data requirements for 3-D vision tasks, it is difficult to apply deep learning-based approaches to 3-D reconstruction from SAR images. To address these challenges, we reformulate the SAR images 3-D reconstruction as a sequence-to-sequence prediction problem and propose an end-to-end deep learning framework based on a feature hybrid Transformer network, called SAR-3DTR. The proposed method first extracts visual features from SAR image sequences. In addition, considering the unique scattering imaging properties of SAR images, we also extract and construct scattering features, which represent rich physical structure information of target by introducing a novel vector of electromagnetic scattering features (VESFs) module. Finally, Transformer network learns 3-D features and generates model prediction by exploring global correlations in visual, scattering, and 3-D spatial feature domains. Experiments conducted on the synthesized aircraft target dataset and the MSTAR ground target dataset show that our method has superior quantitative and qualitative results and reaches state-of-the-art performance on SAR 3-D reconstruction.