Link of Transformers in CV and NLP: A Brief Survey

Transformer, a new type of neural network based on the self-attention mechanism, has revolutionized the field of natural language processing. Since Transformer has a more powerful representation ability than Convolutional Neural Networks (CNN), researchers are trying to apply it to computer vision tasks. In the past, most of the review papers summarized Transformers in the field of computer vision and natural language processing, respectively, which isolates the relationship between the two fields and fails to show the correlations and differences between the two fields. In order to better demonstrate the mutual promotion between the two fields, this paper will combine the development of Transformer in the two fields to provide a comprehensive review. Furthermore, we will review these Transformer models by their application scenarios and analyze their advantages and disadvantages. Finally, we discuss the challenges and some future research directions for Transformer.