DVT: transformer-based style transfer from reality to art

In advancing Neural Style Transfer (NST) for video applications, this study introduces a novel Transformer-based framework that enhances temporal consistency and computational speed. The approach utilizes a Vision Transformer (ViT) architecture, adapted to exploit the temporal dynamics inherent in video sequences. To achieve seamless style consistency across frames while ensuring real-time performance, we incorporate a hybrid Transformer-Residual learning module coupled with a bi-directional frame recurrence strategy. Our DVT: Dual-frame Video Transfer also integrates advanced model acceleration techniques, including network pruning and knowledge distillation, to foster an efficient processing environment capable of real-time stylization. Through rigorous experimental validation on diverse video datasets, our model outperforms current NST techniques, delivering superior stylization quality, improved temporal stability, and heightened computational efficiency. The findings highlight the pivotal components of our architecture.