What question did this study set out to answer?

The aim is to develop a transformer architecture that generates sequences in parallel while maintaining global dependencies.

February 28, 2026Open Access

FourierNAT: a Fourier-mixing-based non-autoregressive transformer for parallel sequence generation

Key Points

The aim is to develop a transformer architecture that generates sequences in parallel while maintaining global dependencies.
Utilized Fourier-based mixing in the decoder for sequence generation.
Implemented discrete Fourier transform with learned frequency-domain gating.
Conducted experiments on WMT14 En-De and CNN/DailyMail benchmarks to validate the approach.
Achieved competitive performance on benchmark datasets compared to traditional methods.
Demonstrated reduced coherence gaps in generated sequences due to frequency-domain operations.

Abstract

We present FourierNAT, a novel non-autoregressive transformer (NAT) architecture that leverages Fourier-based mixing in the decoder to generate output sequences in parallel. While traditional NAT approaches often face challenges in capturing global dependencies, our method uses a discrete Fourier transform with learned frequency-domain gating to mix token embeddings across the entire sequence dimension. This design enables efficient propagation of context without explicit autoregressive steps. Empirically, FOURIERNAT achieves competitive results on WMT14 En-De and CNN/DailyMail benchmarks, highlighting that frequency-domain operations can mitigate coherence gaps often associated with NAT generation. Our approach underscores the potential of integrating spectral-domain operations to accelerate and improve parallel text generation.

FourierNAT: a Fourier-mixing-based non-autoregressive transformer for parallel sequence generation

Key Points

Abstract

Cite This Study