What question did this study set out to answer?

The aim is to develop a novel transformer-based neural network, SepFormer, for improved speech separation.

May 13, 2021

Attention Is All You Need In Speech Separation

Key Points

The aim is to develop a novel transformer-based neural network, SepFormer, for improved speech separation.
Introduced SepFormer, a neural network architecture based on transformers for speech separation.
Utilized a multi-scale approach that leverages multi-head attention mechanisms.
Evaluated performance on WSJ0-2/3mix datasets.
Achieved SI-SNRi of 22.3 dB on WSJ0-2mix and 19.5 dB on WSJ0-3mix datasets.
Outperformed previous systems while being significantly faster and less memory-demanding.
Demonstrated effectiveness even with downsampling of encoded representations by a factor of 8.

Abstract

Recurrent Neural Networks (RNNs) have long been the dominant architecture in sequence-to-sequence learning. RNNs, however, are inherently sequential models that do not allow parallelization of their computations. Transformers are emerging as a natural alternative to standard RNNs, replacing recurrent computations with a multi-head attention mechanism.In this paper, we propose the SepFormer, a novel RNN-free Transformer-based neural network for speech separation. The Sep-Former learns short and long-term dependencies with a multi-scale approach that employs transformers. The proposed model achieves state-of-the-art (SOTA) performance on the standard WSJ0-2/3mix datasets. It reaches an SI-SNRi of 22.3 dB on WSJ0-2mix and an SI-SNRi of 19.5 dB on WSJ0-3mix. The SepFormer inherits the parallelization advantages of Transformers and achieves a competitive performance even when downsampling the encoded representation by a factor of 8. It is thus significantly faster and it is less memory-demanding than the latest speech separation systems with comparable performance.

Mark Helpful

Bookmark

Relay