This is the authors' abstract. We don't add key points for this paper.
This paper presents a novel spoken dual learning (SDL) frame-work aimed at the concurrent training of spoken language understanding (SLU) and generation (SLG) models. The central proposition is to consider SLU and SLG as conditional text generation tasks steered by prompts, thereby offering consistent goals and context-aware direction. SDL navigates the inherent dual task constraints through an innovative transformation technique utilizing noisy transcripts and intents. The blueprint of SDL incorporates high-profile, pre-trained speech models such as Wav2vec2 or Whisper, serving as the backbone. Extensive experimentation on datasets of varying sizes (small, medium, and large) provides empirical evidence demonstrating that SDL exhibits higher performance levels compared to models that are independently trained, as the cycle consistency of mapping be-tween speech, semantics, and text acts as effective regularization.
Answers are grounded in this paper and cite their sources.
No takes yet. Share an insight, caveat, or question.
Yang et al. (2024) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: