SeqαGAN: Sign Language Sequence Generation Based on Variational and Adversarial Learning

Recent studies on sign language recognition (SLR) mainly focus on deaf individuals communicating with those who can hear. However, less attention has been given to enable hearing individuals to effectively communicate with the deaf, especially when they do not understand sign language (SL). To overcome this, we introduce SeqαGAN, a unique SL sequence generator. This tool is different from the standard variational autoencoder (VAE) + generative adversarial networks (GAN) model used commonly for creating images. Instead, our approach fuses conditional VAE and conditional GAN for generating sequences. SeqαGAN merges variational and adversarial learning to create versatile, recognizable data, which helps avoid mode collapse. The generated data could enhance SLR performance by supplementing existing resources. Tests on well-established SL datasets confirm the effectiveness of this new skeleton-based SLG model.