Discriminative Mixture Variational Autoencoder for Semisupervised Classification

In this article, a deep probability model, called the discriminative mixture variational autoencoder (DMVAE), is developed for the feature extraction in semisupervised learning. The DMVAE consists of three parts: 1) the encoding; 2) decoding; and 3) classification modules. In the encoding module, the encoder projects the observation to the latent space, and then the latent representation is fed to the decoding part, which depicts the generative process from the hidden variable to data. In particular, the decoding module in our DMVAE partitions the observed dataset into some clusters via multiple decoders whose number is automatically determined via the Dirichlet process (DP) and learns a probability distribution for each cluster. Compared to the standard variational autoencoder (VAE) describing all data with a single probability function, the DMVAE has the capacity to give a more accurate description for observations, thus improving the characterization ability of the extracted features, especially for the data with complex distribution. Moreover, to obtain a discriminative latent space, the class labels of labeled data are introduced to restrict the feature learning via a softmax classifier, with which the minimum entropy of the predicted labels for the features from unlabeled data can also be guaranteed. Finally, the joint optimization of the marginal likelihood, label, and entropy constraints makes the DMVAE have higher classification confidence for unlabeled data while accurately classifying the labeled data, ultimately leading to better performance. Experiments on several benchmark datasets and the measured radar echo dataset show the advantages of our DMVAE-based semisupervised classification over other related methods.