Semi-supervised Clustering Through Representation Learning of High-dimensional Count Data

High-dimensional count data arise in many modern applications, such as electronic health records (EHR) and RNA sequencing, with sparse, heterogeneous observations driven by complex latent structure. These characteristics make modeling and prediction challenging, especially when labels are limited. We propose SCORE, a semi-supervised representation learning framework for clustering and embedding construction from high-dimensional count features. SCORE is built on a Poisson-adapted latent factor mixture model that supports incorporating external pre-trained feature embeddings when available, enabling efficient extraction of latent cluster membership and low-dimensional representations. To scale inference and learning to large datasets, SCORE uses a hybrid algorithm that combines expectation-maximization with a Gaussian variational approximation, leveraging a small labeled subset to refine estimation on a large pool of unlabeled samples. We establish convergence guarantees for this hybrid procedure, quantify approximation errors introduced by the Gaussian variational approximation, and derive error rates under increasing embedding dimensions. Our theory and experiments show that incorporating unlabeled data improves accuracy and reduces sensitivity to label scarcity, yielding strong finite-sample performance relative to existing approaches. We demonstrate SCORE for predicting disability status in multiple sclerosis using EHR data, where it learns informative patient embeddings with strong predictive performance, highlighting its practical value.