Semantic Representations based on Neural Topic Models

Topic Models is a tool to extract semantics from textual collections. However, in spite of extensive work on topic modeling, including its evaluation using Language Models (LM), there is need for more research to measure, analyze and validate the topics learned. To this end, we present a Neural Topic Model called ABT which leverages semantic vector spaces (derived from language models) to uncover latent semantic representations in text collections. ABT enables one to explore the themes covered by a given textual collection and analyze them as a hierarchical structure of topics. We demonstrate the effectiveness of ABT by analyzing two text corpora: (1) CliCR, a collection of 10,539 real clinical cases; and (2) MovieLens, a dataset consisting of 9,742 movie titles. We have experimented with different language models to investigate the effects of using specific tasks to tune the neural network. The topics that resulted from our study highlighted concepts that are shown to be effectively used as prominent topics to represent the datasets. The results show that each experimented LM focuses on distinct features to produce the topics and may generate topics with different grades of generality and specificity. Our study sheds light on recent findings of the related literature, which state that the topic coherence metric does not align with assessments performed by human evaluators.