Hidden Markov model with Pitman-Yor priors for probabilistic topic model

Empirical studies of natural language have demonstrated that word frequencies follow power law distributions. However, standard statistical models often fail to capture this property. The Pitman-Yor process (PYP), a Bayesian non parametric model capable of generating power law distributions, has been widely used in probabilistic topic models to handle data with an infinite number of components. However, existing PYP topic models rarely account for the relationships between topics. Hidden Markov models (HMMs) are popular models for modeling topic relationships. To address this limitation, we propose a probabilistic topic model that combines HMM with Pitman-Yor priors. The posterior inference was performed by using variational Bayes methods. We applied our method to text categorization and compared it with two related topic models: the hidden Markov topic model and hierarchical PYP topic model.