Two-dimensional correlated topic models

Abstract

Latent Dirichlet allocation (LDA) is a widely-used topic model where a set of hidden topics is learned to model a collection of data in the form of bag-of-words. Correlated topic model (CTM) extends LDA, modelling topic correlation by using logistic normal prior for topic proportion vectors, instead of Dirichlet prior. However, in the case of bag-of-words from multiple data sources (for instance, streams of time-stamped measurements in sensor networks), where each word in a document is labeled with one of data sources from which the data comes, it is desirable to consider correlations across topics as well as across data sources. In this paper we present two-dimensional correlated topic model (2D-CTM) where we use a matrix-variate logistic normal distribution for a topic proportion matrix, in which correlations across topics as well as across data sources are captured by two covariance matrices of the matrix-variate normal distribution. We develop a mean-field variational inference algorithm for approximate posterior inference in our model 2D-CTM. We apply 2D-CTM to the problem of human activity recognition and sport tactic analysis, using data collected on multiple on-body sensors, with comparison to existing topic models.