Deep unfolding inference for supervised topic model

Abstract

Conventional supervised topic model for multi-class classification is inferred via the variational inference algorithm where the model parameters are estimated by maximizing the lower bound of the logarithm of marginal likelihood function over input documents and labels. The classification accuracy is constrained by the variational lower bound. In this study, we aim to improve the classification accuracy by relaxing this constraint through directly maximizing the negative cross entropy error function via a deep unfolding inference (DUI). The inference procedure for class posterior is treated as the layer-wise learning in a deep neural network. The classification accuracy in DUI is accordingly increased by using the estimated topic parameters according to the exponentiated updates. Deep learning of supervised topic model is achieved through an error back-propagation algorithm. Experimental results show the superiority of DUI to variational Bayes inference in supervised topic model.