Topic Modeling of Large Scale Social Text

Abstract

In order to solve the problem of topic modeling for large-scale social short texts, this paper studied the parallel LDA modeling method and the dynamic topic model that can capture the dynamic characteristics of the topic. After that, a dynamic topic modeling method for large-scale text sets is proposed, which is based on the data decomposition and post-clustering method. It divided the whole corpus into independent fragments according to different features (e.g., time feature) and modeled the corpus in parallel. Then, it clustered the local topic in later stage. Experiments show that compared with DTM its execution time is less and it can capture the dynamic characteristics of the topic more effectively.