A Genetic Programming Framework for Topic Discovery from Online Digital Library

Various topic extraction techniques for digital libraries have been proposed over the past decade. Generally the topic extraction system requires a large number of features and complicated lexical analysis. While these features and analysis are effective to represent the statistical characteristics of the document, they didn't capture the high level semantics. In this paper, we present a new approach for topic extraction. Our approach combines user's click stream data with traditional lexical analysis. From our point of view, the user's click stream directly reflects human understanding of the high-level semantics in the document. Furthermore, a simple, yet effective, piecewise linear model for topic evolution is proposed. We apply genetic algorithm to estimate the model and extract topics. Experiments on the set of US congress digital library documents demonstrate that our approach achieves better accuracy for the topic extraction than traditional methods.