Open AccessMulti-feature automatic abstract based on LDA model and redundant control
Author(s) -
Dan Tang,
Yu Sun
Publication year - 2020
Publication title -
journal of physics. conference series
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.21
H-Index - 85
eISSN - 1742-6596
pISSN - 1742-6588
DOI - 10.1088/1742-6596/1693/1/012211
Subject(s) - automatic summarization , computer science , redundancy (engineering) , the internet , multi document summarization , information retrieval , quality (philosophy) , similarity (geometry) , feature (linguistics) , information overload , artificial intelligence , natural language processing , world wide web , linguistics , philosophy , epistemology , image (mathematics) , operating system
With the continuous popularization of computer application technology and the rapid development of Internet technology, there has been an explosion of information in every field, and more and more information is transmitted and stored on the Internet in the form of electronic text. Automatic summarization can solve the problem that the speed of traditional hand-written summaries cannot keep up with the speed of electronic text information generation. Automatic summarization can use a computer to generate a comprehensive and accurate coherent short essay on one or more documents, so people can quickly understand the main content of the text, which greatly improves the efficiency of people’s learning and work. Nowadays, automatic summarization technology has achieved rich results, but Chinese document summarization is still in the shallow semantic analysis of text, and there are problems such as redundancy of summarization and poor quality of abstract content. In view of the above problems, this article is in the research and implementation of Chinese automatic summarization technology. On the one hand, LDA is used to mine the topic distribution of documents to obtain high-quality abstracts; on the other hand, the similarity between the sentences to be extracted is calculated to remove redundant information.