Study on New Term Weighting Method and New Vector Space Model Based on Word Space in Spoken Document Retrieval

Recently, TV news programs are broadcast from all over the world owing to the broadcast digitization. In this situation, TV viewers want to select and watch the most interesting news. In order to satisfy this requirement, news database has to be constructed which has automatic topic segmentation and retrieval function. In this paper, focusing on topic retrieval function, we propose a new spoken document retrieval method. There are two types of problems in spoken document retrieval. The first problem is how to eliminate the occurrence of error words caused by speech recognition. The second problem is how to extract the important words from spoken documents. In order to solve these problems, term weighting methods play an important role, because it can decrease the dimension of vector space model and usually increase the retrieval performance. We propose, in this paper, a new term weighting method "mutual information incorporating TF-IDF". We compared it with conventional TF-IDF, mutual information, TF-IDF based on word frequency and obtained a good result by our proposal method in spoken document retrieval as a result. At present, the vector space model and LSI are most widely used in information retrieval model. However, the similarity computed by standard cosine measure used in vector space model only indicates the similarity in common words between a query and documents. It is a problem that these models can't compute the similarity between different words included in a query and documents. From this view point, we propose a word space model which can compute the similarity between different words.