Dynamic term selection in learning a query from examples

Learning a query from training examples is an important problem in information retrieval, filtering, and text categorization. We propose two methods that dynamically compute the number of terms to be selected from the training examples and added to an individual query. At the basis of both algorithms lies the observation that, given a set of terms ranked in order of decreasing weight, there seems to be a correlation between the curve of term weights and the average precision. More precisely, the average precision increases as long as the curve of term weights is sharp. When the curve becomes flat, the average precision either decreases or remains constant. The new approach aims to maximize the average precision of each query, while also reducing the number of terms per query and hence decreasing the response time of the system. Experimental results on five collections show that variable length queries generally behave better than fixed length queries.