Predicting the Best System Parameter Configuration: the (Per Parameter Learning) PPL method

Search engines aim at delivering the most relevant information whatever the query is. To proceed, search engines employ various modules (indexing, matching, ranking), each of these modules having different variants (e.g. different stemmers, different retrieval models or weighting functions). The international evaluation campaigns in information retrieval such as TREC revealed system variability which makes it impossible to find a single system that would be the best for any of the queries.While some approaches aim at optimizing the system parameters to improve the system effectiveness in average over a set of queries, in this paper we consider a different approach that aims at optimizing the system configuration on a per-query basis. Our method learns the configuration models in a training phase and then explores the system feature space and decides what should be the system configuration for any new query.The experimental results draw significant conclusions: (i) Predicting the best value for each system feature separately is more effective than predicting the best predefined system configuration; (ii) the method predicts successfully the optimal or most optimal system configurations for unseen queries; (iii) the mean average precision (MAP) of the system configurations predicted by our approach is much higher than the MAP of the best unique system.