Premium
Analysis of biomedical and health queries: Lessons learned from TREC and CLEF evaluation benchmarks
Author(s) -
Tamine Lynda,
Chouquet Cécile,
Palmer Thomas
Publication year - 2015
Publication title -
journal of the association for information science and technology
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.903
H-Index - 145
eISSN - 2330-1643
pISSN - 2330-1635
DOI - 10.1002/asi.23351
Subject(s) - clef , computer science , information retrieval , clarity , task (project management) , terminology , query expansion , unified medical language system , biochemistry , chemistry , linguistics , philosophy , management , economics
A large body of research work examined, from both the query side and the user behavior side, the characteristics of medical‐ and health‐related searches. One of the core issues in medical information retrieval ( IR ) is diversity of tasks that lead to diversity of categories of information needs and queries. From the evaluation perspective, another related and challenging issue is the limited availability of appropriate test collections allowing the experimental validation of medically task oriented IR techniques and systems. In this paper, we explore the peculiarities of TREC and CLEF medically oriented tasks and queries through the analysis of the differences and the similarities between queries across tasks, with respect to length, specificity, and clarity features and then study their effect on retrieval performance. We show that, even for expert oriented queries, language specificity level varies significantly across tasks as well as search difficulty. Additional findings highlight that query clarity factors are task dependent and that query terms specificity based on domain‐specific terminology resources is not significantly linked to term rareness in the document collection. The lessons learned from our study could serve as starting points for the design of future task‐based medical information retrieval frameworks.