The TREC VIdeo Retrieval Evaluation (TRECVID): A Case Study and Status Report

The TREC Video Retrieval Evaluation (TRECVID) is an annual international effort, funded by the US Advanced Research and Development Agency (ARDA) and the National Institute of Standards and Technology (NIST) to promote progress in content-based retrieval from digital video via open, metrics-based evaluation. Now beginning its fourth year, TRECVID aims over time to develop both a better understanding of how systems can effectively accomplish video retrieval and how one can reliably benchmark their performance. This paper is a case study in the development of video retrieval systems and their evaluation as well as a report on the TRECVID status to-date. After an introduction to the evolution of TRECVID over the past 3 years, we report on the most recent evaluation TRECVID 2003 in terms of the 4 tasks (shot boundary determination, high-level feature extraction, story segmentation and classification, search), the data (133 hours of US television news), the measures, the results obtained, and the approaches taken by some of the 24 participating groups.