Putting Information Retrieval Theory Into Practice

This paper describes a semester project for an undergraduate computer science senior elective course, CSCI 379 Computer Science Topics – Information Retrieval and Web Search, taught at Bucknell University in the fall semester of 2002. In this course, students working in groups developed a Web search engine using information retrieval theory. The project includes implementing a basic Web server which acts as the dispatcher, an indexing component which builds the inverted indexing system for search, a ranking component which ranks the documents based on term frequency (tf) and inverted document frequency (idf), a retrieval component which takes the user query and retrieves the documents based on the ranking, and an crawling component which collects documents from the Web. The project is very practical in that students have to build a complete system, yet it involves many theoretical aspects of the information retrieval, algebra, and probability. It is an ideal project for a senior level course which requires a combination of the knowledge students have learned in their college years.