Relevance Ranking Metrics for Learning Objects

This paper develops the concept of relevance in the context of learning object search. It proposes a set of metrics to estimate the topical, personal and situational relevance dimensions. These metrics are derived mainly from usage and contextual information. An exploratory evaluation of the metrics shows that even the simplest ones provide statistically significant improvement in the ranking order over the most common algorithmic relevance metric. Moreover, the combination of the metrics through the RankNet learning sorts the result list 50% better than the base-line ranking. The paper also presents open questions in the field of learning object relevance ranking that deserve further attention.