The Evaluation of Research Papers (Or How to Get an Academic Committee to Agree on Something)

The College of Commerce and Finance at Villanova University sponsors an annual research awards program. Its objectives are to encourage and reward faculty research efforts. The process of selecting papers to win the award turned out to be far more complex than we initially expected. After many unsuccessful attempts at developing a manageable procedure, we used the analytic hierarchy process to structure the selection process. We have used this approach for the last three years without any serious problems. The faculty and external advisors using the system believe that it can better capture the subjective judgments of the evaluators than any of the previous approaches.