Evaluation of a Recommender System for Single Pilot Operations

This paper discusses the quality of a recommender system implemented in a simulation to assist with choosing a diversionary airport for distressed aircraft. In the third of the series of studies investigating the feasibility of ground-supported single pilot operations (SPO) a recommender system was used by 35 airline pilots as an aid for selecting diversionary airports. These pilots, acting as ground operators, used the recommender system from a ground station when off-nominal events required them to provide ground support to a single piloted aircraft. The unique circumstances imposed by each of the scenarios required the ground operators, together with the recommender system, to consider the relative importance of different factors when recommending an airport. Post-trial questionnaires were used to evaluate the recommender system. Results indicated that the pilots did not find the recommender system very transparent and did not always trust its initial recommendation. However, pilots did appear to find the recommender system to be effective in supporting them with the high workload in off nominal situations, and interactions with the system appear to have been satisfactory. Pilots also reported in post simulation surveys a desire to have better explanations for those recommendations. Findings will inform the development of future iterations of the recommender system, as well as influence SPO procedures and further development of a prototype ground station