Improving quality of prediction in highly dynamic environments using approximate dynamic programming

In many applications, decision making under uncertainty often involves two steps—prediction of a certain quality parameter or indicator of the system under study and the subsequent use of the prediction in choosing actions. The prediction process is severely challenged by highly dynamic environments that particularly involve sequential decision making, such as air traffic control at airports in which congestion prediction is critical for smooth departure operations. Taxi‐out time of a flight is an excellent indicator of surface congestion and is a quality parameter used in the assessment of airport delays. The regression, queueing, and moving average models have been shown to perform poorly in predicting taxi‐out times because they are slow in adapting to the changing airport dynamics. This paper presents an approximate dynamic programming approach (reinforcement learning, RL) to taxi‐out time prediction. The taxi‐out prediction performance was tested on flight data obtained from the Federal Aviation Administration's (FAA) Aviation System Performance Metrics (ASPM) database on Detroit International (DTW), Washington Reagan National (DCA), Boston (BOS), New York John F. Kennedy (JFK) and Tampa International (TPA) airports. For example, at the Boston airport (presented in detail) the prediction accuracy by RL model was 14running‐average model. In general, the RL model was 35–50% more accurate than the regression model for all of the above airports. Copyright © 2010 John Wiley & Sons, Ltd.