Predictive feedback augmentation for manual control of an unmanned aerial vehicle with latency

Teleoperation of unmanned aerial vehicles is hampered by communication delay, which causes feedback from command inputs to take considerable time to be displayed to the operator. For an international internet connection, round trip latencies can reach 500 ms. The satellite connections used for military unmanned aerial vehicles can have latencies in the order of seconds. This delay presents a substantial control problem, which has been solved in the past by control abstraction (instead of “roll left” the aircraft might be instructed “go to these coordinates”). Manual control remains difficult. This study borrows the client-side prediction concept from multiplayer video games to attempt to address the control delay to allow manual control. An estimate of the change in the vehicle state due to the commands that are yet to affect the feedback is computed and then the feedback that the pilot receives is modified to reflect this predicted change. Because of this change, the pilot can see immediately the effect of the control inputs. This study has explored the concept and built a prototype system functional in real time for flight testing with qualitative results presented.