Fused Reality for Enhanced Flight Test Capabilities

Systems Technology, Inc. has developed a Fused Reality (FR) Flight system that allows a virtual environment to be integrated with a test aircraft so that tasks such as aerial refueling, formation flying, or approach and landing can be accomplished without additional aircraft resources or risk of operating in close proximity to the ground or other aircraft. Furthermore, for the first time, the dynamic motions of the simulated objects can be directly correlated with the responses of the test aircraft. The system allows real-time observation of and manual interaction with the cockpit environment that serves as a frame for the virtual out-the-window scene. The feasibility of the FR Flight system was successfully demonstrated using a Calspan Learjet in-flight simulator that was flown by three evaluation test pilots. Helmet/head tracking in the Learjet cockpit using infra red cameras and reflective markers allowed a virtual scene that was driven by actual aircraft motion to be presented to the pilot at the controls, while a safety pilot monitored the flight. The out-the-window scene was replaced in real time using filtering techniques, framed by the near cockpit environment that was left intact. Thus, the evaluation pilots were able to approach a virtual tanker and track an aerial refueling drogue that was extended from a Buddy Store on the wing of the tanker. The pilots were able to monitor key aircraft parameters from a virtual head-up display or by gazing at the actual cockpit instruments. This paper describes the experimental environment and flight test results from this successful technology demonstration.