Text Size
Fused Reality: Welcome to the Latest in Simulation
December 5, 2012

Dryden pilot Troy Asher, left, readies himself in the cockpit of a Learjet that once at altitude will use the new Fused Reality flight simulator for an aerial refueling drill.Dryden pilot Troy Asher, left, readies himself in the cockpit of a Learjet that once at altitude will use the new Fused Reality flight simulator for an aerial refueling drill. (NASA/Tom Tschida)

Simulation is an excellent tool that allows pilots an opportunity to train in an unfamiliar aircraft or to gain experience and confidence for a new mission.

A disadvantage of simulation has been that the pilot does not have the sense of being in the cockpit and some of the cues, like motion, that come with that experience are often missing. Systems Technology Incorporated, or STI, of Hawthorne, Calif., developed a new simulation system as part of a Dryden Small Business Innovative Research agreement that can bridge that gap.

The new system is called Fused Reality and it allows a pilot to use a blend of actual flying and a flight environment assembled with live video and virtual reality to provide for the most realistic simulation experience currently available. Bruce Cogan, Dryden research engineer and Phase 2 contracting officer's technical representative, said the company's success would benefit Dryden and NASA.

Asher is seen from another view with the Fused Reality helmet.Asher is seen from another view with the Fused Reality helmet. (NASA/Tom Tschida) "They delivered beyond what was expected and they are providing us with a system that will allow us to take the next step, which is putting the system in a NASA DFRC aircraft to support flight research experiments. This will be possible by interfacing Fused Reality with a Piccolo autopilot (a commercially available flight management system) to provide aircraft position data. It (Fused Reality) is good technology that NASA can really use."

Aerial refueling training is one example of how this technology can be used. While an aircraft is in flight, the pilot uses a helmet that permits him or her to see the sky view coming in from a camera, while the out-the-window view of the "drogue" and "aircraft" he or she is to mate with for refueling are from a virtual reality element of the simulation.

The environment through the helmet is achieved through a video board and an image generator drawing from live video. Sunlight from the aircraft's windows provides the virtual frame for the virtual world, explained Edward Bachelder, STI technical director and inventor of Fused Reality.

"Sensors on the aircraft add to the flight simulator. It's the motion of the aircraft that drives the simulation," Bachelder said. "We could not have developed this without NASA. NASA provided the dollars and the inspiration."

Following a successful set of ground tests this summer, Dryden pilots Troy Asher and Jim Less flew the system from Mojave Airport with the Calspan Corporation's Learjet 25 variable stability in-flight simulator.

"It is a cross between a simulator and a video game," Asher said. "It's not like anything I've seen before. You get into the familiar environment of the airplane and the goggles display virtual objects (like a tanker out the window) on top of what's really there. It's like being in a very realistic cartoon where all the acceleration and tactile cues come from the airplane."

Less sees uses for Fused Reality.

"This system has great potential for training and flight test. In flight test, this system would allow us to evaluate experimental control laws against operationally representative tasks such as aerial refueling, formation flying, target tracking, or even landing. These tasks would be displayed virtually to the pilot and could be accomplished safely away from the ground and other aircraft. This would permit more realistic testing of new systems with greatly reduced risk," Less said.

Fused Reality could be used on a number of Dryden aircraft including the T-34, the TG-14 and the F/A-18 aircraft, Cogan explained. The system may also have a role in NASA's Aviation Safety program as part of flight research programs at Wichita State University, Wichita, Kan., and the University of Iowa in Iowa City, Iowa, Cogan said.

Dryden researchers had an opportunity to fly the Fused Reality system during a presentation at Dryden this summer.Dryden researchers had an opportunity to "fly" the Fused Reality system during a presentation at Dryden this summer. (NASA/Tom Tschida)

David R. Landon, STI CEO, said with the Fused Reality System, "the magic is in the math, not in the hardware." In other words, the system's complex algorithms, or mathematical equations, are what make it work. That's important, Landon said, because every few years a customer could invest in the most modern computer, helmet mounted display and the latest version of the Fused Reality software and turn any cockpit in about 30 minutes into a simulator at a fraction of the cost of building a new simulator or upgrading a current simulator.

Key partners also include the Calspan Corporation and the United States Test Pilot School at Edwards Air Force Base.

Whatever the future is for this new technology, NASA provided the support for industry to develop and commercialize it. It's part of NASA's mission to commercialize technology and the Fused Reality system is a government and industry success story.

Image Token: 
Image Token: 
Page Last Updated: July 28th, 2013
Page Editor: NASA Administrator