Text Size

Students Use Xbox Kinect to Enhance Visualizations of Earth
Students at NASA have found that dance battles and bowling aren't the only things you can do with Kinect videogame technology.

Instead of linking Microsoft's Kinect to an Xbox, students in NASA's training and development program, DEVELOP, put the Kinect into their 3-D virtual environment to display Earth-observing satellite data.

Overhead view of HIVE structure

Click to enlarge

An overhead view of the entire HIVE structure, including the infrared emitters (lower left), models of the liquid-crystal stereographic shutter glasses (lower center) and the short-throw rear projectors (lower right). Credit: NASA/DEVELOP

The virtual experience, known as the Highly-portable Immersive Virtual Environment (HIVE), is within a structure consisting of three white projector screens, all angled at 120 degrees. To see the 3-D maps, graphs and charts, viewers are given 3-D glasses to wear as they walk up to the screens.

"These are more high-tech than the glasses you get at movie theaters," explains Jasmine Walker, the project lead for the HIVE team at NASA's Langley Research Center in Hampton, Va. "They blink at 120 hertz per second and work with infrared emitters to give you a 3-D effect."

Once inside the HIVE, visualizations of Earth, generated from NASA satellite data, appear to jump off of the screens, immersing you in a virtual environment.

To display the data for viewers, someone on the team must sit at a computer to click through the visualization. However, the students working on the HIVE during this year's spring term felt that was too cumbersome.

"We wanted to make the HIVE a more user-friendly interface, so instead of using a mouse and keyboard, we thought, why not use your body?" says Walker.

Kinect technology allows just that -- computer input with no remote controls, just body movements and speaking. The HIVE team, which consists of Jasmine Walker, Michael Linisinbigler and Nathan Walker, gained access to Kinect's raw data streams and manipulated the codes to fit their needs.

A DEVELOP student demonstrates the ability to alter visualizations within the HIVE using the Microsoft Kinect and hand gestures. The visualization show here is an engineering model of NASA's Clouds and the Earth's Radiant Energy System (CERES) instrument. Credit: NASA/DEVELOP

"Now while you are in the HIVE, whenever you want to change the visualization you just give a gesture and verbal command," explains Jasmine Walker. "For example, when I move my hand and say 'computer, move,' the visualization instantly moves forward."

Jasmine Walker explains the only input the Kinect doesn't recognize in the HIVE is facial recognition and finger tracking. However, now that navigating the visualizations is simpler, visitors to the HIVE could direct themselves through the 3-D environment.

Satellite data the HIVE team turns into visualizations often come from other DEVELOP teams' research projects, which address social issues with Earth science research.

Last summer the HIVE team displayed a visualization supporting NASA's SERVIR initiative, which monitors and forecasts environmental changes. The visualization features Moderate Resolution Imaging Spectroradiometer (MODIS) fire data from the Fire Information for Resource Management System (FIRMS) to display fires across India and surrounding areas over several days. The scales of the flames in the 3-D visualization were based on the intensity of the fire, so if the fire was bigger, the flame image was bigger as well.

The HIVE team also integrated a visualization that shows the effects of the Gulf of Mexico oil spill in 2010. The data, which came from the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) satellite, was layered with MODIS data to create a visual narrative.

While the data is visualized in a way that is easy for general audiences to understand, the HIVE team likes to have a story told while the visualization is playing.

"We often have a team member who worked on the research in the HIVE to explain the significance while we are displaying the data," says Jasmine Walker.

The HIVE is extremely portable, taking only 30 minutes to set up, so the team can travel to various locations to show off the research projects. Now that they have integrated the Kinect technology, visualizing data has become even simpler -- and more fun.

"Everyone on the team likes video games, and we really wanted to increase the immersion experience," says Walker. "We've seen others integrate the Wii into projects like this, but we wanted to be different. If the Kinect can do everything a Wii can without a controller, we figured we could too."

Jennifer LaPan
NASA Langley Research Center