NASA Ames Research Center is developing machine vision capabilities to enable autonomous rovers and better ground operations. These capabilities are being developed for terrestrial, aerial, orbital, and planetary robots.
The future of human and robotic space exploration requires robotic precursors and robotic assistants that can safely and reliably interact with their environment and effectively team with their human partners. The machine vision capabilities developed at Ames will enable autonomous navigation, human robot interaction, spacecraft and surface structure inspection, and improved remote robotic operations.
Left: Alignment of 3D shape from stereo.
Machine Vision for Robot Autonomy
NASA Ames has developed machine vision for the K9 rover to navigate to rocks and other features for close inspection with contact instruments. K9 uses a visual tracking system which combines a 2D interest point based tracker and a 3D shape alignment technique to keep track of features and navigate to them. K9 also uses vision to watch for obstacles that present a hazard to the rover. Once K9 arrives at its goal, vision is used to find a place on the rock that is safe for the instrument and plan an arm motion that safely puts the instrument in place for a measurement.
The Personal Satellite Assistant (PSA) uses machine vision for free flying in a space station mock-up. Vision enables PSA to track and follow astronauts, navigate to particular locations in the station, keep track of its position, recognize and decode fiducial markers, and avoid hazardous obstacles.
Caption: Visual feature tracking for navigation.
The autonomous rotorcraft RMAX uses machine vision for control in flight. Stereo vision helps RMAX take off and land safely, and 2D tracking helps RMAX find its way and navigate where it needs to go.
MER MI image processing
Machine Vision for Ground Operations
The Ames Stereo Pipeline is a reconfigurable set of stereo vision tools for producing photorealistic 3D models. The Ames Stereo Pipeline has been used by scientists and rover operators in several NASA missions, from the IMP cameras on the 1997 Mars Pathfinder Lander to the MER rovers currently investigating Mars
Top: PSA; Bottom: RMAX
The Bayesian Vision group has developed 3-D super-resolution software. This software takes many images of a scene as input, and builds a 3-D model of the area at higher resolution than the input images. The 3-D model includes the geometry (shape) and surface reflectance properties (color).
The Microscopic Imager (MI) instrument on MER is a high resolution, narrow depth of field camera. Only part of each image is in focus, requiring the instrument to acquire a set of images at several distances. NASA Ames provided the MI Toolkit to MER in order to register and merge these images into a single image. The MI Toolkit also included stereo vision to produce 3D surface reconstructions from image pairs with horizontal offsets.
Images from robot cameras provide a wealth of information for robot autonomy. Onboard a robot, machine vision algorithms process this large volume of data and quickly extract information that is useful for autonomous activity, including planning, state estimation, navigation, and control.
Right: MER MI image processing.
For ground operations of remote robotic explorers, machine vision provides humans an immersive telerobotic environment for improved situational awareness as well as tools for image analysis. This can improve operations and contribute to the understanding of environments being explored.