Search Ames


Ames Technology Capabilities and Facilities

Text Size

High Resolution Human Performance Modeling for Human-Robotic Teaming
Millisecond-level models of skilled human-performance allow safe, reliable and efficient planning and control of human robotic teaming in activities such as EVA Crew Exploration, Vehicle maintenance, and planetary in situ resource utilization.

This research lays out a path to a system that will support safe in-space assembly, robotic networks and resource utilization, all key Strategic systems that will support safe in-space assembly, robotic networks and resource utilization. Model-based design of Human-robotic (H/R) teams will enable quantitative analysis of team efficiency such that robust, reliable teaming can be planned and executed affordably. We have made great advances in the use of modular software components for human performance modeling allowing reusable model components that provide economies in H/R team development toward affordable team autonomy.

Research Overview
Our goal is to allow prediction of how humans, software agents, and robots all contribute to overall system performance. Current methods for modeling and predicting human performance either require specialized training or are too rigid to easily accommodate modeling of human and robotic performance. Most other cognitive architectures require that the modeler understand cognitive psychology and have sophisticated programming skills. The Constraint-based Optimal Reasoning Engine, or CORE, allows prediction of task times both early and late in practice (i.e. , from unskilled to skilled performance) and allow prediction of speed/accuracy trade-offs in the execution of skilled activity.

Modeling human performance

Right: Modeling human performance during vehicle maintenance.

X-PRT, a Graphic User Interface-based tool to support fast and accessible modeling with CORE will enable even novice users to effectively, create, debug, and visually verify the performance of their models. As a guided-entry interface, it assists the user in describing the essential steps in the task without imposing a particular sequence on those steps. It is well known that as users learn a task they begin to interleave elements. For example, before finishing one step, they might glance ahead at something they are going to need in the next step. The X-PRT interface enables even modelers to easily describe the task or set of tasks, the devices used (mouse, keyboard, screen, etc. ), and the cognitive strategy for achieving the task, such as moving the mouse, or turning a knob, or clicking a button. CORE then applies embedded (but modifiable) rules and constraints to the behavioral elements of the task described, and generates accurate a priori prediction of skilled performance including the characteristic anticipatory interleaving behavior demonstrated by experts.

The key is constraint satisfaction: performance can be more accurately predicted from a set of architectural, strategic, environmental, and task constraints. By defining these constraints separately instead of pre-emptively binding them to each other, CORE allows more flexibility in how their relationships can be computed. This maintains their independence from arbitrary constraints imposed by the machine or the software algorithms used to model the task. Together, CORE and X-PRT enable almost anyone to model new human-machine interaction quickly and affordably. CORE automatically predicts how long it would take for humans (from novice to expert) to to perform a task, as well as the probability of errors.

Space exploration activities such as CEV maintenance, EVA, and lunar drilling operations that require tight-loop interaction between humans and robotic agents place a high premium on team resources. For robots, the critical resource is power. For humans in space, the critical resources are power and air. For both, this translates into time as the most critical resource. CORE (Constraint-based Optimal Reasoning Engine) has demonstrated that very accurate generative models of human performance can be created with high (~50 ms) resolution allowing maximally efficient scheduling and reactive re-scheduling of H/R team time-on-task. By integrating a theory of composition from millisecond-level cognitive building blocks and task information, larger models of human performance are created.

Screenshot from the Constraint Editor Model

Right: Screenshot from the Constraint Editor Model.

The time it takes for an astronaut to perform a task on a computer, in the cockpit of a spacecraft, or on a planet with a robot may mean the difference between the mission’s success or failure, and even life or death. The system interfaces that astronauts use are critical to how quickly, accurately, and safely they can do a job. To develop and evaluate new system interfaces for future space missions, the HCI Group at NASA is developing new, more agile and accurate technologies for modeling human performance.