CRUISE (CRUISE) - 04.20.16
CRUISE is a technology/crew operations demonstration experiment for a voice guided procedure execution and mixed manual & data-handling instructions (procedural display). Science Results for Everyone
On future missions into deep space, humans and machines will need to work together to handle unexpected, complex and potentially hazardous situations without help from Earth. This investigation demonstrated voice-guided teamwork between crew and computer. While the procedures were simple and fast, crewmembers had concerns about losing situational awareness, especially in the event of an error or the need to return to previous steps. Crew movement was limited by having the headset attached to a laptop for voice-activated interface, so Wi-Fi or a Bluetooth-type headset may be needed. This work will help improve mission efficiency and astronaut autonomy and comfort on long missions.
Mikael Wolff, ESA-ESTEC, Netherlands
Paolo Nespoli, ESA-EAC, Cologne, Germany
Astrium GmbH, Bremen, Germany
Sponsoring Space Agency
European Space Agency (ESA)
ISS Expedition Duration 1
September 2012 - September 2013
Previous ISS Missions
Crew autonomy for human exploration missions beyond LEO will require new crew information systems on-board. In non-space domains, as well as in commercial mass markets, many of the required building blocks for future crew assistants are already deployed; e.g. voice assisted user interfaces, and electronic instructions with command and control elements integrated therein. We believe that indeed also current Columbus flight operations can benefit from improvements of on-board crew operations support tools. The Cruise technology demonstrator (CRUISE) is a first step towards the deployment of such tools on Columbus, whilst preparing integrated ePartner solutions for exploration missions.
The CRUISE demonstrator shall be performed in the Columbus module since the operational scenarios are concerned with Columbus subsystem operations. The demonstrator shall be left on-board so it can be re-executed should opportunities arise.
Each assigned crew member shall perform 1 CRUISE session each during flight. Two non-flight sessions for each crew member shall also be conducted:
1. In conjunction with training there shall also be a CRUISE demonstration session.
2. Post-mission each crew member shall also perform a session. In addition, The recorded video for that crew member shall be used for the post mission session.
The CRUISE technology demonstrator is divided in two tests:
1. Voice activated procedure viewer (vaPV) 1: support for hands-busy procedure execution through voice input technology.
2. Procedural Display (PD); merging a crew procedure operations product (ODF - Operations data file) with real-time system data and command elements to support nominal ISS flight operations.
Note that only ODF crew procedures in checklist format [SSP 50253, NASA - ODF Standards] will be utilized. Both tests shall be executed on a stand-alone T61p laptop which shall not be connected to any on-board data or communications systems.
The vaPV is implemented as a Columbus system laptop version of the standard ISS crew procedure viewer (also known as iPV - International procedure viewer), enhanced with voice navigation capabilities such as Next, Previous (step). vaPV uses standard validated ODF procedures, processed in real-time to allow voice navigation commands. Enrolment by the test subject to the voice recognizer of the vaPV is not required as the system is based on a commercial speaker independent recognizer engine. vaPV does not yet support hands- and eyes-busy tasks since TTS (text-to-speech) functionality is not implemented. Note that the vaPV demonstrator always allows for parallel voice and traditional keyboard/pointer interaction with the procedure viewer.
The PD test is an implementation of a concept defined in [SSP 50313, NASA - Display and Graphics Commonality Standards] By implementing the required real-time data and command fields in the nominal procedure flow instead of in a separate display (or by feedback from ground control centre) the user can focus on the ISS task at hand instead of spending cognitive resources on finding and interacting with fields and buttons in a potentially cluttered laptop synoptic display. This predicted improvement is partially made possible due to incorporation of flight software user interface elements into on-board ODF procedures.
The operational scenarios the test subject will execute deals with:
1. A Columbus preventive ECLSS maintenance task from the IFM book - return grid sensor housing internal cleaning - for the voice activated procedure viewer test.
2. Simulated Columbus sub-system operations - sample procedures from the ECLSS (Environmental Control Life Support System) book - for the Procedural Display test.
Both tests are implemented as add-ons to the operational Columbus system software package Lapap (Laptop application). For both tests the activity will only be performed with the enhanced CRUISE user interfaces; allotted crew time will not allow for a “before/after” comparison. Instead, subjective data will be collected to assess the difference.^ back to top
With the nature of the work as an astronaut, any advances in technologies, or in procedures and protocols using such technologies, will help to improve mission efficiency and performance and make the lives of astronauts more comfortable in orbit, especially during longer-duration human exploration missions outside of low-Earth orbit. This type of research not only serves to improve astronaut efficiency on such missions, it also serves to improve astronaut autonomy which is a necessity for such future missions where communications delays will become a key factor.
The technologies used within this research are already used as a means to improve performance on Earth.
Crew operations efficiency and autonomy during long duration exploration missions with communications delays and no ground supervision will improve significantly if crew laptops (and subsequently mobile computing devices) incorporate technologies and document structures from the terrestrial PC and mobile device domains.
The proposed demonstrator posits that:
- Hands-busy procedure execution using voice input/output technology
- Crew procedures with real-time command and telemetry elements included will significantly improve user experience and performance, such as shortening task-to-completion time whilst reducing system-human error incidents.
Future human missions beyond LEO set high operational, human factors, and technical demands for a distributed support system. There is a need to enhance human-machine teams’ capabilities to cope autonomously with unexpected, complex and potentially hazardous situations. The long term goal is to provide such mission execution crew assistant (MECA) features implemented as personal ePartners [MECA phase 3 Final Report, TNO et.al., 30 Mar 2012]. This goal will require an iterative design and assessment cycle. This CRUISE demonstrator is a step in such a development process, demonstrating two salient building-block technologies for crew information systems, both involving various aspects of multi-modal user interface technology applied to on-board flight operations.
Crusade operations will let crew perform familiar ISS system operations but with enhanced crew laptop software, devices, and operations products. Thereby metrics of both objective (e.g. time for task completion) and subjective character (e.g. user experience) can be collected during and after every test session. By comparing in-orbit user experiences with pre- and post-mission test experiences, insight will be acquired on the transfer of ground test results to in-orbit results.
An explicit objective of the CRUISE technology demonstrator is to pave the way for operational use of the building block technologies voice activated procedure viewer and Procedural Display for Columbus operations, and prepare for ePartner deployment on missions beyond LEO. Some previous on-board experiments evaluating voice recognition have been performed e.g. [Applied Speech Recognition/Synthesis Techniques for Advanced Crew Terminal, Phase 2 Executive Summary, NLR et.al, Aug 1998], [WEAR Executive Summary, Space Applications Services, 5 May 2011] and [SDTO 17005-U Clarissa conversational procedure viewer and navigator, excerpt from SSP50448, revE, Sep 2008] but to date pre-processed crew procedures have been required whilst vaPV (Voice activated procedure viewer) makes use of validated normal ODF (Operations data file) crew procedures in XML (Extensible mark-up language) format.^ back to top
Decadal Survey Recommendations
Information Pending^ back to top
The conclusions of the on-orbit tests and post-test evaluations from crewmembers were that the Procedure Display was a simple interface and it was fast. Concerns were identified with overall loss of situational awareness especially in the case if an error occurred or if a crewmember needed to back up to a previous task step. Concerns with the voice activated interface due to the limitation of using a headset that was cabled to the laptop and therefore limiting movement for the crewmember. A suggestion was made to upgrade headset microphone to a Wi-Fi or Bluetooth system for the headset.
^ back to top
^ back to top
Ground Based Results Publications