Mobile Procedure Viewer (MOBIPV) - 10.18.17

Overview | Description | Applications | Operations | Results | Publications | Imagery

ISS Science for Everyone

Science Objectives for Everyone
The current International Space Station (ISS) laptop-based crew information system has certain drawbacks with respect to operator mobility and user interface options. The Mobile Procedure Viewer (mobiPV) investigation demonstrates new technologies and operations concepts for ISS crew procedure execution, both for onboard crew and ground teams.  The investigation also offers wireless operation, voice navigation, and real time (over the shoulder) video streaming of workplace activities and synchronized in-flight/on-ground procedure execution. The mobiPV system could be deployed for future crew procedures and contingency operations, with a potential to significantly shorten anomaly processing and resolution.
Science Results for Everyone
Move over, Siri! mobiPV technology offers wireless operation, voice navigation and real-time video streaming of crew work activities as well as synchronization of flight and ground procedures. This test of that technology provides evidence that the devices can improve astronaut efficiency, reduce crew time spent working  procedures, and allow ground teams to follow and anticipate crew activities. Potential hardware improvements based on crew feedback include a barcode reader for faster stowage operations and tools that allow collaboration between many ground teams or simultaneous operation by multiple astronauts. The system could significantly optimize crew and payload utilization time.

 

The following content was provided by Mikael Wolff, and is maintained in a database by the ISS Program Science Office.
Information provided courtesy of the Erasmus Experiment Archive.
Experiment Details

OpNom: mobiPV

Principal Investigator(s)
Mikael Wolff, European Space Agency - ESTEC, Noordwijk, Netherlands

Co-Investigator(s)/Collaborator(s)
Keshav Chintamani, Space Applications Service SA, Belgium
Matthias Maurer, ESA-EAC, European Astronaut Centre, Cologne, Germany

Developer(s)
European Space Research and Technology Centre (ESTEC), Noordwijk, Netherlands

Sponsoring Space Agency
European Space Agency (ESA)

Sponsoring Organization
European Space Agency

Research Benefits
Earth Benefits, Space Exploration

ISS Expedition Duration
March 2015 - September 2015; March 2016 - February 2018; -

Expeditions Assigned
43/44,47/48,49/50,51/52,53/54,55/56

Previous Missions
Information Pending

^ back to top

Experiment Description

Research Overview

  • The current crew procedure viewer is a web application running on T61p laptops, which has certain drawbacks with respect to operator mobility and user interface options. The supporting ground team(s) cannot follow the procedure execution in detail, thereby creating low situational awareness in the team, and potential issues with the on board task execution, making it very difficult to support in real-time operations.
  • The Mobile Procedure Viewer (mobiPV) experiment demonstrates new technologies and operations concepts for ISS crew procedure execution as task support, both for on board crew, and for the supporting ground expert team. The system is optimized for manual hands-on on board activities by providing maximum handling flexibility through wireless operation, voice navigation, and real time (over the shoulder) video streaming of workplace activities, as well as synchronized in-flight/on ground procedure execution for both. The mobiPV design is suitable for supporting both payload and system tasks. The mobile procedure viewer aims at demonstrating improved task performance for ISS crews, and for ground expert teams that have real-time “immersion” of the on board activity through a ground version of mobiPV installed at their premises.
  • The mobiPV system is fully compatible with on board crew procedure systems, so it could be deployed for future crew procedures. This collaborative configuration can also be used during contingency operations, with a potential to significantly shorten anomaly processing and resolution.

Description

The Mobile Procedure Viewer Short Duration Mission (mobiPV-SDM) experiment - focused around an onboard mobile and personal procedure viewer with productivity and collaboration add-ons - aims to demonstrate improved task performance by an onboard crew member performing payload or system hands-busy tasks, for example inflight maintenance or an experiment set-up. The performance improvement also encompasses the supporting teams of ground experts by providing them with real-time “immersion” of the onboard activity through a ground version of mobiPV installed at their premises. With permitted data links to the ground (i.e. Ku-band Forward/Return), the mobiPV system can support a real-time collaboration environment (“shared workspace”) for the distributed flight operations team; the crew member together with his/hers ground expert team. This collaborative configuration can also be used during contingency operations, with a potential to significantly shorten anomaly processing and resolution.
 
The mobiPV system
The mobiPV-SDM experiment evaluates improved technologies and operations concepts in response to the current shortcomings of iPV:
  1. The mobiPV system is mobile, using wireless techniques and rechargeable batteries so tethering to a fixed point in the Columbus module is not required
  2. MobiPV supports hands-busy crew activity better than the laptop based iPV by providing an additional user interaction option: voice navigation
  3. MobiPV is a configurable system consisting of commercial-off-the-shelf (COTS) hardware devices, and ESA custom software.
  4. Through a point-to-point data connection between 2 identical mobiPV systems, crew and a ground expert team can work on a “shared workspace”.
Main mobiPV end-user services
The mobiPV system provides the following main crew information system and collaboration services:
  1. Voice navigation of the procedure execution function
  2. Real time video streaming of workplace activities (“looking over the shoulder”)
  3. Voice loop for crew – ground conversations
  4. ODF procedure execution progress synchronized between flight and ground user; i.e. the green step marker is always at the same step on the 2 connected systems
  5. Real time photo documentation
  6. Annotation of procedure steps in multi-media format (text, images, audio and/or video clips.)
The listed services are identical for the onboard and ground users. The point-to-point communication between the two mobiPV systems is implemented with the improved Ku-band Forward/Return data link. The software applications implementing the services relies on standard IP network protocols. Note: When the Ku-band link is not available, the onboard mobiPV system is operational in its stand-alone mode.

^ back to top

Applications

Space Applications
Crew members on board the ISS perform many tasks, ranging from scientific experiments, to maintaining systems, and grappling visiting spacecraft with robotic arms. Remembering the details of how to perform all these operations would be impossible – and it’s not necessary. Instead, crew procedures are available for every job and many complicated tasks are performed step by step, just like following an instruction manual. This is simple and secure, but there is room for improvement.  The mobiPV system has the potential to free up a crew member’s time to concentrate on vital training tasks, while allowing mission controllers more freedom to assign other tasks as needed.

Earth Applications
The mobiPV experiment results can be applied in many multi-cultural/multi-lingual terrestrial environments, where an operator is separated from his/her expert, and a real-time hands-on task – e.g. a complex corrective maintenance task – is required to be performed.

^ back to top

Operations

Operational Requirements and Protocols

The 3 Protocols below are executed to gather qualitative and quantitative data to assess level of success in reaching the objectives:
 
Protocol 1 - Stand-alone mobiPV operations
 
The crew member uses mobiPV to perform an on board hands-busy scheduled task. Conventional ground support services (e.g. space-to-ground (S/G) voice loop) are available for this Protocol.   
 
Protocol 2 - Ground team connected in “read-only” mode
 
The crew member performs an on board hands-busy scheduled task, with the ground team participating by viewing “over-the-shoulder” and having its procedure viewer synchronized with the on board one. Apart from that, only conventional ground support is available for this Protocol.
 
Protocol 3 - Full real-time team collaboration
 
Full bi-directional connectivity between the ground and on board mobiPV systems supporting a scheduled hands-busy activity.
 
After each experiment session a list of questions is displayed to the users, both on board and on the ground. Answers can be provided by voice, or as input to a text file in the mobiPV system. Eurocom’s opinions will also be solicited.
 
The experiment should be performed by 1 crew member only, but if for any reason that crew member cannot execute all protocol sessions, qualitative data for experiment analysis will be acquired even if another crew member runs a session.
 
Sessions and Session Durations Estimates
 
The estimated total duration is 4 hours and 15 minutes + to be determined (tbd) minutes for the complete mobiPV SDM experiment. The tbd minutes are the nominal duration of the selected payload and system operations activities.
 
Analysis Approach
 
The mobiPV experiment is mainly analyzed qualitatively, using questionnaires (to be filled in immediately after a Protocol execution), and debrief sessions with the users on the ground and on board. Collected Columbus Lab video recordings, with mobiPV log-files are also the basis for the analyses. Some very basic quantative data, for example time for task completion, are also used for overall analysis.
 
As it is impossible to fully simulate the network performance end-to-end (crew member to/from ground expert center) prior to the mission, quality of service aspects are also analyzed based on subjective data from on board and ground mobiPV users.

^ back to top

Decadal Survey Recommendations

Information Pending

^ back to top

Results/More Information

mobiPV (mobile procedure viewer) provided evidence that consumer smartphone and tablet devices can improve astronaut efficiency and reduce crew time working through a procedure  via voice activated commands or touch screen commands.  The system also allows for ground teams to follow and anticipate crew activities, and together flight and ground team members are better equipped to quickly resolve issues. This first trial allowed for the next step in advanced hardware to provide easier tools for the crew based upon their comments such as barcode reader for faster stowage operations, complemented by mixed reality technologies, as well as tools that will allow for multi-collaboration with many ground teams or multiple astronauts simultaneous operations. These results will assist in optimization of crew system and payload utilization time.
 

^ back to top

Results Publications

    Boyd A, Fortunato A, Wolff M, Martinez Oliveira D.  mobiPV: A new, wearable real-time collaboration software for Astronauts using mobile computing solutions. SpaceOps 2016, Daejeon, Korea; 2016 May 16-20 10 pp.

^ back to top

Ground Based Results Publications

^ back to top

ISS Patents

^ back to top

Related Publications

^ back to top

Related Websites
mobiPV at ESA Erasmus Experiment Archive
New Ways of Doing: Improving Operations

^ back to top


Imagery