Description of Driving Event:
Potential Inadequacy of NASA Agencywide Software Safety Policy Requirements
Lesson(s) Learned:
NASA's Agencywide software safety policy allows projects latitude to tailor their software safety plan for safety-critical software. It does not, however, require projects to obtain center Safety and Mission Assurance (S&MA) approval of the tailored software safety plans nor does it require Verification and Validation (V&V) per se. While the software assurance standard does mention V&V, it does not require any independence of V&V for safety-critical software.
Recommendation(s):
27a. NASA should require approval of a project's tailored software safety plan by both the center S&MA organization and by one administrative level higher than that making the request. 27b. NASA's software safety plan should require formal V&V of safety-critical software. Testing alone does not suffice. 27c. NASA should develop an explicit policy that requires independent V&V for safety-critical software.
Evidence of Recurrence Control Effectiveness:
27a. NASA agrees with the intent of this recommendation but believes the requirements for formal system safety program plans and software management plans exist and, with proper and firm enforcement, fulfill the objective of this recommendation. To be sure that these requirements are perfectly understood, the Office of Safety and Mission Assurance (OSMA) will update NSS 1740.13, "NASA Software Safety Standard," to explicitly state that the program/project manager for programs/projects perform an assessment to determine, based on the level of criticality and risk, the scope and level of Independent Verification and Validation (IV&V) to be planned. The results of the assessment will be formally reviewed by Center Safety and Mission Assurance (SMA). The program/project manager, in consultation with SMA, will tailor an approach to ensure that the appropriate V&V requirements are established and implemented. The OSMA will place more emphasis on the implementation and enforcement of these existing requirements. Process verification, recently established in the OSMA, will be used to evaluate and enforce these existing policy and requirements more aggressively. NASA is committed to assuring that required program management plans and any subordinate plans such as software or safety management plans cover the essential requirements for programs where warranted by cost, size, complexity, lifespan, risk, and consequence of failure. Additional changes are being incorporated into NPG 7120.5, "NASA Program/Project Management Guide" (currently under development), to ensure that necessary and sufficient requirements will be fulfilled for programs having software vulnerabilities. SMA organizations at each level are to be a party to these decisions and are to intervene where necessary to assure that proper and clearly documented decisions are made by the appropriate level of management. The Program Management Councils could play a role in adjudicating any issues with the content of program management plans. 27b. NASA agrees with the intent of this recommendation and is confident that NPD 2820, "NASA Software Policies" (currently under development), will ensure that software management and/or safety plans developed for any NASA program/project will specify the level of V&V and types of testing that should be implemented. NPD 2820 policy relating to software V&V states that NASA will create and/or acquire and maintain software through risk-based management. Risk management products shall be documented or referenced in a management plan. NASA will employ V&V, IV&V, and other trusted verification techniques for appropriate risk mitigation based on the cost, size, complexity, lifespan, risk, and consequence of failure. NSS 1740.13 is not a NASA Software Safety Plan. The level of detailed requirements that the ASAP is recommending be in NSS 1740.13 more appropriately belongs in the documents that the programs and projects will prepare in response to NSS 1740.13. These details need to be documented in the Software Management Plan (SMP), the System Safety Program Plan (SSPP), or the Program Management Plan (PMP). NASA maintains that the requirement stated in NASA-STD-2100-91, "Software Documentation Standard," suffices as a requirement for addressing the issue of V&V for programs. NHB 1700.1, "NASA Safety Policy and Requirements Document," requires NASA program managers to publish and maintain an approved NASA Safety Management Plan (SMP). The program manager is responsible for approval of the NASA SMP and contractor Safety Program Plan (SPP). The system safety manager (SSM), assigned by the program manager (PM) from the Center Safety and Mission Assurance organization, prepares the SMP. The SSM, reviews the contractor's SPP and provides recommendations to the PM. The SSM, upon review of the contractor's SPP, will have the required insight into the approach for software V&V to ensure that a proper balance of analyses, inspections, and testing is planned for the entire life cycle of the program. NASA's SMA organizations at the NASA Centers are currently involved in the review of V&V plans for software and do make recommendations to the PM. 27c. NASA agrees with the premise that safety-critical software is a prime candidate for IV&V; however, it is NASA's position that all software determined to be safety-critical by engineering or safety analyses need not be subjected to IV&V. To be sure that program/project managers plan for the proper level of both V&V and IV&V from the outset, the OSMA will update NSS 1740.13, "NASA Software Safety Standard," to explicitly state that program/project managers perform an assessment to determine the scope and level of IV&V based on the level of criticality and risk. The results of the assessment will be formally reviewed by SMA. This way, the program/project manager, in consultation with SMA, will tailor an approach to ensure that the appropriate V&V requirements are established and implemented. NPD 2820 has incorporated the proper approach for software on NASA programs/projects; the directive requires program managers to employ IV&V, V&V, and other proven verification techniques for risk mitigation, based on cost, complexity, risk, and consequence of failure. NPG 7120.5, "NASA Program/Project Management Guide" (currently under development), will reflect some of the requirements now found in documents that program managers may not normally review for compliance.
Documents Related to Lesson:
N/A
Mission Directorate(s):
- Exploration Systems
- Aeronautics Research
Additional Key Phrase(s):
- Aerospace Safety Advisory Panel
- Computers
- Independent Verification and Validation
- Policy & Planning
- Research & Development
- Safety & Mission Assurance
- Software
Additional Info:
|
