Lesson Info:

• Lesson Number: 0791
• Lesson Date: 1999-02-01
• Submitting Organization: JPL
• Submitted by: Wil Harkins

Subject:

Abstract:

The identification of project/task risk just prior to launch/delivery becomes a costly and time-consuming activity. Early and continuous risk ratings enable management to focus on the issues with the highest probability of impacting mission success. Project management is provided with visibility to a concise subset (<5 percent) of a large information base focusing on the key problematic areas in a timely fashion.

Description of Driving Event:

This Lesson Learned is based on Reliability Practice number PD-AP-1305, from NASA Technical Memorandum 4322A, Reliability Preferred Practices for Design and Test.

Benefit:

Risk rating enables management to focus on the issues with the highest probability of impacting mission success. Project management is provided with visibility to a concise subset (< 5 percent) of a large information base focusing on the key problematic areas in a timely fashion.

Implementation Method:

A two-factored rating is assigned to each Problem/Failure (P/F) report. Figure 1 is a graphical outline of the rating system and indicates the basis of the "Red Flag" risk determination. All closed P/F reports having a potentially significant or major failure effect (Failure Effect Rating of 2 or 3), coupled with uncertain corrective action effectiveness (Failure Cause/Corrective Action rating of 3 or 4), are defined as "Red Flag" P/F reports.

[D]
Figure 1. P/FR Rating Chart

"Red Flag" Classification:

1. Each Red Flag P/F report has an attached statement that summarizes the symptom, cause, effect, potential mission impact, and corrective action for the P/F report, as well as the rationale for accepting the residual risk.
2. The Project/Task Manager and the Contractor Project Manager (if applicable) review, approve, and sign all Red Flag P/F report closures to acknowledge understanding and acceptance of the defined residual mission risk.
3. 3. All Red Flag P/F reports are discussed at subsequent formal reviews and reported in Monthly Project/Task Management Reports.

Each individual Red Flag P/F report is evaluated for its contribution to the overall mission risk. The total set of Red Flag P/F reports must be evaluated for any presentation of mission risk. The Red Flag P/F report contribution to mission risk is a topic for discussion at the preship and the prelaunch reviews.

In addition, any unrated P/F report open for 60 days or longer is reviewed and assigned a two-factored rating to determine if there are potential Red Flag issues.

As indicated in Figure 1, the two numerically indicated components of the P/F report rating system are the Failure Effect Rating and the composite Failure Cause/Corrective Action Rating.

Failure Effect Rating:

The first factor, the "Failure Effect Rating," is an assessment of the consequence or impact of the P/F if it had occurred in flight. It is not an assessment of the adequacy of the corrective action. Redundancy is not considered in making the assessment. The assessment is either 1, 2, or 3 based on the following criteria:

1. Rating 1: Negligible effect on mission performance.
   1. No appreciable change in functional capability.
   2. Minor degradation of engineering/science data.
   3. Support equipment or test equipment P/Fs.
   4. Support equipment, test equipment, or operator-induced P/Fs.
   5. Workmanship failures found at initial test.
   6. Drawing errors found at initial test.

Note: For items c, d, e, and f, after correction of the underlying P/F cause, the hardware/software must be certified to be in flightworthy condition and the failed test must be successfully completed.

2. Rating 2: Significant degradation to mission performance.
   1. Appreciable change in functional capability.
   2. Appreciable degradation of engineering/science data.
   3. Significant operational difficulties or constraints.
   4. Reduction in lifetime.
   5. Significant impact on system safety.
3. Rating 3: Major degradation to mission performance or catastrophic impact on system safety.

Failure Cause/Corrective Action Rating:

The second factor, the "Failure Cause/Corrective Action Rating," is an assessment, with confidence, that the failure cause has been understood correctly, and the confidence that the corrective action is effective and will preclude recurrence of the P/F. The assessment is either 1, 2, 3, or 4 based on the following criteria:

1. Rating 1: The cause of the P/F has been determined with confidence by analysis and/or test. An effective corrective action has been determined, implemented, and verified. Residual risk is considered minimum.
2. Rating 2: The cause of P/F has not been determined with confidence. However, corrective action is considered effective in precluding the observed P/F symptom. Residual risk is considered minimum.
3. Rating 3: The cause is considered to be "known" and "understood" with confidence. The correction accomplished within constraints (e.g., time, resources, test capability) does not satisfy with confidence all doubts or concerns regarding the corrective action and its effectiveness for precluding recurrence; or, the correction is considered to be a symptomatic treatment without ensuring an effective overall correction of the basic cause. There is residual risk.
4. Rating 4: The cause has not been defined with confidence. The correction of the P/F is considered to have an uncertain effectiveness for precluding recurrence. There is a residual risk.

Technical Rationale:

The P/F report risk rating system is a time-proven technique that permits a Project/Task to close all P/F incidents in a timely and efficient manner and yet retain the critical information relating to mission risk in a highly visible and specific format.

References:

1. "Problem/Failure Report Independent Review/Approval," Reliability Preferred Practice PD-AP-1304

Lesson(s) Learned:

The identification of Project/Task risk just prior to launch/delivery becomes a costly and time-consuming activity. Even with a protracted, intensive study, some key issues may be missed.

Recommendation(s):

Problem/failure (P/F) reports are assigned a two-factor set of ratings: a failure effect rating and a failure cause/corrective action rating. The composite rating is used to assess the hardware/software residual launch and mission risk. The high risk P/F reports are labeled "Red Flag".

Evidence of Recurrence Control Effectiveness:

This practice has been used on Mariners, Seasat, IRAS, VGR, VIK, GLL and MGN programs.

Documents Related to Lesson:

Mission Directorate(s):

• Exploration Systems
• Science
• Space Operations
• Aeronautics Research

Additional Key Phrase(s):

• Administration/Organization
• Configuration Management
• Flight Equipment
• Ground Equipment
• Hardware
• Launch Vehicle
• Parts Materials & Processes
• Policy & Planning
• Spacecraft
• Test & Verification

Additional Info:

Approval Info:

• Approval Date: 2000-04-17
• Approval Name: Eric Raynor
• Approval Organization: QS
• Approval Phone Number: 202-358-4738