In response to the Columbia accident in 2003, then NASA Administrator Mr. Sean O’Keefe established the NESC. Its first Director, Mr. Ralph Roe, with support from the Chief Safety and Mission Assurance Officer and former astronaut Mr. Bryan O’Connor, organized the NESC with a specific mission: provide independent assessments of NASA’s toughest technical issues while striving for a culture of engineering excellence. After 15 years and more than 850 assessments, the five remaining original members reflected on the NESC’s efforts to implement an operations model that would help return the Shuttle to flight and address the complex issues that would come with a new era of spacecraft development.
Testing the NESC Model
In November 2003, the newly-formed NESC assembled its first assessment team to address concerns with the propulsion bus design for the CALIPSO spacecraft scheduled to launch in early 2006. The satellite’s propulsion bus had been manufactured using mechanical fittings to contain highly toxic propulsion fluids rather than the welded systems typically used by NASA. The NESC was asked to review the bus design and assess the risk of propellant leakage.
“Our approach was to lay out a fault tree that showed everything that could possibly go wrong and determine if the appropriate controls were in place during the build process to ensure those fittings wouldn’t leak,” said NESC Director Mr. Tim Wilson. The assessment provided the data NASA and the CALIPSO Program needed to determine the mission’s safety for flight. It also served as the first test run of the NESC operations model – an institutionalized tiger team approach where quickly assembled teams from a ready pool of NASA technical experts are set to task on a problem.
“We put our team together, laid out the problem, and tackled it,” said Mr. Wilson, who at that time had just transitioned from more than 15 years with the Shuttle Program to become the NESC’s first Chief Engineer at KSC. When offered the NESC position, Mr. Wilson felt it was an opportunity to do something different and challenging, and in the wake of Columbia, a way to move forward. “It was a positive response to the Columbia failure,” he said, “and I hoped my participation might help ensure such a thing would not happen again.”
Working to Establish Credibility
During the first four years, a little more than 50% of NESC assessments focused on the Shuttle Program and the International Space Station (ISS). They ranged from analyzing recurring Shuttle flight anomalies and evaluating hail strike damage on external tank foam to investigating reduced adherence between the protective coating on the orbiter wing’s leading edge and the underlying substrate.
“It was high visibility and high profile work, and the answer was not obvious,” said Mr. Wilson of the adherence issue, which carried implications for the Shuttle and the safety of its crew, especially after Columbia. “It took a tremendous amount of work and a dedicated team to plow through all of the data and make sense of it.” The assessment led to improved nondestructive evaluation (NDE) inspection methods and is one Mr. Wilson felt made significant contributions to the program.
Those early assessments were challenging for more than just their technical complexity. Staffing assessment teams “meant prying resources away from other programs and Centers to be a part of our team,” he said. “That was a hard sell.” Programs weren’t fond of the idea of sending their top talent to work at a new organization, even though the positions were meant to be temporary, no more than 5 years. And once assembled, a team of recognized senior discipline experts did not necessarily lend the NESC credibility overnight. “Programs didn’t really know what we were about and weren’t quick to trust us because we didn’t have a track record.”
That was especially true with the Shuttle Program. “The perception was that you couldn’t bring in someone from outside the program and engage them in a problem – that there just wasn’t time to come up to speed and understand all of the subtleties of Shuttle systems. Fortunately, when we brought in our then discipline expert for power and avionics and plugged him into a major Shuttle issue, he was able to very quickly give them useful, value-added answers.” That went a long way to gaining the confidence of the Shuttle Program, he said.
In tandem, the NESC was working out its internal logistics. Bringing together a NASA-wide group of engineers for the NESC was a lesson learned in cross-culturalism, said Dr. Michael Gilbert, who started as the NESC Chief Engineer for LaRC and is now a Principal Engineer. “We were learning quite a bit in those days about who we were and how we were going to operate.” During an assessment to address failure risks with Shuttle’s primary reaction jet drivers, engineers from both human and robotic spaceflight intensely debated the issues. “That’s when many of us realized that significant cultural differences existed between the human and robotic sides of NASA engineering.” There was a learning curve to climb as everyone worked to understand each other’s engineering approach and how they characterized the risks involved.
As the NESC took on more assessments, there were occasions when NESC positions were contrary to a program’s position on launch readiness or could impact cost or schedule. Though uncommon, it showed the NESC model, with its independent reporting path to NASA’s highest technical leadership, was working. NESC positions could be elevated to the Office of the Chief Engineer and ultimately, the NASA Administrator. “When you read the Columbia or Challenger accident reports, you find that upper level managers who were making the final launch decisions were unaware of technical disconnects down the line,” said Dr. Gilbert. For him, these instances made the NESC contribution to the Agency crystal clear. “It’s not that the NESC has the final say. It’s that we make sure the decisions that are made are fully risk-informed.”
Continuing the Mission After Shuttle
When the Space Shuttle retired in 2011, the NESC was already engaged in assessments for the newly-formed Commercial Crew Program (CCP), which was working with companies such as Boeing, SpaceX, and Sierra Nevada on the development of vehicles to transport cargo and crew to the ISS (see graph on right). The NESC also undertook initiatives that would ultimately benefit NASA’s newest exploration spacecraft, Orion Multi-Purpose Crew Vehicle (MPCV), such as designing and building a composite crew module to gain knowledge in composite construction and evaluating alternate launch abort system designs. The NESC also led efforts to update Apollo-era shell buckling knockdown factors, used today by the Space Launch System (SLS) to lighten the launch vehicle and reduce material costs.
Relationships built during Shuttle and ISS assessments had laid a foundation of trust. Former NESC members were also fostering credibility. “There have been a number of people who came to the NESC and then moved on to other positions at NASA,” said Mr. Clinton Cragg, an NESC Principal Engineer. “They knew our capabilities and would come to us because they knew we could help.”
A retired U.S. Navy submarine commander, Mr. Cragg’s first NESC assessment was to provide flight rationale following the repair of cracks found in the Shuttle’s main engine gimbal joint flow liners. New to the NESC and NASA, he worked with a veteran NASA engineer to help him “learn the ropes.” He has led many NESC assessments since, including several for stakeholders outside of NASA, such as the Air Force and Navy in support of issues with their fighter aircraft fleets and the country of Chile with their rescue of 33 trapped miners.
“I can get the relevant expertise, whether from NASA, industry, or academia, and put together a highly-qualified team in no time at all. That’s a big strength of the NESC. And because the teams are small and concise, you can change on a dime the direction you want to take with an assessment so you can get to the truth quicker.”
Since her start at the NESC, Dr. Cynthia Null’s NESC assessment work has involved the interaction of humans with aircraft systems, full-scale engine test stands, flight-test range systems, spacecraft systems, and spacecraft ground systems. As the NASA Technical Fellow for Human Factors, Dr. Null has watched those systems become more complex and challenging for operators.
“The hardest engineering problems are at interfaces: between subsystems and between operations, environments, software, and people. Working challenging technical problems with a systems engineering approach enables NESC multi-discipline teams to increase technical understanding, identify risk, and provide alternative approaches,” she said. “I have learned a great deal from my NESC colleagues and the larger community of scientists and engineers who provide critical expertise to our work. I am grateful that I’ve been able to lend a hand.”
Adapting the NESC Model to NASA’s Evolving Mission
While its operating model has changed very little over the years, the NESC has adjusted its composition periodically to stay aligned with the Agency’s programs. Previously joined disciplines such as Electrical Power and Avionics were separated to address the amount and scope of work coming in and additional disciplines were added to cover emerging work areas such as cryogenics, space environments, and systems engineering. “We target issues that have the most payback for the Agency as a whole and crosscut multiple programs,” said Mr. Wilson. “And we try to ensure our work lines up with the NASA Chief Engineer’s priorities.”
NESC expertise has also evolved to support new spacecraft and emerging technology. Mr. Hank Rotter, the NASA Technical Fellow for Environmental Control and Life Support (ECLS), was already a NASA veteran when he joined the NESC, bringing ECLS experience amassed since 1963. His early assessments addressed issues such as coolant pump failures on the ISS Extravehicular Mobility Unit. But over the past 15 years, his work has included assessments for the ISS, Mars Science Lab Rover, Solar Probe Plus, Europa Clipper, Orion, and for life support systems on the Air Force F-22 and Navy F/A-18 aircraft. The opportunities to apply his skills and continue learning have kept him at the NESC. “I stayed because of these challenges,” said Mr. Rotter, “and because the types of assessments we work make a difference.”
Where Mr. Wilson feels the NESC has most evolved is in how it connects with its stakeholders. “The way we engage is more collaborative than before. We’ve learned that it helps us give them better solutions.” By design, the NESC is independent of programs and projects so it can maintain objectivity, but keeping the programs and projects involved offers them ownership in the process. “This improves communication and makes it more likely they will act on our recommendations,” Mr. Wilson said.
As new human spaceflight projects come online, the NESC will again work real time problems as it did with Shuttle. “We have to continue to focus on the right things and deliver high-value products and not become complacent in the work we do. Staying vigilant, doing good work, and communicating our results is important,” said Mr. Wilson. “I will not tell you that we are perfect, but I think we add value where we engage, and we’ve built a reputation for being able to help.”