This article is from the 2016 NESC Technical Update.
Dealing with Software Complexity
Static program analysis is a critical part of software assurance and is performed to discover specific types of coding defects (commonly referred to as bugs) and security issues without actually executing the program.
Numerous commercial and open source tools have been developed to automate manual static analysis, a significant improvement over manual code inspections, which were limited to about 300 lines of code per inspector per day. These manual inspections are impractical for programs like the Orion Multi-Purpose Crew Vehicle, for example, which incorporates more than two million lines of source code.
Automated open source code checkers use numerous heuristics to inspect the code for issues of coding standard violations, variable assignments, divide by zero possibilities, questionable syntax, consistency issues, complexity measures, unchecked input values, and numerous other well-known defects that historically have caused failures. The checkers can report defects that may impact code maintenance or defects that produce security flaws. The National Security Agency maintains a suite of code (Juliette Test Suite) that includes examples of historical errors. When the Juliette Test Suite is analyzed by a static analysis tool, the results indicate how effective the tool is at detecting these historical errors.
Applying Static Code Analysis for Improved Software Assurance
“The NESC was requested to perform a static code analysis of safety-critical software used to automate key aspects of launch vehicle range safety,” said Michael Aguilar, NASA Technical Fellow for Software.
“We formed a multi-Center team that included key personnel from NASA’s Independent Verification and Validation (IV&V) Facility, ARC, and JPL to initiate a static code analysis of the NASA Autonomous Flight Termination System (AFTS). Our objective was to provide extensive implementation analysis of the source code and related AFTS support tools.”
The NESC team performed an initial static analysis of the AFTS code in February 2016 that included 10 automated analysis tools (CLANG, Coverity, Codesonar, CppCheck, Fortify, Polyspace, Semmle, Understand, IKOS, and SeaHorn). Some AFTS operations support tools, written in ADA, were also analyzed.
“When static analysis tools are run, we find they have very little overlap. They find and miss different defects. Running several tools allows for better results,” Mr. Aguilar said.
Improving Access to Static Analysis Tools
During the assessment, the NESC team investigated the use of a Software Assurance Market Place (SWAMP*), a portal that enables software developers and researchers to access multiple tools to perform static code analysis. “The bare-bones SWAMP uses free and open source tools. Comparing the tools, we found a combined set of free and open source outputs from several tools produced very good results,” Mr. Aguilar said. Key to the usage of static analyzers is identifying a core set of important defects that affect both operation and security — a “must fix” set of defects that could become a future software implementation standard.
Leveraging this assessment experience, the NASA IV&V Facility is currently developing a NASA SWAMP that would allow software projects access to analyze source code Agency-wide, behind the NASA firewall. “We plan on releasing SWAMP configured with these free and open source tools. SWAMP can be reconfigured to include commercial tools the project has licenses for,” Mr. Aguilar stated.
“We would like to implement two flavors. The first would be “SWAMP-in-a-Box” that installs on the software developers infrastructure to enable developers to access the static code tools that are included with the SWAMP. We also envision a NASA SWAMP portal, potentially hosted on the NASA Engineering Network, which would be accessible to all NASA code developers. By implementing an Agency-wide access to static analysis, many more NASA software projects will be able to run static analysis on their developed source code.”