“Compromises such as those of VoIP provider 3CX highlight the gaps in software supply chain security – and the need for a new approach to supply chain risk management”
By Charlie Jones, Director of Product Management, ReversingLabs
Given the ubiquitous nature of software in today’s digital age, nearly every large enterprise maintains an internal software development program, while simultaneously consuming third party commercial off-the shelf (COTS) software to operate their business. Regardless of what party develops the software, a large portion of it will consist of externally maintained open source components. In fact, Synopsys estimates that 96% of code bases in 2023 contain at least one open source component. This explosion of external dependencies that modern software development practices introduce creates an attack surface that is difficult to quantify, track, and control.
3CX and Software Supply Chain: An Emerging Threat
The March 2023 software supply chain attack on 3CX is a great example of how organizations struggle to manage security risks of both internally developed and third party software. According to an account of that incident by Mandiant, the cybersecurity firm contracted to lead 3CX’s incident response efforts, the initial attack vector leading to the compromise of 3CX was a malicious COTS software installation package (X_Trader from Trading Technologies) that was downloaded to an employee’s PC. The tainted third party software provided a backdoor into 3CX’s network that was leveraged by malicious actors to move laterally and compromise the company’s build environment. Ultimately, they implanted malware into the company’s flagship desktop phone software. This Trojanized product was then published to thousands of unsuspecting downstream consumers, providing attackers a back door into those environments, as well.
These types of cascading attacks on the software supply chain are not going away anytime soon. In fact, The European Union Agency for Cybersecurity (ENISA) predicts “Supply chain compromise on software dependencies” to be the #1 emerging threat by 2030.
With the volume and diversity of attacks on software supply chains growing, organizations need a mechanism to protect themselves that supports the breadth of custom-developed, open source, and closed source components that make up their software estate. Industry frameworks, like SLSA, provide a foundation for organizations to establish a core set of terminology and models to describe what needs protecting.
Existing Controls and Technologies: Not Fit for Purpose
Application security testing (AST) is not a new concept, and there is no shortage of tools available to identify, mitigate, and manage risk throughout the software development lifecycle. However, technologies that support this capability are often targeted toward software developers. That especially presents an obstacle to consumers of closed source commercial off-the-shelf (COTS) software, where source code is often not made available for analysis. Even for consumers of OSS COTS, it may be challenging to find the corresponding source code or be certain that its source corresponds to the executable code that is used.
Today, security teams that must assess the risks posed by COTS software (open and closed source) are greatly limited in the options that they have. Further, the tools and technologies they do have at their disposal were not built to address the risks posed by modern software applications, which tend to be large, dynamic, and complex. This begs the question: ’how can organizations apply a consistent methodology to evaluate the security risk presented by both internally developed and externally procured software?’
One way is through greater use of static binary analysis.
Static Binary Analysis: A Consistent Methodology for Developers and Consumers
Static binary analysis is the process of examining a compiled program without executing it. Files are not executed, hence a detailed analysis may be performed in an efficient and cost-effective manner. It may take steps such as recursively unpacking software binary files, extracting metadata from embedded objects (e.g. executables, libraries, icons), and identifying internal static threat indicators.
Using this method, organizations can deconstruct and analyze compiled open source and closed source software packages without constraints like requiring access to its source code, its software build environment, the cooperation of the software vendor (other than perhaps gaining legal permission to do this analysis), or the need to engage in manual testing.
When static binary analysis is enriched with additional security capabilities, such as threat intelligence and file reputation services, organizations can derive further security testing benefits that are critical to detecting and preventing software supply chain attacks. The combination of these capabilities also make it possible to identify sophisticated software tampering attacks like SolarWinds SUNBURST and the 3CX hack by identifying and analyzing suspicious behaviors introduced between subsequent release/update packages.
Finally, this comprehensive testing method can help development organizations tamp down on some common, unforced security lapses like secrets (e.g. credentials, private keys and access tokens) exposed in published source code that can be leveraged by an attacker to gain unauthorized access required to carry out an attack (e.g. CircleCI and CodeCov). If secrets are embedded in the binaries, a static binary analysis tool might be able to detect them.
Taking Action: Establishing Control Stage Gates to Prevent Attacks from Materializing
Finding the right level of oversight and control enforcement is an age-old issue for security practitioners, who desire to be viewed as value creators rather than business inhibitors. That requires security functions to provide security assurance without overly invasive and frequent evaluation mechanisms that can cripple lean operations and slow the time to market for new business opportunities.
Within the context of the software development and consumption lifecycles, performing static binary analysis of COTS software packages and subsequent releases (patches, hotfixes, etc.) prior to deployment can rapidly detect certain kinds of vulnerabilities without significantly slowing operations.
Static binary analysis is best used as a “Final Exam” for fully compiled software packages that can establish trust that no additional components, dependencies, or digital objects (e.g. installation/driver files) will be appended to the package before distribution or acquisition which may introduce unforeseen risks.
For publishers, this means performing static binary analysis of the final compiled artifact prior to publishing to downstream customers. If publishers have a desire to “shift left” and implement this control earlier in the SDLC process, they can also enforce binary analysis on any externally managed component or dependency (e.g. open source library from NPM) prior to committing it to the larger build. However, they would still need to analyze the final compiled artifact, as the process of building it might introduce a problem not detected in earlier enforcement.
Finally, where business appetite exists and skilled resources are readily available, organizations can pursue more sophisticated manual threat hunting efforts when targeting specific attack techniques. There are a number of open source tools that can support these activities such as Ghidra, a software binary reverse engineering solution for analyzing compiled code that was developed and released for public use by the National Security Agency (NSA) of the United States.
Other Approaches: Dynamic Behavioral Analysis, Protected Builds, and Reproducible Builds
Static binary analysis is not the only way to detect vulnerabilities and malicious code in compiled binaries. A different approach, called dynamic behavioral analysis, is to run the program and attempt to detect behaviors that suggest unintentional vulnerabilities and malicious behavior. However, dynamic behavioral analysis can require significant resources to gain non-trivial coverage, because it has to run for a period of time. Also, its effectiveness depends on the inputs it is given and the behaviors that the system can detect. Static binary analysis requires fewer resources, and when more resources are available, the approaches can be combined.
All projects should protect their build processes; improving those protections can prevent many other problems. However, while open source software projects can demonstrate their protections, it’s harder for closed source projects to clearly explain the protections they undergo. Many open source software projects can be built as reproducible builds (that is, where it’s possible to demonstrate that the binary is generated from the putative source code). Independently verifying reproducible builds can counter many build problems, but this approach requires the source code; this is available for open source software projects but is often not available to consumers of closed source software. Even for open source software, changes may be required to implement reproducible builds.
In Summary
Incidents like the 3CX hack are proof that “business as usual” in the application security testing space is not sufficient. If organizations want to keep pace with an evolving threat landscape, a new approach to risk management is needed that supports modern software supply chain needs. Binary analysis helps fill this gap by providing organizations with a method for analyzing different software types in a consistent and repeatable manner.
About the Author
Charlie is currently a Director of Product Management and subject matter expert in supply chain security. Formerly a consultant at PwC, Charlie has 10 years experience delivering strategic transformation initiatives, specializing in cyber security, third-party risk management, and IT audit programs for Fortune and FTSE 100 companies across all three lines of defense.
An active member of the global cyber security community, Charlie regularly publishes thought leadership, participates in industry working groups, and helps shape international cyber security standards through his position on the Technical Advisory Panel for the UK Cyber Security Council.
Recently honored with the prestigious CSO 30 Award, Charlie is recognized as a top cyber security leader in the UK, demonstrating outstanding business value, innovation, and contributions to the wider cybersecurity community. A seasoned speaker and thought leader in the digital trust domain, Charlie leverages his global consulting experience to offer pragmatic solutions to uplift the cyber security posture of small, medium, and large businesses.