By Daniel Nebenzahl, CTO of Scribe Security
The use of Software Bill of Materials (SBOMs) is becoming increasingly essential in managing software supply chains. Regulations like the NIST Secure Software Development Framework (SSDF) are rolling out globally, and the demand for SBOMs is growing. Commercial and open-source SBOM creation tools are now widely available, and open-source package managers are adding SBOM generation features. However, the talk around SBOMs mostly revolves around their generation and less about their consumption. The main consumption use case is for evaluating dependencies known-vulnerabilities risk, by mapping the dependencies listed in the SBOM to CVEs (Common Vulnerability Enumeration – the standard of documenting and publishing vulnerabilities). In this blog post, we propose using SBOMs alongside the Open Source Security Foundation (OpenSSF) Scorecard to evaluate a product’s risk.
OpenSSF Scorecard and Rules
The OpenSSF Scorecard is a tool for assessing the trustworthiness of open-source projects based on a checklist of rules. The evaluation provides both a final score and a score for each check, allowing Scorecard users to create their evaluation criteria. The typical use case of the OpenSSF Scorecard is to enable developers to take security risks into account as part of an open-source package-selection process.
One rule, for example, is the “Binary Artifacts Rule”; one of the things that make open-source trustworthy is its deliverance as source-code. The source code is transparent and I, as a consumer, can peek into the code and see that it is not malicious. If it is a popular enough project – probably I can save my resources and rely on related publications by other consumers. On the other hand, open-source software that contains binary-artifacts is less transparent; the user of this software does not really know what is running. It may pose a risk. That is the reason the “Binary Artifact Rule” is ranked as “high risk” (see check documentation). A 10 score means that no binary artifacts were detected, while 0 means that 10 or more were detected.
In the same manner we use an SBOM to generate a CVE list and evaluate the vulnerabilities of dependencies of a software artifact, we can use an SBOM to generate OpenSSF Scorecard evaluation of the open source projects used in this product.
As a use case, let’s experiment with a simple policy: Deny use of products that include at least one open-source dependency that scored 0 in the Binary Artifacts check.
Experimenting with SBOMs and Scorecard
In order to evaluate this idea, let’s experiment with two open-source products. We will create an SBOM for these products, and using the SBOM – generate a Scorecard report for each open-source dependency in the SBOM. Such a report can be generated using the OpenSSF Scorecard publicly available dataset, accessible through Google Big-Query.
The projects we choose were:
1. Kibana – a very popular front-end app to visualize Elasticsearch data, written in javascript.
2. Antrea – a very popular K8s networking solution, written in Golang.
Following is a comparison of the OpenSSF Scorecard dataset of these projects:
| Project | Total dependencies | Covered | Not Covered |
| Kibana | 3447 | 2616 | 831 (24%) |
| Antrea | 176 | 157 | 16 (9%) |
The fact that github is the default package location for go-lang projects may be the reason for the OpenSSF Scorecard’s higher coverage of Antrea.
| Project | Number of open source dependencies that failed the “Binary Artifacts” check (scored 0) | Policy Decision |
| Kibana | 2 | Investigate |
| Antrea | 0 | Pass |
In terms of alert-fatigue, this is a manageable result; the Kibana presents us with only 2 dependencies to investigate while Antrea passes.
Lets deep dive into the Scorecard results of the Kibana suspected dependencies.
| Dependency | Description | Aggregated Score | Policy Decision |
| webpack | Webpack is a module bundler. Its main purpose is to bundle JavaScript files for usage in a browser, yet it is also capable of transforming, bundling, or packaging just about any resource or asset. | 7.8 | Pass |
| nwsapi | NWSAPI is … a right-to-left selector parser and compiler written in pure Javascript with no external dependencies… it has become a popular replacement of the native CSS selection and matching functionality in newer browsers and headless environments. | 4 | Further investigate |
Given the webpack’s high score (it is in the top 5% scored packages), we decide not to further investigate it. For a sanity check, when looking up the webpack project binary artifacts we find that they are merely testing executables.
Further investigating nwsapi’s github repo we find that indeed there are binary artifacts but these are used for building the package and are not run as part of the package. Thus we conclude that the nwsapy package passes too, and approve the use of the Kibana product.
Conclusion
In this post we have demonstrated how applying the OpenSSF Scorecard to product SBOMs can assist in evaluating product-level risks. We have evaluated two products’ compliance to a policy rule, and further analyzed the noncompliance issues. This proof-of-concept demonstrates the potential for utilizing SBOMs with OpenSSF Scorecard data to evaluate product-level security risks.
About the Author
Daniel Nebenzahl is a Co-Founder and CTO at Scribe Security – End to End Software Supply Chain Security. Daniel’s background includes development of cybersecurity solutions, performing and leading cybersecurity research teams. Daniel can be reached here: Email, LinkedIn