🎙️ Submit your talk for: OpenSSF Community Day Europe by July 12

Tag

Software Supply Chain

Jun 02
Love0

What’s in the SOSS? Podcast #62 – S3E14 The Ghost in the Dependency Tree: Navigating Open Source End-of-Life with HeroDevs

By Podcast

Summary

In this episode of What’s in the SOSS, host CRob sits down with Isaac Wuest, Product Line Leader at HeroDevs, to explore the critical and often overlooked “gray area” of the software supply chain: End-of-Life (EOL) software. While the industry heavily relies on CVEs to track vulnerabilities, Isaac explains how maintainer abandonment creates a vacuum where risks are present but remain undiscovered and unreported. From the origins of HeroDevs supporting AngularJS to the nuances of the EU Cyber Resilience Act (CRA), this conversation provides a practical framework for distinguishing between inherent hazards and actual risk in your dependency tree.

Conversation Highlights

00:04 Host CRob welcomes Isaac Wuest from HeroDevs to discuss secure open source ecosystems.
00:45 The HeroDevs origin story: How Google sunsetting AngularJS created a need for secure drop-in replacements.
02:44 Isaac’s path to open source: Transitioning from product management to supporting maintainers.
04:06 Exploring the “Gap” in CVEs: Why dictionary-based vulnerability tracking misses EOL and malicious packages.
07:03 The challenge of “Maintainer Attestation”: Why most open source projects lack a formal EOL calendar.
09:52 Compliance and Risks: How EOL dependencies create blank spots for security professionals and auditors.
11:27 The Shark in the Tank: Using a food regulation analogy to differentiate between hazard and risk.
13:22 Navigating the EU Cyber Resilience Act: Preparing for increased manufacturer accountability in software.
14:08 Maintainer Abandonment: Identifying the moment a project stops receiving patches without formal notice.
16:14 Scanning for Gaps: Why standard industry tools currently struggle to provide a complete EOL picture.
18:49 Practical Remediation: Recommendations for researching upgrade paths using tools like endoflife.date.
20:49 Analyzing SBOMs: How engineers can leverage free datasets to identify and fix deep dependency risks.
23:00 Rapid Fire: Coffee, Star Wars, spicy food, and the favorite apocalyptic robot.
25:01 Final Thoughts: A call to action for educating yourself on your application’s EOL exposure.

Transcript

CRob (00:04.053)
Welcome, welcome, welcome to What’s in the SOSS, the OpenSSF’s podcast where I talk to people that are in, around, and creating this amazing open source ecosystem that we all benefit from. Today, we have a pretty interesting guest with some pretty cool and timely topic. We have Isaac from Hero Devs. Isaac, welcome to the show.

Isaac Wuest (they/them) (00:26.474)
Awesome, thanks for having me. I’m glad to be here.

CRob (00:29.197)
Yeah, so everybody that listens to our podcast may or may not know the HeroDev story. Could you maybe explain a little bit about your company that you’re representing and then maybe give us a little peek into your open source origin story? Like what got you into this space?

Isaac Wuest (they/them) (00:45.694)
Absolutely, absolutely. So the story of Herodevs actually goes back to the story of AngularJS and when Google decided to sunset support for Angular. So Herodevs as a company started a few folks that were on the Angular team, realized that Google was sunsetting support and that there would be breaking changes and people would need a secure version that was supported and being patched in the future.

CRob (00:52.256)
Mmm.

Isaac Wuest (they/them) (01:11.754)
And that was the birth of HeroDevs. HeroDevs as a company has expanded out of AngularJS to many other types of large open source frameworks where we offer secure drop-in replacements for the end-of-life versions. And that was kind of the inception of the company. What we do today is keep those versions secure. That being said, what we’re now doing and kind of moving more into is that end-of-life space more broadly, saying how can we support not just individual frameworks,

But when things go end of life, how can we make sure that there are secure versions available across a deeper range down in the dependency tree for companies?

CRob (01:52.545)
Makes a lot of sense. I just had the opportunity to read the recent Sonotype stated supply chain report. And I think the new number of dependencies for commercial applications is around 1,100 on average. So this is a pretty, pretty big issue, huh?

Isaac Wuest (they/them) (01:59.79)
Mm.

Isaac Wuest (they/them) (02:06.754)
Yep, yep.

Isaac Wuest (they/them) (02:11.178)
It really is. We actually partnered with Sonatype on some sections of that report there came from some of our data. you’re absolutely right. It’s a growing issue, both because the velocity of open source ecosystems, like the rate of new packages and new versions, continues to accelerate. And as you know, more more CVEs are getting reported year over year. So there’s this whole volume problem that is, I’m sure we’ll talk more about aspects of that. yeah.

CRob (02:15.585)
Thanks.

CRob (02:31.965)
yeah.

CRob (02:37.109)
We sure will. But before we get to that, how did you get into open source? What brought you to this meeting today?

Isaac Wuest (they/them) (02:44.744)
Absolutely, absolutely. So my interest in open source goes back about three to four years at this point. So a previous company that I worked at, I was good friends with several developers who were in the open source space. And they said, hey, we think you might really enjoy the space and enjoy learning about open source as a product manager. And I said, absolutely, I’d love to. So I first started working in open source in that environment where you’re working directly with maintainers. So I got to know

several hundred maintainers and was like, this is a really, really robust and a really, I love the ethos of the open source community. So I feel like you kind of get a taste of it and you go, wow, this is for me and I’ve been in the open source space ever since. Not as someone doing the hard work that everyone else does, but as a PM thinking, how can I support and help with all the important work everyone else is doing?

CRob (03:38.525)
What I love about open source is not everybody has to be the developer. There’s a room for a lot of different skills. That’s pretty great. Speaking of some different skills, and this is something that a lot of developers sometimes struggle with, let’s talk about security. Broadly, the ecosystem uses a tool called CVE, which used to stand for common vulnerabilities and exposures.

Isaac Wuest (they/them) (03:46.913)
Absolutely.

CRob (04:06.889)
And that was kind of a dictionary of these are the discovered vulnerabilities and this is how you can go learn more to protect yourself. CVS is an interesting methodology in a program. Had a couple challenges over the last few years. And there are also whole categories of things that lay people may see as a problem. Like your.

the end of life you teased or malicious packages. And these are things that aren’t accounted for in the CVE methodology. So from your perspective, what does that gap in the CVE program, what kind of risks or problems does that, not ignoring, but that not accounting for these other types of problems, what does that incur to consumers of open source?

Isaac Wuest (they/them) (04:59.083)
Yeah, yeah, that’s a really good question. And I will say that the answer to that question is still evolving. Now, my angle, like my perspective on it from the end of life side, is that, know, CVEs, of course, rely on this kind of public disclosure where someone discovers it, someone who’s kind of certified, I think they’re called a CNA, this is someone who’s allowed to report CVEs in the standard way, reports them.

And that’s great. Of course, CVEs have their limitations. So we have these other scores that are evolving, your EPSS and the known exploited and that system. But the predicate for all of those are that the vulnerability has been discovered and reported. Now, the problem there is that as packages or versions within packages in the open source space go end of life, meaning they’re no longer getting

security updates from that upstream maintainer, that upstream community. As that occurs, a second thing is happening as well. They’re not going to get patches or updates for CVEs. In addition, they’re typically no longer investigated for vulnerabilities to begin with. So it creates this, we just don’t even know, right? Maybe it’s vulnerable, maybe it’s not. Oftentimes, the maintainers around those packages don’t have the time to…

CRob (06:10.783)
Right.

Isaac Wuest (they/them) (06:21.557)
retroactively investigate every CVE on their entire version range to even see if it applies or not. So that space, those unknowns, create a real concern that the broader security market and the SDA space is trying to figure out how to react to.

CRob (06:39.937)
So from your experiences, especially dealing with upstream maintainers, how much attention does upstream pay to end of life? Is this something like I know commercial vendors generally will advertise we do support between X and Y dates. So from your experiences upstream, what does end of life look like or how much do they advertise that stuff?

Isaac Wuest (they/them) (07:03.373)
Yeah, it’s a good question. it’s extremely variable. So for large frameworks, if we’re talking about your Angular’s, your Views, your Springs, you can often use tools like endoflife.date or go directly to their sites and you’ll see their schedules. When, these are the versions, this is when it goes, LTS for that long-term support. And then here’s when LTS is dropped and that’s when it goes end of life. The problem is that that covers a very thin slice of the total number.

packages that are in open source. Most maintainers don’t have time to publish a calendar for their specific packages. As a result, it’s often very ambiguous. Unless we’re talking about those big frameworks, those big libraries, outside of that, it’s kind of like the maintainers know when they’re going to generally they’ll stop supporting major release lines once it’s two or three in the past.

When you talk with maintainers, what you learn is that it’s kind of ambiguous even for many of them. They’ll say, well, if a CVE is reported, I might be able to release a security patch. I might not. It kind of depends how busy I am. They’re volunteering their time. And that ambiguity makes it really challenging to know, well, how secure is it? Is it supported? That initial problem of are we investigating and reporting and patching risk on those older versions?

CRob (08:30.645)
And so again, from your experiences, how have developers or projects articulated this or do they at all?

Isaac Wuest (they/them) (08:38.871)
Great question. So apart from those large frameworks where they publish them on their sites, most of the time it’s an array of methodology. So sometimes they’ll post in the readme’s on their projects. You’ll see information about what’s being end of life. Status is like deprecated where in certain registries, NPM and others, can mark, a maintainer can mark and say, this release line or this version is deprecated. That’s a bit of a proxy for end of life.

Sometimes though, I’ve seen Twitter or X, like social media, be a primary mode where the maintainers of packages are saying, this is when I’m gonna be no longer supporting version 1.1.1. And it’s highly variable and that makes it really brittle for any large enterprise to rely on.

CRob (09:28.769)
Mm-hmm. Well, hey, let’s switch our focus a little bit. Let’s move a little bit more downstream. So let’s say I’m a security professional at an organization, or I’m a developer that’s working on some type of internal app that’s leveraging open source components. What does it mean to them when some dependency or package that they’re using becomes end of life?

Isaac Wuest (they/them) (09:52.814)
Yeah, that’s a great question. I will say, firstly, they’re still learning what it means in terms of compliance frameworks. So when something goes into life, there’s the security implications that we just talked about. Hey, it’s not going to get security patches, and it likely won’t get reports of vulnerabilities that may exist on it. Now, if I’m a security professional working at a company that

CRob (10:03.139)
yeah, yeah.

Isaac Wuest (they/them) (10:19.915)
You know, we have some number of applications we’re building that use that dependency. I know immediately that there’s this kind of blank spot there where even if risks aren’t being flagged by, you know, my security scanning tool of choice, I know that there may still be risks present. That might be a problem for that own company’s security posture, where they have their own standards and say, Hey, we can’t rely on that. Oftentimes what I see though, is that

The main motivation is the fact that it puts them out of compliance with these frameworks that exist. Where if you’re dealing with health data, or you’re dealing with credit card data, and you’re subject to HIPAA, high trust, PCI, as a security team, you can’t tolerate dependencies that won’t get updates. That’s too much risk. And that’s created a real vacuum in the space of open source end of life.

CRob (11:17.857)
And so from your perspective, if something is end of life, does that automatically mean that package is a problem? What options do people have?

Isaac Wuest (they/them) (11:27.883)
Yeah, that’s a great question. So there’s an analogy I sometimes draw from in the food regulation space. If you ever get into how FDA or the EU thinks about risk in food. there’s a difference between risk and hazard. Hazard is just anything can kind of, anything could be a hazard. If I go to an aquarium, the fact that there’s a shark in a tank is a hazard.

Oh boy, if something were to happen, I fell in that tank, slipped, However, the risk is quite low. security teams within companies are trying to figure out the balance within the context of end of life. Is end of life is inherently hazardous. How much risk does it represent though? And based on the risk, what should we do? How much effort should we as a company invest? We can tolerate some risk, we can’t tolerate all risk.

And then the question becomes, what are my options if I’m an engineer or security person who I discover I have several end of life packages in my dependency tree for some application? What should I do? The first problem they have is, how risky are these packages? And the fact that they’re end of life means, what are my options for fixing it? Do have to migrate? Can I patch it myself? That becomes the natural.

kind of next questions and problems that these folks are facing.

CRob (13:01.087)
And we’re definitely going to see that in the coming months and years as the EU’s Cyber Resilience Act comes online, because manufacturers are held accountable for all components within their products. And they need to do that risk calculus to understand what am I going to do if there is a critical vulnerability in one these things that doesn’t have support potentially upstream.

Isaac Wuest (they/them) (13:22.626)
Yeah.

You’re absolutely right. The landscape around shipping secure software continues to get more more strict, which I think is a very healthy orientation for the industry. It’s important that we’re responsible as technologists for the code we’re shipping, even if that’s open source code. But that doesn’t mean that it’s not going to be a really challenging problem to solve as the EU cybersecurity

actor or many others continue to gain traction.

CRob (13:58.525)
And with your interactions both up and downstream, are people aware of kind of this end of life opportunity that’s before them?

Isaac Wuest (they/them) (14:08.641)
folks are becoming increasingly aware. There’s actually a distinction now in two different types of end of life that’s been emerging when talking about it. there’s end of life, similar to those large frameworks and the calendars I was mentioning, that’s often categorized as maintain or attested end of life. And that attestation is often the very compliance-y legal term, where the maintainers are attesting that it is end of life or that it is supported.

And if I’m an auditor, I can rely on that when I’m auditing some large company’s project. The category that’s emerging as a term is maintainer abandonment when it comes to end of life. Now there’s other terms you’ll hear that are similar. Really all we mean is there’s a point at which that maintainer can no longer continue to support. It could be a specific kind of major release line. They’ve moved on.

It could be even specific miners that they’re no longer going to be supporting. you think about that SIMVR, that 1.1. Or it could be the entire package gets abandoned. So they rename the package, or the maintainer has experienced burnout, and they’re no longer able to support it. They’re going to need to take a step back. No one’s filling that vacuum or that void. That maintainer abandonment category is proving to be a real opportunity.

to say how do we identify and report on this even when the maintainer doesn’t have a way to clearly tell everyone and broadcast like the big players in open source do the large frameworks.

CRob (15:41.439)
Interesting. Thinking about the tools we have today and end of life, have for good or bad, the CVE program, we have software builds materials, we have a whole fleet of scanners, AI or not. We have bug bounty programs. Now, from your perspective, do these tools and maybe ones that I didn’t mention, do they give a consumer

Kind of that full security picture or are there gaps?

Isaac Wuest (they/them) (16:14.625)
Right now, the gaps are really starting to show. Most of the available kind of industry standard tools, be it SBOMs, Cyclone DX or SPDX, even the biggest kind of formats out there, or most scanning tools as well, they’re still focused on publicly reported vulnerabilities as the top priority. And then they do talk about end of life. Most of the end of life that you see reported,

are those maintainer attested schedules, where your big frameworks tend to be tracked well, which is really important to be clear. Those big frameworks often represent a lot of risk and a lot of pain if you have to make a migration or something like that. Very few tools exist today in the security scanning space that have a robust picture, a complete picture of that maintainer abandonment side. There’s people doing kind of innovative work there trying to

and understand, but it’s a pretty open field space right now even for these large security scanning tools.

CRob (17:19.947)
Well, that’s always been an historic challenge, of that information sharing upstream is, you know, is the project active? When was the last commit? How many stars and like all that kind of normal lifecycle stuff helps, but is not that developer attestation saying on this day, I’m stopping this particular stream.

Isaac Wuest (they/them) (17:32.488)
Absolutely.

Isaac Wuest (they/them) (17:42.722)
Yep, yep, you’re absolutely right. And yeah, that’s been a challenge for some time. It’s also often a challenge because these tools need certainty in order to report on a scan. Like you said, they get that complete picture. They wanna be able to say with confidence, it’s this or it’s that. It’s supported or it’s not supported. And when you talk with maintainers, there’s a lot more gray there where they say, well, I could support it if I need to. If someone tells me I’m willing to try and do them a favor, because open source is so…

CRob (18:06.209)
Mm-hmm.

Isaac Wuest (they/them) (18:11.329)
Everyone’s so willing to pitch in and help each other. And that’s a feature of the space, but it becomes a bug for those consumers, those large enterprises that need certainty when they’re reporting on compliance, alignment, and all that sort of stuff.

CRob (18:15.595)
There it is, yep.

CRob (18:26.721)
So I’m thinking about this from the developer perspective and end of life. Are there any practical steps that you would recommend a project to consider or do to help be a little bit more communicative around this to deflect a lot of downstream questions? You know, ideally.

Isaac Wuest (they/them) (18:49.335)
Yeah, yeah, absolutely. There are some practical things that are available today. And then there’s some opportunities where I’m really hoping as an industry, security scanning focuses on solving this problem more broadly. Today, though, I the way it’s typically done when I talk with enterprises and work with developers on the front lines is they usually get a notice that something has a CVE on it. And then,

I mean, I’m sure you know well that they end up going and researching. You grab that package name and the version, and you go to Google, and you start looking at the registries. And you say, is this supported? If there’s not an obvious patch version I can just upgrade to, Like oftentimes, engineers stumble upon the reality of it being end of life if it’s not that large framework where there’s a clear calendar. That’s, of course, painful because it means

Do I need to migrate? What do I have to find a replacement, et cetera? So my recommendation today is there’s tools like endoflife.date. That’s a great tool, site and project out there. There’s other ways to go research. However, there are some free tools available that you can load an SBOM or a manifest into. There’s one that we’ve been working on called the eoldataset.com that you can.

Loads free tool that has a large database detecting maintainer abandonment. I’m hopeful many other organizations work on this problem so that engineers don’t have to spend far too much time researching everything one off and get some confident data about what is almost certainly end of life and what is likely supported even for that long tail of packages.

CRob (20:31.329)
That’s good advice. if from your perspective, how accessible, how easy are these systems to kind of get in there and get the data you need so you can make your risk decision? Do I want to continue with this particular branch or not?

Isaac Wuest (they/them) (20:49.293)
Yeah, so if for existing SEA tools, most of them, of course, they’re going to tell you about that maintainer attested. And if you load in your SBOM, there’s usually some other version or remediation path where they say, hey, here’s what you can do about it. There is a supported version. And at that point, it becomes a question of how quickly do you need to migrate? Or can you get some sort of support right now? For the other side of it.

when it comes to maintainer abandonment. I would definitely recommend using the tool I just mentioned where you can load in that SBOM. And once you get that full picture of, here’s all those other packages that are end of life or likely end of life, the next step becomes, OK, well, are there supported versions that I can upgrade or migrate to? Most of them there are. Then it becomes a question of, of course, breaking changes.

How challenging is that? Is it deep in the dependency graph such that the version is constrained and it becomes this kind of gotcha problem? And oftentimes, even knowing something is end of life, that’s not enough for engineers. They need to know what are their upgrade paths if they have to fix that one package, that one version. Our tool that I mentioned that’s free to use has some of those, but many others.

do as well. Your SEA tools often do contain migration options and engineers can often use those tools. They’ll load in data sets like ours or others about end-of-life data that then connects to the dependency graphs and says here are your migration options. You’d have to upgrade these two direct dependencies. That gets you into the versions you need that are no longer end-of-life deeper down.

CRob (22:42.657)
This is an amazing topic that I think not everyone is aware of it as they should be. So thank you for joining us today and kind of sharing a little bit about end of life. But let’s move on to the rapid fire part of our talk.

Isaac Wuest (they/them) (22:57.357)
you

CRob (23:00.405)
That’s spicy. I have a handful of questions that we’ll ask you. Just give me the first thing that comes off the top of your head. What’s your favorite beverage?

Isaac Wuest (they/them) (23:09.441)
Got it.

Ooh, I’m gonna have to go with a cappuccino. I love coffee, but I love espresso, so give me that. Give me that right there. Yes, I need that caffeine.

CRob (23:14.593)
Ooooo

CRob (23:20.193)
Yum. All right. Star Trek or Star Wars?

Isaac Wuest (they/them) (23:24.885)
I’m so sorry, but it’s gonna be Star Wars for me. It’s gonna be Star Wars. Okay, good, good, good. Well, I appreciate Star Trek. I certainly love it, but Star Wars, there’s just something about it that speaks to me.

CRob (23:27.723)
There are no wrong answers, but Star Wars is a pretty good one.

Well, do you prefer spicy or normal food?

Isaac Wuest (they/them) (23:42.024)
I love spicy. Please.

CRob (23:43.751)
ooo that’s spicy

Isaac Wuest (they/them) (23:47.853)
Yeah, I don’t understand people that don’t like spicy food. It’s just more exciting. It’s more interesting.

CRob (23:49.249)
Thank

Right? Exactly. And then, you being that we are in the a new age here, who’s your favorite apocalyptic robot?

Isaac Wuest (they/them) (24:06.029)
Ooh, my favorite apocalyptic. is this, ooh, is this existing AIs or is this fictional robots? my goodness. I think I could have one of each. could have one of each. Okay, for fictional, I’ve been reading all these dystopian books that involve robots. I love Heinlein’s The Moon is a Harsh Mistress. There’s an AI there, Mike or Michelle, not a robot that causes an apocalypse, but one that kind of helps solve kind of a crisis.

CRob (24:13.622)
Sure.

You could have one of each.

Isaac Wuest (they/them) (24:35.373)
So that’d be my favorite fictional. Ooh, for existing, I don’t know. There’s a number of AI models I’m pretty scared of right now. We’ll see. And I don’t want to badmouth any, because maybe in two or three years, I’ll be hoping that they take care of me.

CRob (24:35.489)
Nice.

CRob (24:43.297)
You

CRob (24:50.241)
That’s our dream. Well, thank you for playing along, Isaac, being a good sport. And as we wrap up, do you have any closing thoughts or a call to action for our listeners?

Isaac Wuest (they/them) (25:01.161)
Absolutely. Well, thank you so much for bringing me on. I really appreciated the conversation. This was really exciting and interesting. What I would definitely say is, for anyone listening, if you’re worried about End of Life and the potential exposure that any of your applications have, feel free to check out what we’ve been working on. We have a free tool with a data set called eoldataset.com. And we’re looking just to give away as much data that we have about the End of Life space. So if this is something that was interesting to you and you want tolearn more about what might be end of life, or you want to contribute thoughts and ideas about how to improve the data set, please let us know. We’re very interested. yeah, with that, that’s my main…

CRob (25:43.189)
Well, Isaac from HeroDevs, thank you for joining us today. And I want everyone to kind of listen to this, maybe take some action, do a little research to educate yourselves. And I’d like everyone to stay cyber safe and sound out there. Happy open source and folks. That’s a wrap.

May 29
Love0

Aligning on Machine-Readable Signals as the Foundation for Due Diligence

By Blog, EU Cyber Resilience Act

By Madalin Neag, EU Policy Advisor, OpenSSF

Introduction

The software supply chain has reached a level of complexity where manual oversight is no longer a viable strategy for security or regulatory compliance. Modern systems depend on vast, rapidly evolving networks of components, making manual, paper-based approaches to due diligence impractical. Machine-readable, continuously generated security signals are therefore the only realistic way to support Cyber Resilience Act (CRA) due diligence at scale. These signals already exist across open source ecosystems as a natural byproduct of standard development practices, though they remain fragmented across tools, repositories, and pipelines. Crucially, these signals are best understood as mechanisms for transparency rather than assurance: they expose observable characteristics of software development and operational behavior without constituting guarantees, certifications, or transfers of liability. 

This shift is driven by a need for technical accuracy. Static documentation and point-in-time attestations cannot reflect continuously evolving software systems. This limitation has been underscored by recent U.S. enforcement actions, such as Department of Justice settlements involving inaccurate cybersecurity compliance certifications, which highlight how formal attestations can later be treated as misleading when they diverge from actual system behavior, significantly increasing legal exposure.

Established approaches such as continuous compliance, evidence-based assurance, and secure-by-design all rely on the same principle: replacing subjective, point-in-time claims with dynamic, verifiable proof, automated data that reflects actual system behavior.

Within this model, roles remain clearly separated. Upstream open source projects may choose to publish security-relevant signals in machine-readable formats, while manufacturers, who bear the legal responsibility under the CRA, consume and interpret this information as part of their due diligence processes. This preserves the foundational “no warranties, no liabilities” principle of open source. Participation from upstream remains strictly voluntary and must not introduce legal obligations, certification expectations, or shifting of compliance risk and liability to the project community, ensuring that ecosystem sustainability is maintained while enabling effective downstream risk management.

This discussion builds on earlier reflections on voluntary attestation models under the Cyber Resilience Act, particularly Article 25, which enables voluntary security attestation programmes to support manufacturer due diligence for products incorporating free and open source software while preserving the separation between upstream development and downstream regulatory responsibility. From a systems perspective, however, Article 25 also exposes an important limitation of paper-based attestation approaches. Static, human-authored representations of security struggle to remain accurate within environments defined by continuous change, where rapidly evolving components and deeply nested dependencies can quickly render point-in-time attestations incomplete or outdated. This leads to a broader architectural observation: effective due diligence at scale increasingly shifts away from narrative declarations toward machine-readable, continuously updated security signals embedded directly within development and release workflows. In this framing, voluntary attestation is valuable not as a mechanism for upstream certification, but as a way to enable structured, interoperable security data that downstream systems can automatically consume and evaluate. Machine-readable signals thus become the practical substrate for operationalizing the intent of Article 25 in complex software ecosystems, preserving voluntariness, avoiding any conflation of transparency with assurance, and enabling evidence-based due diligence aligned with the dynamic nature of modern software systems.  

Due Diligence under the CRA: a continuous risk-based process

Due diligence under the CRA must be understood as a continuous, risk-based obligation rather than a procedural formality. As clarified in the European Commission’s FAQ and further complemented via the CEN PT1 standard, it is not a checklist to complete or a document to obtain, but an ongoing responsibility carried by manufacturers placing products with digital elements on the market. Its purpose is to ensure that third-party components, regardless of origin, do not compromise the cybersecurity of the final product.

At its core, due diligence requires manufacturers to make informed, traceable decisions about the software they integrate. This includes understanding the origin and role of components, evaluating their security characteristics, and determining whether their use is appropriate within the context of the product. These activities form a continuous lifecycle process covering evaluation, integration, monitoring, and remediation. The level of scrutiny applied is inherently risk-based and contextual, and depends on the role, exposure, and criticality of each component within the manufacturer’s system.

This obligation is dynamic by nature. Software components evolve, vulnerabilities are disclosed continuously, and integration contexts change over time. Due diligence therefore extends across the entire lifecycle, requiring manufacturers to revisit earlier assumptions and adjust mitigation strategies as new information becomes available. This creates a continuous feedback loop between upstream changes and downstream risk decisions.

The regulatory expectation is that this process is demonstrable through technical documentation that allows decisions and risk assessments to be traced and verified. The emphasis is not on collecting predefined assurances, but on ensuring that decision-making remains consistent, auditable, and defensible over time.

For open source components, due diligence relies on observable project characteristics rather than formal assurances. Manufacturers assess elements such as maintenance activity, responsiveness to security reports, release practices, and the availability of structured security documentation, drawing on signals that reflect how a project is actually developed and maintained. These signals can be aggregated into continuously updated, machine-readable evidence reflecting the current security posture of both the component and its dependencies. This approach does not create any dependency on upstream attestations: under the CRA, manufacturers remain solely responsible for their assessments, while any transparency provided by open source projects is entirely voluntary and does not constitute certification or liability. Machine-readable security signals therefore function primarily as decision-support inputs within downstream risk management processes. They improve the quality, consistency, and scalability of due diligence activities, but they do not replace the manufacturer’s obligation to exercise independent judgment and accountability. 

Machine-Readable Signals in Practice

A mature ecosystem of tools already generates machine-readable signals that can support due diligence under the CRA. These signals span multiple layers of the software lifecycle, from component identification to vulnerability management and build integrity. Standards such as SPDX and CycloneDX enable structured software bills of materials (SBOMs), while frameworks like SLSA define levels of build provenance and integrity. Complementary technologies such as Sigstore provide cryptographic mechanisms to verify artifacts, and formats like CSAF and VEX support the structured exchange of vulnerability and exploitability information. Tools such as SBOM CVE Check, Dependency-Track, Syft, and Grype operationalize these standards by enabling SBOM-driven component analysis and automated vulnerability scanning, while SBOMQS provides additional validation of SBOM quality and compliance against established standards.

Within the OpenSSF ecosystem, these capabilities are reinforced by a growing set of complementary tools that standardize, expose, and operationalize security-relevant signals across different layers of the software lifecycle. Some focus on repository security posture and development practices, others on supply chain integrity and provenance, while newer systems increasingly aggregate these signals into unified, queryable models suitable for large-scale risk analysis and automated due diligence workflows. At the project governance and security posture layer, OpenSSF Scorecard provides automated checks on repository hygiene and secure development practices, while the Best Practices Badge and Open Source Project Security (OSPS) Baseline initiatives offer structured indicators of project maturity and security adoption. OpenSSF Security Insights extends this approach by introducing a standardized, machine-readable format for publishing security policies, development processes, and maintenance practices, enabling more consistent interpretation of project security posture across tools and downstream consumers. Complementing these efforts, LFX Insights aggregates operational and community signals related to project activity, contributor diversity, governance, and security posture, helping organizations evaluate the long-term sustainability and operational health of dependencies over time. Together, these tools transform otherwise fragmented repository metadata into reusable signals that support risk-based evaluation without requiring additional compliance artifacts from maintainers. 

At the supply chain integrity layer, frameworks such as in-toto provide cryptographically verifiable attestations describing individual steps within software build and release pipelines, strengthening provenance visibility and artifact integrity. SBOMit builds on this model by combining SBOM generation with in-toto attestations and signed supply chain layouts, enabling verifiable component composition during the build process. Related tooling such as Protobom and Bomctl improves interoperability and operational reuse of SBOM data. Protobom provides a format-neutral intermediate representation that allows SPDX and CycloneDX documents to be transformed and consumed consistently across heterogeneous tooling ecosystems, while Bomctl enables structured manipulation, merging, and management of SBOM trees across complex dependency environments.

Increasingly, these signals are being aggregated into higher-level analytical systems capable of supporting continuous, ecosystem-scale risk analysis. GUAC (Graph for Understanding Artifact Composition) demonstrates this direction by ingesting SBOMs, provenance attestations, vulnerability reports, OpenSSF Scorecard results, and related metadata into a continuously queryable graph model. This enables dependency-aware analysis of upstream risk exposure, artifact relationships, and vulnerability propagation across software ecosystems. Architecturally, systems such as GUAC illustrate a broader shift within software supply chain security: compliance and due diligence increasingly become problems of correlating continuously generated technical evidence rather than collecting static documentation.  

Additionally, tools and frameworks such as OSS Review Toolkit (ORT), Community Health Analytics in Open Source Software (CHAOSS), and OpenChain extend this landscape by enabling deeper analysis and contextual understanding. ORT integrates dependency, license, and vulnerability analysis into reproducible workflows, while CHAOSS provides metrics on project activity, health, and sustainability. OpenChain complements these by defining standards for open source compliance and supply chain governance, helping organizations establish consistent, auditable processes for managing open source use. Together, these perspectives allow manufacturers to assess not only technical risk but also organizational maturity and long-term sustainability of the components on which they depend.

This direction is also reflected in European cybersecurity guidance. ENISA’s Security by Design and Default Playbook highlights machine-readable signals as a mechanism for making security both demonstrable and verifiable within development processes. By enabling continuously generated, machine-consumable evidence that can be automatically validated and reused, this approach reinforces the shift from static documentation to dynamic, lifecycle-integrated assurance, directly supporting scalable due diligence.

European initiatives further build on this foundation by operationalizing these signals into compliance workflows. EU-funded projects such as CRACoWi, CYBERFORT, CONFIRMATE, OSCRAT, and OCCTET ingest machine-readable inputs (including SBOMs, vulnerability data, and provenance information) and transform them into risk assessments and technical documentation. These platforms demonstrate how compliance can be implemented as a continuous, automated process, reinforcing the complementarity between upstream signal generation and downstream consumption.

Taken together, these tools form an interoperable ecosystem of continuously generated signals. They already provide most inputs required for effective due diligence, demonstrating that the necessary data exists within current development workflows. This confirms a key architectural reality: compliance at scale is fundamentally a data integration problem, not a documentation problem.

A large share of due diligence-relevant information is already present within open source repositories. Security policies (SECURITY.md, security.txt), contribution workflows (CONTRIBUTING.md), release histories (changelogs), issue templates, licensing files, and repository governance practices (branch protection, maintainer authentication) collectively describe how software is developed, maintained, and secured. Together, they provide a rich baseline for assessing development discipline, update reliability, and supply chain integrity, without requiring additional compliance artifacts.

However, these signals are often distributed across heterogeneous formats and locations, making them difficult to discover and reuse at scale. Improving their visibility through lightweight structuring or simple indexing can significantly reduce friction. This is not about adding new artifacts, but about improving the accessibility of existing ones.

The viability of this model is already demonstrated in practice. Many open source projects publish structured security information as part of their normal operations. Projects such as K3s provide comprehensive self-assessments describing architecture and security considerations, while the Argo project maintains clear documentation of its vulnerability disclosure processes. Other initiatives, including Privateer and other projects within the CNCF ecosystem, expose structured security information directly within their repositories. These projects demonstrate that “CRA readiness” is essentially an extension of existing high-quality security engineering. Documenting security posture is already a well-established practice; what is missing is not the content, but a consistent way to connect that content to automated due diligence workflows.

Guidance from OpenSSF security assessments further supports this practice by helping projects think systematically about their security posture, development processes, and risk boundaries.

This becomes particularly critical at scale. Modern systems depend on hundreds or thousands of components, making manual evaluation (and reliance on manual, individualized attestations) infeasible. Machine-readable signals enable automated collection, continuous updates, and consistent analysis across dependency graphs, transforming due diligence into a reproducible computational workflow aligned with contemporary software realities.

Voluntary Upstream Participation and Ecosystem Engagement

It is critical to start with a baseline legal protection: maintainers and stewards are not “suppliers” in a commercial sense and are not required or expected to sign contracts or guarantee compliance outcomes. Under the CRA, open source projects are under no obligation to support compliance activities. Responsibility remains exclusively with manufacturers placing products on the market. This legal boundary is intentional and reflects the regulation’s effort to preserve the open source model, particularly the “no warranties, no liabilities” foundation that enables broad participation and innovation. More broadly, the legislative intent of the CRA is to protect the “long tail” of community-driven innovation, ensuring that smaller projects, individual maintainers, and informal communities are not burdened with obligations designed for commercial actors.

At the same time, many projects already choose to publish security-relevant information such as vulnerability handling policies, release processes, or build metadata. These actions improve transparency and usability for downstream users, but they do not create legal obligations, warranties, or liability. They are best understood as voluntary engineering practices that reduce friction and make projects easier to adopt within regulated environments.

A central principle throughout this model is the clear distinction between transparency and assurance. Transparency refers to voluntarily published, descriptive information about how software is developed, maintained, and secured. Assurance, by contrast, implies some form of validated guarantee, certification, or assumption of responsibility. Under the CRA and within open source ecosystems, transparency must never be interpreted as assurance. 

Any attempt to interpret voluntary signals as certification risks undermining both the legal structure of open source licensing and the intent of the CRA. From a regulatory perspective, oversight remains focused on products placed on the market rather than upstream development processes. As a result, any security signals published by projects function as inputs into downstream evaluation, not as regulatory objects in themselves.

Within this framework, upstream security signals serve only as inputs into downstream due diligence. Manufacturers remain responsible for evaluating those inputs and making risk-based decisions. The availability of better signals can improve that process, but it does not shift accountability or create dependencies on upstream participation.

At the same time, the CRA introduces an important change in incentives. Manufacturers can no longer rely on passive consumption of open source components without understanding their security implications. When gaps in security signals are identified, the most effective response is not to request formal assurances but to improve the upstream ecosystem through tooling, documentation, funding, or engineering contributions. This includes supporting SBOM generation, provenance tooling, vulnerability disclosure processes, or automation of security pipelines.

This dynamic creates an opportunity for a more balanced and sustainable relationship between upstream and downstream actors. By investing in the security and transparency of the projects they depend on, manufacturers not only support their own compliance efforts but also strengthen the resilience of the broader ecosystem. This shift does not alter legal responsibilities, but it encourages a model of shared interest where better upstream practices benefit all participants without imposing new obligations on open source maintainers. 

Building a Scalable Model for Cyber Resilience

Modern software systems routinely incorporate thousands of independently evolving components, making static documentation obsolete almost as soon as it is produced. In this context, scalable due diligence cannot rely on manual, document-driven approaches. Machine-readable security signals provide a scalable alternative: continuously generated, verifiable, and aligned with dynamic software supply chains.

A scalable due diligence model therefore depends not only on machine-readable signals, but also on correctly interpreting their nature. They are evidence of behavior, not guarantees of outcome. Maintaining the distinction between transparency and assurance is what allows these signals to be useful without distorting responsibility or imposing unintended obligations upstream. 

The CRA establishes a clear downstream responsibility model, but its effectiveness depends on implementation. When grounded in automation, interoperability, and continuously updated evidence, due diligence becomes operational rather than procedural. This approach builds on existing engineering practices and widely adopted tooling, enabling scalable risk assessment without imposing new burdens on upstream maintainers. This direction is consistent with ENISA’s Security by Design and Default Playbook, which emphasizes machine-readable security attestations as a foundation for demonstrable and continuously verifiable security across the software lifecycle.

Crucially, this model preserves the sustainability of the open source ecosystem. It avoids shifting liability or compliance expectations onto maintainers, while still improving transparency through low-friction, machine-readable signals. Much of the required information already exists within projects today; the challenge is not creation, but integration and consistent reuse. By treating compliance as something assembled from technical evidence rather than declared through static attestations, the process remains both accurate and adaptable over time.

Ultimately, a shift toward continuous, evidence-based due diligence ensures cybersecurity can scale alongside software complexity. It enables manufacturers to manage large dependency landscapes efficiently, supports ecosystem resilience, and fosters more meaningful upstream–downstream collaboration. Compliance is not a one-time declaration but an ongoing capability that strengthens both regulatory outcomes and the integrity of the digital infrastructure.

About the Author

Madalin Neag works as an EU Policy Advisor at OpenSSF focusing on cybersecurity and open source software. He bridges OpenSSF (and its community), other technical communities, and policymakers, helping position OpenSSF as a trusted resource within the global and European policy landscape. His role is supported by a technical background in R&D, innovation, and standardization, with a focus on openness and interoperability.

May 18
Love0

Taking Stock of the State of European Cyber Resilience Act (CRA) Compliance: An Urgent Wake-up Call for the Open Source Ecosystem

By Blog, EU Cyber Resilience Act, Global Cyber Policy

By Christopher (CRob) Robinson, OpenSSF

For the better part of two years, discussions surrounding the European Cyber Resilience Act (CRA) have been somewhat theoretical: mapping requirements, debating definitions, and analyzing how the requirements will impact our amazing ecosystem. But folks, it’s mid-2026, and the CRA is live. Theory is officially in the rearview mirror as implementation milestones roll out over the next two years. 

I’ve just finished reviewing the finalized 2026 CRA Awareness and Readiness Report, a joint effort with LF Research experts, and to be blunt, the results are a sobering reality check. Despite tireless community work, the broader ecosystem is far from ready for CRA compliance.

CRA Awareness Has Stalled 

The most disappointing finding is that awareness surrounding this regulation has decreased year-over-year. Today, 66% of respondents remain unfamiliar with the CRA, a slight increase from 62% in 2025. That means a growing portion of the software ecosystem is unaware of a regulation with global consequences and hefty fines. 

The geographic disparity is even more alarming. In the United States and Canada, nearly 72% of respondents are unfamiliar with the regulation. It cannot be understated: if you are a North American company selling software products into the EU market, you are legally required to comply with the CRA. However, the majority of the neighborhood is still walking unprepared toward a September 2026 reporting deadline. 

Why the “Consume and Forget” Model is No Longer Possible

For years, organizations have treated open source like a free lunch: grabbing code and assuming the lights are being kept on by someone else. Under the CRA, that posture is no longer tenable. Manufacturers now bear the legal responsibility for the security of the components they integrate. For some (read: most) this is a stark wake up call. 

Despite that, 51% of manufacturers still passively rely on upstream projects for security fixes. In the new world of the CRA, “passive” is a level 10 risk.

Private Forks Are Not the Answer (They’re Worse) 

Many of you have tried to dodge the upstream journey by maintaining private forks, but inefficient code is still inefficient code, and now we have the bill to prove it. The report shows that maintaining private workarounds is a massive form of technical debt, costing organizations an average of $258,000 in labor every single release cycle. With some release cycles as short as a matter of hours, these costs can quickly get out of hand. 

For large organizations (5,000+ employees), this burden exceeds 11,152 labor hours per cycle. Maintaining these divergent codebases is a giant bill for a strategy that actually makes supply chain transparency worse. Contributing fixes upstream isn’t just being a “good neighbor” – it’s the only financially rational path forward.

For the last several years, the OpenSSF community has observed traditional vulnerability disclosure systems buckling under the strain of volume of discoveries being reported through them. Data from the report points to a surge of 394% increase in Common Vulnerabilities and Exposures (CVEs) and an 811% spike in vulnerabilities that fall within the High+ severity categories in the first quarter of 2026. Several factors contribute to this trend:

  • Transparency: Open source is open and transparent, which means the community cannot hide vulnerabilities behind opaque processes or paywalls. 
  • Project Growth: Year-over-year we’re seeing an explosion of MORE open source projects.
  • Ubiquity: Open source is quite literally the majority of software used globally. 
  • AI Tools: More users are leveraging Large Language Models (LLMs) and other tools to explore and analyze software. The transparency of open source software offers a low barrier of entry for those using these new tools and test code. 

Globally, regulations like the CRA are codifying long-standing security guidance into law. This shifts security from a “nice-to-have” recommendation to a legal requirement backed by heavy non-compliance fines. 

How Does Upstream Investment Improve Your Security Posture? 

On the bright-ish side the data reveals a clear correlation: organizational diversity is a strong predictor of a project’s security posture. When more organizations invest in a project, that project becomes more resilient, making upstream investment a direct catalyst for your own compliance posture. Organizations have an important role in their own security health through their participation in open source projects.

However, the participation of small and medium-sized enterprises (SMEs) is crucial to the entire ecosystem, they are the backbone of the industry. Currently, over half of European SMEs remain unfamiliar with the CRA, creating a significant gap in project diversity. Directed investment in SME engagement is essential to prevent compliance from becoming a structural barrier to innovation. By funding the support and tools these smaller players need to remain compliant, we ensure the entire upstream supply chain remains robust and competitive.

What OpenSSF Resources Can Help Organizations Prepare for the CRA? 

While we wait for the full 2026 report to drop, the tools to succeed already exist. Our previous research, Unaware and Uncertain: The Stark Realities of Cyber Resilience Act Readiness in Open Source, highlighted these same gaps a year ago. It’s time to start acting. The tools to succeed already exist and practitioners who find our resources rate them highly:

This ecosystem is rife with the talent and the collaborative instincts to meet this challenge. The December 2027 deadline is a forcing function, but it’s an opportunity to build a software supply chain that is actually secure by design.

Europe is leading the way in protecting consumers globally. Despite our geographic distance in the U.S., the oceans between us all do not provide isolation from this regulation any longer. Software and products with digital elements are built with hardware, software, and firmware created through international collaboration. That fact feeds the global economy and makes manufacturers globally responsible for CRA adherence. Events that happen “over there” DO truly affect everyone.  

The results of the CRA research conducted with our peers in LF Europe is truly grave. A significant amount of work and collaboration has occurred across the ecosystem since CRA enforcement. It is shocking to look back at all this work done by both the OpenSSF and its partners and see that 39% of manufacturers, who have BILLIONS of euros at stake in potential non-compliance penalties, are still unaware and uncertain about their requirements.  

The next stage in our shared journey together unfolds  in September 2026 when the vulnerability reporting obligations are enforced. There is not much time to prepare. Organizations have a narrow window to audit their upstream dependencies and establish the processes needed to report and patch new vulnerabilities as they emerge. The more complex aspects of the CRA are currently a year out, coming due December 2027. Please, take action today to protect yourselves, your companies, the upstream maintainers on whom you depend, and your customers.

The OpenSSF encourages everyone that benefits from open source software to consider the beauty and complexity of the open software world. Every day in software repositories, chat channels, and mailing lists a talented cohort of developers co-engineer the tools you use and love. We ask that organizations and their leaders understand that free software is NOT free. Being a responsible consumer and participant in the  ecosystem creates benefits for everyone. With CRA in our midst, there is ample opportunity to make this shared space better and more secure for everyone. My hope is that we can rise to that opportunity.

Stay Ahead of the CRA

Be the first to read the 2026 CRA Research Report. Subscribe to our newsletter for an alert when it releases the week of June 9 (European Open Source Security Forum in Brussels).

Get involved with the OpenSSF Global Cyber Policy Working Group.

About the Author

Christopher Robinson (aka CRob) is the Chief Technical Officer and Chief Security Architect for the Open Source Software Foundation (OpenSSF). With over 25 years of experience in engineering and leadership, he has worked with Fortune 500 companies in industries like finance, healthcare, and manufacturing, and spent six years as Program Architect for Red Hat’s Product Security team.

May 05
Love0

What’s in the SOSS? Podcast #60 – S3E12 Packaging, Transferring, and Deploying Software in Air-Gapped Environments with Zarf

By Podcast

Summary

Host Sally Cooper is joined by Brandt Keller, a Staff Software Engineer at Defense Unicorns and Maintainer of the OpenSSF Sandbox Project, Zarf. Brandt discusses Zarf’s origins as a tool designed to reliably package, transfer, and deploy software components (like container images and Helm charts) specifically for critical, air-gapped environments that lack internet connectivity. The conversation explores Zarf’s evolution, highlighting its current role in introducing security gates, improving transparency, and consolidating various management and SBOM tools into a single, declarative workflow. Finally, Brandt explains how Zarf’s declarative manifest model is helping to secure open source software by reducing the cognitive burden on maintainers and giving integrators confidence in upstream artifacts.

Conversation Highlights

00:01: Welcome and Introduction to Brandt Keller and Defense Unicorns
02:01: What is Zarf and its history: Solving the air-gapped use case
04:33: Zarf’s critical function today: Security, transparency, and packaging
09:18: How Zarf has evolved: From niche tool to agnostic distribution and GitOps integration
12:07: Zarf’s role in OpenSSF and securing open source software
16:05: Rapid Fire and Call to Action (Zarf.dev)

Transcript

Sally Cooper (00:01.748)
Hello, hello, and welcome to What’s in the SOSS, where we talk to amazing people that make up the open source ecosystem. These are engineers, developers, maintainers, researchers, and all manner of contributors that make open source so great. I’m Sally, and today I’m really excited to be joined by Brandt Keller from Defense Unicorns. Brandt, thank you so much for being here.

And to get us started, can you tell our listeners a little bit about yourself, your role, and the kind of problems you focus on at Defense Unicorns?

Brandt Keller (00:35.742)
Absolutely. Thanks for having me. so yes, I’m Brandt Keller, primarily, a Staff Software Engineer at Defense Unicorns where I get to, know, I have the privilege of getting to focus on open source software, and maintaining that across both the open source software, projects that we have created as well as kind of the intersection of all of the things that we depend on as a company, we want to be able to be, you know,

appropriate stewards for not only consuming, but also trying to, you know, identify how we can contribute back. And so my role in particular is that of a maintainer for Zarf, which is an OpenSSF Sandbox Project. And outside of that, trying to be more of a, you know, kind of advocate in a few different spaces.

The software supply chain integrity group, working group under the OpenSSF. I try to be a contributor there, in the CNCF spaces. also contribute to, the security and compliance technical advisory group as a technical lead. And so kind of, you know, broad span, but have the definitive privilege of getting to kind of work with communities, build communities, and build technology.

Sally Cooper (02:01.972)
Oh, wow. This is so exciting to have you on the show, especially to talk about Zarf. So you mentioned Zarf. And I was just wondering if you could tell our listeners, I’m sure many of them know what Zarf is. But for those who don’t, what is Zarf and what’s its history?

Brandt Keller (02:19.02)
Yeah, for Zarf in particular, it’s, it’s evolved for sure. But I think kind of at the, at its cusp, it’s always been about trying to take this process of like, where does our software come from? and that the answer to that is varying and nuanced for everyone. but wherever it comes from, let’s try to find a way that we can package it up in a way that’s reliable and repeatable to make it secure. Ultimately, ultimately we really want to lean into that security posture.

and so what happens is software comes from many different places. people are running Kubernetes everywhere. and they have to collect all the disparate puzzle pieces in order for things to run. They have to collect their container images, their helm charts, every other file that they need to run an application. Maybe it’s their application. Maybe it’s an open source application, that their environment relies on. and we want to make that.

As easy as possible in such a way that it’s, you know, very transparent. You have everything you need. And when I say that, you know, you can maybe get to your environment and find out that you missed a piece and maybe that’s okay. But for Zarf in particular, we’ve always built for the air gapped use case, the most critical environments that don’t have connectivity back to any upstream internet connection. they have a pretty big problem here where there is no reaching back and grabbing that thing. You have to go back out and bring it back in. And that’s, been a lot of consternation for people who operate in these environments. so. Zarf really wants to make it. Let’s let’s package all that up into a single archive and make it so it’s easily transferable. It’s repeatable. So if somebody wants to take that same set of applications.

They can package it up and it’s declarative. The manifest really supports this. and in the end we have made it so that it’s, it’s a lot more repeatable to deploy to air-gapped environments where typically that has always been a lot of a juggling of many different artifacts and many different problems.

Sally Cooper (04:33.275)
Wow, yeah, that context is really helpful, especially for understanding where ZARF came from and what it was originally designed to solve. And from there, it kind of naturally leads to how ZARF is applied today. I was wondering if you could differentiate when trying to solve distinct critical environment problems.

Brandt Keller (04:53.442)
I think in the way that it’s kind of like well-postured to help people and its critical function today is to be more than its sole collection and transfer processes. We want to grab the things, we want to make sure that we have everything we need because we can’t easily reach back out. We have to go back out to the internet.

Grab more things if we don’t have them. so we want to make that process as easy to use. want to be able to kind of put those security constraints on the connected side so that we don’t bring anything that we shouldn’t bring with us to an environment that is, let’s say security critical. If you’re operating in those environments in every single piece of the, you know, software puzzle is scrutinized. we want to be very careful that nothing accidentally.

Sally Cooper (05:40.496)
Yeah.

Brandt Keller (05:51.936)
malicious or otherwise makes it into those environments. so there’s a, there’s a, you know, a wide opportunity here to introduce kind of like some security gates. how can we ensure that it’s going to be functional when it gets to the critical or secured environment? This air gapped environment in particular, is, know, first and foremost, where we, want to, you know, kind of help the community prosper. but on the outside of that is, one transparency.

If you’re pulling artifacts, where did you pull them from? What are they comprised of? And again, for those who may be operating this space, you can see, well, it’s like, well, I’m doing all those things today and that’s wonderful. but you’re probably doing them with a variety of tools. You have your, you know, container image management tool. have your Helm chart management tools. have your SBOM tooling that will then scan your.

Software artifacts. So you know what they’re comprised of know what that inventory looks like. And for Zarf, we really wanted to wrap all that up and more into a single process. You create this manifest. And after that manifest is created, you do as our package create, it’s going to grab all that stuff. We’re going to be creating SBOMS on the fly for those artifacts and putting them in the package so that again, we transfer this.

We still see things burned to CDs, as well as, that may seem. And, we want to make it so that when that artifact is in the environment, it’s very, it’s very transparent. You can look at it and be what, what am I about to install? and so there’s a, I think that’s the first layer of Zarf, which is what’s package and transfer it. there are other tools that do this and that’s wonderful. We like to kind of collaborate on solving this problem.

How do we make these artifacts more portable? but then Zarf’s kind of, you know, second, would say superpower is really to enable deployment of software. so how do you know what you deployed? Where did, where did, what are its pieces? how can you work with those pieces and kind of version control them so that the kind of parts and pieces can, you know,

be sustainable and sustainability is kind of a really big part of what we want to solve. but on the outside of that, how do we do all of that without that air gapped environment in particular, having to juggle a lot of disparate or possibly insecure infrastructure. in particular, the, problem that we most often see is that when you want to operate in an air, air gapped environment, you have to bring all this other infrastructure with you.

You have to kind of stand up a registry to pull images from and more often than not, those aren’t secured with TLS because we just want to get it up and running. We want to see that it works. and so for Zarf, we really took a step back and said, how can we bring the, everything we need for the environment to operate in such a way that we’re not going to be left with these disparate puzzle pieces of, infrastructure running that could be potentially insecure. That could be hard to maintain, hard to manage and again, leaning back into ultimately not very sustainable.

Sally Cooper (09:18.49)
One thing that stands out about Zarf is that as more teams use it in real world conditions, the project itself has continued to mature. I’m just wondering in your opinion, how has Zarf evolved?

Brandt Keller (09:34.062)
I think that it’s evolved in a variety of ways, um, kind of on the, on the cusp of in the kind of the very early days, was what’s all of a very niche use case to some. And we say that in such a way that it was like, it was very much a single, uh, Kubernetes distribution, um, air gap tool. And on its onset since then we’ve kind of seen that it really does.

prosper when we look into how can this be a, you know, a tool that is very agnostic of this underlying Kubernetes distribution model, right? Now we have cloud vendors that we want to operate with air gap in the cloud. Maybe not something people always think about, but it is very prevalent. firewalls as kind of the most basic use case of kind of isolating.

cloud infrastructure from the rest of the internet or the cloud. But then we’ve got, you know, the onset of many different new Kubernetes distributions being released, many different problems to solve where you just want to be able to transfer files and maintain, maintain state on those. And so we kind of see the evolution of, you know, taking what we’ve always wanted to solve, which is to make the transfer and deployment process easier, and then integrating the rest of the ecosystem around that.

people came to us and were requesting, you how do we integrate the GitOps ecosystem with Zarf? And how do we make it so that, you know, the mutation model, which I won’t go into unless people really want to hear about it, but how do we make it so that when some of the underlying magic is happening, that, you know, hey, that’s a really great thing, but that’s a cool pattern. Could we apply it to, let’s say, GitOps and Git?

And, you know, kind of stepping back and being like, yes, yes, we can, we can make it so that rather than it being, rather than it always being a process in the air gap where you have to transfer the artifacts and then change a bunch of references so that they point to the right place in your new environment. What’s, what’s kind of handle that for the user, let’s make it more of a, you know, nice and consolidated user experience. so it’s evolved in that way, as well as, know, kind of.

as it relates to the OpenSSF, trying to make open source software more secure.

Sally Cooper (12:21)
Love that. the mutation model, I do think we’re going to need a part two for that because now I’m super curious. But with the evolution, it’s especially interesting when you zoom out and you think about software supply chain security. You think about the broader ecosystem. Why is this a good fit for OpenSSF and how does Zarf help secure open source software?

Brandt Keller (12:30.52)
That is one of the most fun parts of my job is to really try to find new avenues, right? Zarf on again, stepping back a little bit to the early days, it’s been software integrators. They want to take software from its source. Usually these are open source projects and they want to package them up and take them to their environments.

But there’s a real like key opportunity that we’re seeing manifest here really within the last, I would say year and even last couple of months where like Zarf’s transparency models, Zarf’s packaging mechanism could be a key enabler for how software consumers look at open source projects and kind of deem, you know, how well are they doing software like supply chain security?

There’s going to be some of these like onset, I’d say issues where again, maintainers of projects, whether they’re sponsored by a company or whether they’re doing it out of their free time, because they love the technology. they only have so much time in the day. And, you know, I think supply chain security, it’s evolving quickly and it’s continuing to evolve. And most of the time that asks maintainers to do more things, right? Initially you were developing an application.

And you’re saying, Hey, it’s free for everyone to use. I’m not, I’m not asking anybody to pay me to do this. and that’s great. And then more people are like, Hey, for supply chain security, I really need to know what the SBOMs look like. Can you add SBOMs to your releases? Can you sign your releases? And you can kind of see each one of these layers is now a new cognitive burden for, maintainers to kind of normally manage, then

They have to manage the processes on top of like performing that activity. so, I kind of, see some like very distinct opportunities where. If the goal is to help secure open source software, we can really do that by using kind of like Zarf’s declarative manifest model in order for upstream projects to produce releases that consolidate all of that burden again into.

Brandt Keller (14:54.294)
A declarative manifest that they orchestrate in their pipeline, just one, one command, right? Hey, let’s do as our package create. And it’s going to create my, everything I need to run an application such as guac, for instance. we’re working with GUAC on kind of this problem statement. and it’s like, there’s some very fascinating opportunities where you’re going to have everything you need to run GUAC. You’re going to have the SBOMs for GUAC. You’re going to have.

You know, the ability to have that release artifacts signed. And so if you go back to that software integrator, they typically have wanted to deploy GUAC. would create their own Zarf package and pull all those artifacts. Well, now Zarf could just, the integrator could just do a Zarf package poll and they would have everything they would need. And they would know that it’s coming from the upstream source. They would have the confidence around it. They could check the signatures, review the SBOMs and kind of.

you know, kind of look at the posture of, Hey, everything I’ve needed now is kind of consolidated into a concise workflow.

Sally Cooper (16:05.62)
Alright, Brant, before we wrap, it’s time for Rapid, Rapid Fire. These are questions I’m going to ask you, you’re going to answer without any overthinking. No explanations, just your first instinct answers. Are you ready for Rapid, Rapid Fire?

Brandt Keller (16:32)
Let’s do it.

Sally Cooper (16:34)
Star Wars or Star Trek.

Brandt Keller (16:35)
Star Wars.

Sally Cooper (16:36)
Solid. Favorite retro video game?

Brandt Keller (16:41)
Oh, uh, Spyro.

Sally Cooper (16:46)
Ooh, nice. Marvel or DC?

Brandt Keller (16:49)
Marvel.

Sally Cooper (16:51)
Excellent. All right, and last one, favorite open source mascot?

Brandt Keller (16:56)
Oh, not counting Zarf?

Sally Cooper (16:59)
Zarf, of course, after Zarf?

Brandt Keller (17:03)
Um, probably Golang Gopher.

Sally Cooper (17:08)
Oh right, I have that one too. And Zarf is really cool. I love that. All right, perfect. No notes. As we wrap things up, what’s your call to action for our audience? If someone’s listening and wants to learn more about Zarf, try it out or to get involved, where should they start?

Brandt Keller (17:25)
They should start as a Zarf.dev, short and sweet and kind of take a look around and really what we’re trying to understand is, you know, what are the needs of users in different critical environments? I like to visit the public sector groups in a variety of different foundations to see kind of what problems they face, because most often than not, the most constrained regulatory environments are the hardest ones to solve for because they can’t rely on the internet or they have additional scrutiny and we really believe that Zarf is a a great avenue to evaluate and understand. And if it’s not, we’d love to hear from you. Like why, why not? And what are the things that we could do to improve?

Sally Cooper (18:10)
Brandt, thank you so much for joining us today and for all the work you’re doing to help secure how open source software is delivered into some of the most challenging environments. And to everyone listening, happy open sourcing and that’s a wrap.

Apr 06
Love0

What’s in the SOSS? Podcast #58 – S3E10 Big Thoughts, Open Sources: Beyond the Hype: Brian Fox on Securing the Agentic Future of Open Source

By Podcast

Summary

In this inaugural episode of Big Thoughts, Open Sources, host CRob sits down with Brian Fox, Co-founder and CTO of Sonatype, to discuss the friction between rapid AI adoption and foundational software security. Brian shares insights from the 11th annual State of the Software Supply Chain Report, revealing the emergence of “slop squatting” and the high frequency of AI models recommending non-existent or vulnerable dependencies. The conversation explores how the Model Context Protocol (MCP) could revolutionize developer compliance and why the industry must fund the critical infrastructure supporting our trillion-dollar open source ecosystem.

Conversation Highlights

00:23 – Welcome: Big Thoughts, Open Sources inaugural episode.
01:01 – Brian Fox’s journey: Apache Maven, Sonatype, and OpenSSF.
02:53 – The critical role of Maven Central in the software supply chain.
03:26 – Decades of security trends: The persistent “Log4Shell” pattern.
05:34 – The “Tribal Knowledge” problem for AI agents.
07:06 – State of the Software Supply Chain Report: AI recommending made-up code versions.
08:09 – Explaining “Slop Squatting” and AI hallucinations.
10:03 – Model Context Protocol (MCP): Turning security tools into AI expert systems.
13:42 – Do not ignore 60 years of software engineering “physics”.
15:11 – The “Vulcan Mind Meld”: Injecting governance data into AI agents.
17:19 – Risks, rewards, and the need for ML SecOps discipline.
19:30 – “Inefficient code is still inefficient code”: Lessons from cloud migrations.
21:01 – Building an “AI-native SDLC” with upfront security.
24:18 – The sustainability crisis: Secure open source builds are not free.
27:17 – Conclusion: Funding open source infrastructure (8 trillion dollars of value).

Transcript

Crob (00:23)
Welcome, welcome, welcome to Big Thoughts, Open Sources, the OpenSSF’s new podcast. We’re gonna dive a little more deeply in with some of the amazing community members, subject matter experts, and thought leaders within open source, cybersecurity, and high technology. Today in our inaugural episode, I’m very pleased to welcome a friend of the show, Brian Fox from Sonotype. How you doing, Brian?

Brian Fox (00:47)
I’m doing well, how are you?

Crob (00:48)
Excellent, we’re super glad to have you today. So maybe just for our audience members that are unfamiliar with your work, could you maybe talk a little bit about how you got into open source and kind of what you specialize in in this amazing ecosystem?

Brian Fox (01:01)
How I got an open source, that’s a long conversation. geez, all the way back in 2002, 2003, I suppose, is when I really, really got involved. I had done some dabbling and some other things before that, but I got involved around that time in Apache Maven. I started writing some plugins.

They’re pretty popular plugins, people still use them these days. And those ultimately got pulled into the Apache project, the official project. I kind stowawayed and came in as a committer. A few years later, I joined up with some other folks that were also working on Maven and we co-founded Sonatype. It’s been 19 years now.

CRob (1:45)
Wow, that’s awesome.

Brian Fox (1:46)
Yeah, and so then I was ultimately the…the chair of the Apache Maven project for a long time. still an Apache member of the foundation. And then more recently, even though it’s been a while, what, four or five years now, we joined the OpenSSF. I’ve been on the Governing Board with you for a while. I’m also on the Governing Board of FINOS, which is the financial open source.

And for the last couple years, also been on the Singapore Monetary Authority’s Cyber
Experts Group. Yeah, that’s fun And so, you know, I’ve spent a lot of time focused on those things. One of the things that Sonatype does for the community we run Maven Central, right? Which for people that don’t know that’s where all the world’s open source Java components come from.

CRob

Yeah, it’s kind of sounds like a big job It is a big job running critical infrastructure for all that kind of stuff And so, you know over the years that’s given us really interesting insights into what’s going on with the supply chain so, you know, that’s kind of that’s sort of what led us to the path that brought me to OpenSSF and all those other things.

CRob (2:53)
Yeah, you and your team have been amazing participants and contributors to our community and just kind of even putting aside all the work with Maven. Just your kind of participation in our working groups and our efforts has been amazing. Yeah, thank you. So today I think you wanted to talk about a topic a lot of people probably haven’t heard about. This little thing called AI. I have a hard time spelling that.

Brian Fox (3:16)
Right?

CRob (3:20)
Let’s just set the stage. What are you thinking about? What do we want to have a conversation around AI about?

Brian Fox (3:26)
Yeah, I think so if we back up a little bit, right? So it was probably around 2011, 2012, I suppose. We started looking at some of the trends that we were seeing within the Maven central downloads. We were seeing things like the most popular version of a crypto library was the one that had a level 10 vulnerability.

fixed and patched years before, but that everybody was still using the vulnerable version. The log for J, log for shell pattern has existed basically forever. It’s not actually new. And so that led us down the path to start doing different things to help our customers A, understand what open source they were using. Way back then, nobody knew. They were like, we’re not using open source. What they really meant was, I don’t think I’m using Linux in open office. They didn’t understand.

that their developers were pulling in all these components. And so the problem space back then was helping them have visibility and then providing data and controls to help them better govern their choices. So we’ve always been trying to help expedite and make it more efficient for developers to make better choices. And so it’s interesting to see this development of AI and all of the kind of things that have come along with it. So that got me thinking, you know, what?

When we started out to build some of the stuff that we built for our customers, my focus at that time was to make it possible to do the analysis in real time so that it wasn’t the case that, we’re just going to do all our stuff and then we’re going to run a compliance scan at the end of the week or end of the month or something. So we were very focused on, it needs to be able to be run every single bill all the time. We need to be able to provide guidelines so that they don’t have to ask the legal team and wait six weeks for an answer, or the security team, right?

We were trying to capture those roles, or those rules, into the system so that they could make better choices in real time. And that was a big thing that organizations needed to be able to scale and become efficient. When you start dealing with thousands of developers, tens of thousands or millions of applications, the tribal knowledge problem kind of falls apart.

CRob (5:33)
Absolutely.

Brian (5:34)
Right? And so you start thinking about what happens with AI, and if you don’t have that stuff in an automated, you know, coded kind of way, how do you feed that to an agent? The agent’s not hanging out with you at lunch. It doesn’t get an onboarding session where we say things like, you know what, we never use an LGPL dependency because we ship our code. Or, you know, we only fix vulnerabilities five and above. Or, you know, whatever the policy may be, those things sometimes can be shared among developers.

CRob (6:02)
Right. and it plays into kind of the classic problem with engineering – is most engineers I’ve met don’t like doing documentation. And with AI entering the chat room becoming this accelerant, it’s making decisions based off of knowledge or lack thereof. if you don’t have your security policy documented, it even goes back to thinking about the early days of Kubernetes.

Where it was a big mental shift for people to have that software defined network inside. And that helped, I think, a lot of organizations get better discipline and rigor where you had less mysterious outages. Because the firewall guy in the back end said, I didn’t do anything, but try test it again.

Brian Fox (6:46)
Right, right. Yeah, for sure. And that’s kind of what we’re seeing now. We’re seeing a lot of that with the, not just with agents. mean, agents are sort of like the next big step and not everybody’s
fully there yet. Some people are dabbling with it. But even just AI assisted coding, you’re seeing the same problem that you come in and you say, hey, I want a new feature. And it just grabs whatever statistically likely thing dependency is going to be in there. We’ve done some studies. We recently released the state of the software supply chain report. It’s a great report. Yeah, thank you. This was our 11th year. We just published it last month. And we did a deep dive on AI recommendations, you know, and we found that 30 % of the time the models were recommending made up versions.

CRob (7:35)
What?

Brian Fox (7:36)
Yeah, just making them up. You know, so it’s kind of shocking. In the real world, you know, if you’ve got a tool, that’s one of those things that fails fast, right? Like it picks a version that doesn’t exist, the thing goes and it immediately blows up and then, you know, Claude or whatever you’re using will go, whoops, and it’ll fix it. So it’s kind of funny, burns some tokens, but the downsides aren’t huge.

If the agent randomly picks a terrible dependency or a very old one that does in fact exist, I would argue that’s worse because there’s no fail fast in that scenario.

CRob (8:09)
Well, you also have the whole problem with slop squatting. Where the models seem to, regardless of what vendor provider you’re getting it from, they seem to fairly consistently suggest the wrong dependencies, kind of like typo squatting.

And so now the bad guys have recognized this kind of fairly consistent behavior and they’re uploading malicious packages with those bad names so that you don’t break the build because it can find what needs.

Brian Fox (8:33)
So instead of it failing fast, it fails fast by grabbing a back door or something. Exactly, that’s exactly right. That’s what slop squatting is what they call it now. Yeah, and so those are some of the challenges that we observe and you kind of take it to the extreme where now you potentially have less sophisticated developers, not classically trained developers using these tools, and they don’t know what they don’t know.

They wouldn’t necessarily stop to say, hey, I want you to now be a security expert and do an assessment of the code you just created. Like somebody who knows better will do that. But if you’ve not lived through the pain that you and I have lived, you wouldn’t think about that. And so on average, these things are going to potentially toss away a lot of the learnings that we’ve known for so many years.

CRob (09:21)
And that’s been a chronic challenge, trying to get the tribal knowledge instantiated, trying to help people make those right decisions. And the AI tools are amazing productivity and efficiency savers, but they are bringing in, as you said, classically untrained professionals that they are not a software engineer. They don’t understand how a system should be architected, or they don’t understand kind of the app sec best practices that help secure the foundation of everything and not let the world fall apart.

Brian Fox (9:59)
The interesting thing is I think they can be if prompted correctly.

CRob (10:02)
Yes.

Brian Fox (10:03)
Right? And that’s where some of the knowledge gap comes in. And I think, what was it last summer, Anthropic released the MCP model control protocol, right? Which is, I’ve spent a lot of time thinking about that pretty deeply and looking at all the tools. And I wrote about this. I think that there’s a high likelihood that we see a lot of the tools we use in software, in the SDLC today, moving more towards providing their capabilities as subject matter experts in “a thing” to an AI agent via MCP.

So I think that, for me, is pretty exciting for a number of reasons. It means, as a tool vendor, I don’t have to create a plug-in for IntelliJ and one for Eclipse and one for VS Code. As an example, MCP can be the same thing for I don’t care what tool you’re using, because it’s interacting with me via this standard API. And I’m kind of talking to it in more or less English prompts. So my ability to deliver the value that we have into whatever tool you feel like using today, and they change every week, is pretty cool.

And I would also argue that the ability to insert that information and to potentially roll out the root prompting that all of the developers are using in these capabilities is better. You’re going to get potentially better compliance than you do today. One of the things I struggled with forever was we created an IDE plugin for our capabilities that it demoed amazingly well. It showed, hey, this dependency has vulnerabilities, or license, or would make recommendations. It was great. But developers just didn’t want to install more plugins. They just weren’t using it, right?

So while it demoed well, the actual usage of it was very low for compliance reasons. That’s a thing we struggle with. Every tool vendor struggles with that. But if you were able to insert that same information into an MCP capability and the company rolls out a root prompt that says something like, hey, every time you’re choosing a new project or a new dependency or trying to assess a dependency, use this MCP server to get up-to-date real-time information, it’s more or less going to do that every time. Right.

CRob (12:13)
Yeah, and I think back to like when I was a baby cyber person going studying for my CISSP, there was a lot of talk in the exam materials about expert systems, which is exactly what I think a best case scenario with these tools can be. It’s you’re expert. I don’t have to necessarily have this expertise. That’s right. But thinking about it takes a lot of knowledge to craft these expert systems. Let’s talk about how some of these models have been trained on potentially less than expert data.

Brian Fox (12:43)
Right, and that’s just, think, the inevitable nature that the frontier models have been trained on, you know, all the stuff that they can find.

CRob (12:49)
The internet.

Brian Fox (12:49)
On the internet, good information, bad information, people talking about terrible dependencies a lot might statistically make that more of recommendation, right? And I think that can be okay as long as you’re plugging in the models that have real data. The things that we’ve seen, you know, when we assess the models is that like I said, they make up versions, they pick old versions arbitrarily, they don’t know about anything newer than when they were last trained, which means both new versions and also vulnerabilities that might be an older version.

So they’re inadvertently recommending, and it’s not even a recommendation really, it’s just using it, right? It’s putting it in there and writing the code around it. Imagine picking Spring, right? It’s just going to go, I’m going to write a Spring app and I’m going to use all Spring 5.

And then when you probe it, then it’s like, oh, sorry, I have to do two major framework updates. You almost have to throw it away and start over. And so if you’re able to plug the right data in up front, you don’t have all of that waste. And again, if you have people who don’t know to prompt it to ask about the latest versions, you can insert that underneath the hood. I think that’s what’s really cool.

Brian Fox (13:59)
But what we’re seeing currently, I kind of wrote about this a little bit too, that I feel like we’re throwing out all the lessons of the past. We’re talking about situations where whole tools, SAST is under fire right now, right? Because when all the code can be just completely generated, what’s the problem with SAST? But I do think that we’ve learned a lot of things over the years if we can figure out how to plug those capabilities into what’s being generated.

I think we can bring all of that forward with us. But the entire SDLC is going to have to adapt to that. It’s not going to be sufficient to say, I’ve got a bunch of developers over here. They’re doing AI assisted development. And then later, we’re going to run a bunch of SAS and produce legacy reports. That’s not going to work. The information has to be fed directly into the AI capabilities up front.

CRob (14:51)
And it’s the classic problem we’ve always had, where security historically is the the last thing done, addressed, it’s bolted on at the end in a lot of cases. And just this AI tooling and just the velocity it has is a huge accelerant for the sins of our past we’ve never actually addressed.

Brian Fox (15:11)
Absolutely, but it also provides the Vulcan mind meld if you want to think of it. You now have that opportunity to plug that right into what the agent is thinking about in the moment. You can’t do that with the humans, but you can do that with the agent. And that’s what I think is potentially exciting about this.

Where I described it recently at a summit, we’re sort of in a bootstrap situation, though, right? Like, we don’t have all of those capabilities. Organizations haven’t rolled them all out. And so we’re sort of in this weird situation, one foot on the boat, one foot on the dock, and it’s not going to end well as we’re going through it. And worse, there are people that are afraid of the MCP protocol. So I hear lots of organizations say, we just block it completely.

Yeah. It’s a little hard to argue that that’s not a reasonable place to start because of the nature of what’s happening. We saw just the other day the latest version of Shai-Hulud came out. Did you see this? And they used MCP capabilities as data exfiltration. And I’m like, come on, guys. There’s so much power in this, but now you’re making it like a bad thing. So I think the industry and the tools and all of that are going to have to work through governance of the MCP capabilities, sanitization inspection of the MCP capabilities just like we’ve seen. So it’s sort of one of these things like when you’ve been around long enough you can recognize the patterns. It’s new and exciting but also the pattern rhymes with a bunch of stuff we’ve done before and what frustrates me is that like everybody charges ahead so fast they just feel like it’s all new it’s all different it’s like yes but let’s not forget everything we’ve learned over the last 60 years of software engineering because the physics is still the same.

CRob (16:50)
Well, and that’s where so our AI / ML working group wrote a paper around ML SecOps. And the paper was really interesting. I recommend the audience check it out. But it was they talked about classic techniques that are assisted and are helpful with AI development. And then it talked about some gaps where we have things like are not documented policies that are kind of a hindrance and something that’s an opportunity in the future to try to get addressed.

Brian Fox (17:18)
Yeah.

CRob (17:19)
But…I’m of two minds about my friends, our new robot overlords, in that it can be extremely helpful, but I don’t see a lot of people reconsidering those lessons of the past of software engineering. To say this is all brand new and totally different, like, well, you’ve got different GPU accelerators and dedicated cores to do things.

And now with this like agentic and ADA architecture where things are more highly distributed, yeah, that’s new twists, but it’s not brand new. We’ve done networking. We’ve done composite applications for decades.

Brian Fox (18:01)
Right. It’s the same thing, you know, we saw when, you know, we were like, oh, everything should be serverless or let’s go to the micro architecture, micro architecture, micro service architecture is going to solve everything until it doesn’t. Right.

Or, you know, that’s no problem, we’ll just put it in the cloud because I can just infinitely scale my machines, right? So I see the same pattern all again, that we sort of say, yes, but this time is different because insert new technology, and then we realize, yes, but everything we know is still true. And that’s what I think we’re sort of grappling with right now as we go through this. What is absolutely true is that, you know, the AI capabilities, the agents, all these things are making everything happen so much faster. That can be good.

can also be bad. If you’ve forgotten all the lessons of the past, you’re just going to create a ton of crap much faster than you could before. And by the time you realize it, it might be too late.

CRob (18:57)
I’m familiar with a lot of enterprises that were going through a digital transformation journey, trying to update their heritage software to newer things and to the cloud to get that scalability and cost efficiency. But a lot of organizations didn’t take that journey, didn’t learn from lessons from the past.

they just crammed what they had out in the cloud, and then a month later they get this giant bill and they’re shocked and confused, or they didn’t understand that this thing wasn’t architected for zero trust, and they’re leaking data everywhere.

Brian Fox (19:30)
Right, right. Or that, or just even the performance reasons why you were excited to infinitely scale, sure, but somebody’s not excited to infinitely pay a bigger bill. Inefficient code is still inefficient code, right? And that’s what I think we’re gonna see with… with AI capabilities is just going to happen faster. And without humans in the loop, it provides less opportunities for us to course correct, which is why I’ve been taking a step back and thinking about how do we do that? How does it make sense? I think for some of the stuff that we’ve been doing as a business, it’s really exciting because we have built up really interesting, unique data sets based on being able to see everything going on with Maven Central. We’ve long had Nexus, the repository manager that’s out there.

We have hundreds of thousands of instances. Those things are proxying for enterprises, not just Maven, but NPM, Nougat, Python, all the things. And so that gives us visibility into other ecosystems so we can understand what’s going on, what’s commonly used in enterprises, these kinds of things. And so all of that data can be fed now directly, like I said, the Vulcan mind meld directly into these tools. And it makes it a lot easier.

So in some ways, when we sort this out, and people become less afraid of MCP capabilities, we can directly inject a stream of high quality data to make all of those things better. But, before businesses can really leverage that, they have to get out of the experimentation phase. They have to roll that out. And these things are kind of interrelated. What we see is that organizations are afraid to let developers just go with AI assisted development because it’s not governed, because they can’t govern it.

And those are echoes of what I saw firsthand during the early days of open source. Like I said in the beginning, people said, we’re not using it. And then I’d tell them, yes, you are. And then their reaction was like, well, just shut it all out. It’s like, right, you can’t do anything. So the reaction that some enterprises have right now of like, we’re just not going to do anything with AI, is just setting themselves up to be left behind.

The right answer is to do it thoughtfully and use tools to help them make better decisions.

CRob (21:43)
So reflecting back, mentioned in your report that you have some guidance for people around AI. What would the top two or three things, if somebody’s thinking about moving more aggressively in this AI direction, what can they take away and do immediately or start thinking about?

Brian Fox (21:01)
Yeah, I mean, think the biggest thing is humans like to…try to take the old patterns and just adopt it to the new new technologies like we were talking about take an inefficient architecture and throw it in a cloud It’s gonna fix everything. No, it’s not and I think that’s true of Let’s call it the AI SDLC right an AI native SDLC Might resemble a normal SDLC, but it should be designed differently, right?

You know trying to do the checks and balances after the fact is even worse than it was with humans You need to think about providing that information upfront so that you get the value in the creation of the code and not try to chase it out. You need to be able to think about how all of these things can be done in parallel with agents, breaking these things down. what I would say is, don’t just try to do what you’re doing today and use AI to do it. Take a step back and really assess how can you adopt this.

It’s sort of like the conversations we were having in the board today about developers, maintainers are getting overwhelmed with AI slop. It’s true. A reaction is to stop allowing that to be contributed, just dismiss everything AI. That’s not a good answer. A better answer is let’s figure out how to help them use AI tools to be able to keep up with that, right? Because that’s what it’s good for. It can review and assess the patches faster than the maintainers and then provide sort of a first pass filter, if you will, right?

But that requires thinking outside of the box. Don’t just try to keep doing what you’re doing and try to keep up with it. Think about how you judo move that into something that makes more sense for your organization.

CRob (23:42)
And this skirts along another kind of project you’re passionate about, sustainability and funding. It is one thing to try to admonish the developers, why aren’t you using AI? But there are real costs involved around this. And, just to say, well, you should use the tool that doesn’t help them when there’s no funding. They don’t have access to infrastructure to be able to do these things. And that’s like, think, it touches on your passion project around trying to help get the package repositories more sustainably funded.

Brian Fox (24:18)
That’s right. Yeah, I mean, if you take a step back and you think about open source when probably you started, certainly when I started, what that really meant was you were donating your time. And you were sharing your thoughts, and you were sharing your words via code. And that was in a time when it was perfectly acceptable. In fact, it was the only choice that you built things and you shipped them off of your laptop.

There was when the Apple MacBook Air launched, the first one. That launched with a version of Maven on it that was signed by my key, my personal key, that was built on my personal laptop. So everybody that bought the launch version of the MacBook Air had my signed code on it. That’s kind of cool.

But also kind of scary, right, when you think about it. Like, what if my laptop was compromised? And that’s the world we live in today. Fortunately, in 2009 or whatever it was, that was a little bit more remote of a chance. And so everybody thinks like, well, that’s crazy. You wouldn’t do that anymore. So what does it mean today? It means you have to have certified builds. Usually that means it’s running in the cloud, and it’s attested to, and all these kinds of things. And that’s not free.

Like I can’t donate that, I’m not a hyperscaler. Most open source gets that infrastructure donated by these big companies, but there’s a lot of opportunity for abuse, right? And these types of things. it’s just, at the end of the day, it’s not free. So the cost of producing open source is not free anymore. It’s not just donating my time with equipment and internet access I already have, right? That’s the big difference. And I think people don’t really recognize that and now fast forward to what we’re just talking about AI the obvious answer to deal with AI, you know Piled on PRS is to have AI assistance help.

Who’s gonna pay for that? It’s literally not free. It costs electricity, last time I checked we still pay for our electricity Regardless

CRob (26:12)
Electricity, water…

Brian Fox (26:14)
Right all of these things, right? These are very…they have very real implications. They’re just not free and so There’s no good answer to that. How does that get aligned? How do we…how do we continue to create open source software that can power all of these industries in a world where it’s not just somebody donating their time and thoughts? There are no good answers. But we’re working towards trying to align that. Because the bulk of open source software, certainly in our world, in these areas, is being consumed by organizations that are selling for-profit software, more or less.

There’s definitely a lot of hobbyists and stuff like that the biggest consumers from our repositories are all the giant companies. I’ve named the top 100. You would know every single one of them. And I’m sure that’s true for all the registries. So there has to be an answer in there. I don’t know the stat off the top of my head, but the Linux Foundation does the census, right? And it’s billions of dollars of economic value that open source creates. Eight billion? Nine billion?

CRob (27:16)
Trillion.

Brian Fox (27:17)
Oh, it’s a trillion now?

CRob (27:17)
It’s eight (8) trillion, I believe.

Brian Fox ()
Eight (8) Trillion dollars worth of economic value being produced by open source…1 % of that would pay for a lot of that infrastructure, and then a whole bunch more. And so I think that’s what ultimately we have to figure out how to balance. AI just makes that worse, because it moves the bar even further.

CRob (27:39)
Interesting conversation. Any final thoughts you want our listeners and viewers to take away?

Brian Fox (27:46)
Well, certainly go take a look at The State of the Software Supply Chain Report.

CRob (27:51)
Great report.

Brian Fox (27:52))
sonatype.com/SSCR Certainly, I’ve also written a number of blogs. You can find those at our website as well. That deep dive, kind of all these topics we touched on here. Yeah.

CRob (28:02)
Excellent. We’ll put some links as we do our summary. So Brian, thank you for our inaugural episode of Big thoughts, Open Sources. I think this was an amazing conversation that we’re gonna continue to be adding onto and reconsidering in the coming weeks and months.

Brian Fox (28:20)
Yeah, thanks for having me kick it off in Napa.

CRob (28:25)
Thank you. Well, I hope everybody stays cyber safe and sound. We’ll talk to you soon.

Mar 23
Love0

Kusari Partners with OpenSSF to Strengthen Open Source Software Supply Chain Security

By Blog, Guest Blog

Cross-post originally published on the Kusari Blog

Open source software powers the modern world; securing it remains a shared responsibility.

The software supply chain is becoming more complex and more exposed with every release. Modern applications rely on vast ecosystems of open source components, dependencies, and increasingly AI-generated code. While this accelerates innovation, it also expands the attack surface dramatically. Threat actors are taking advantage of this complexity with more frequent and sophisticated attacks, from dependency confusion and malicious package injections to license risks that consistently target open source communities.

At the same time, developers are asked to move faster while ensuring security and compliance across thousands of components. Traditional security reviews often happen too late in the development lifecycle, creating friction between development and security teams and leaving maintainers overwhelmed by reactive work.

Kusari is proud to partner with the Open Source Security Foundation (OpenSSF) to offer Kusari Inspector at no cost to OpenSSF projects. Together, we’re helping maintainers and security teams gain deeper visibility into their software supply chains and better understand the relationships between first-party code, third-party dependencies, and transitive components.  

Projects adopting Kusari Inspector include Gemara, GitTUF, GUAC, in-toto/Witness, OpenVEX, Protobom and Supply-chain Levels for Software Artifacts (SLSA). As AI coding tools become standard in open source development, Kusari Inspector serves as the safety net maintainers didn’t know they needed. 

“I used Claude to submit a pull request to go-witness,” said John Kjell, a maintainer of in-toto/Witness. “Kusari Inspector found an issue that Claude didn’t catch. When I asked Claude to fix what Kusari Inspector flagged, it did.”

Maintainers are under growing pressure. According to Kusari’s Application Security in Practice report, organizations continue to struggle with noise, fragmented tooling, and limited visibility into what’s actually running in production. The same challenges affect open source projects — often with fewer resources.

Kusari Inspector helps OpenSSF projects:

  • Map dependencies and transitive risk
  • Identify gaps in attestations and provenance
  • Understand how components relate across builds and releases
  • Reduce manual investigation and security guesswork

Kusari Inspector – Secure Contributions at the Pull Request

Kusari Inspector also helps strengthen the relationship between developers and security teams. Our Application Security in Practice research found that two-thirds of teams spend up to 20 hours per week responding to supply chain incidents — time diverted from building and innovating. 

For open source projects, the burden is often even heavier. From our experience in co-creating and maintaining GUAC, we know most projects are maintained by small teams of part-time contributors and already overextended maintainers who don’t have dedicated security staff. Every reactive investigation, dependency review, or license question pulls limited capacity away from priorities and community support — making proactive, workflow-integrated security even more critical.

By increasing automated checks directly in pull requests, projects reduce review latency and catch issues earlier, shifting from reactive firefighting to proactive prevention. Instead of maintainers “owning” reviews in isolation, Kusari Inspector brings them integrated, context-aware feedback — closer to development and accelerating secure delivery.

This partnership gives OpenSSF projects the clarity they need to make informed security decisions without disrupting developer workflows.

“The OpenSSF welcomes Kusari Inspector as a clear demonstration of community support. This helps our projects shift from reactive security measures to proactive, integrated prevention at scale,” said Steve Fernandez, General Manager, OpenSSF.

“Kusari’s journey has always been deeply connected to the open source security community. We’ve focused on closing knowledge gaps through better metadata, relationships, and insight,” said Tim Miller, Kusari Co-Founder and CEO. “Collaborating with OpenSSF reflects exactly why Kusari was founded: to turn transparency into actionable trust.”

If you’re an OpenSSF project maintainer or contributor interested in strengthening your supply chain posture, use Kusari Inspector for free — https://us.kusari.cloud/signup.

Author Bio

Michael LiebermanMichael Lieberman is co-founder and CTO of Kusari where he helps build transparency and security in the software supply chain. Michael is an active member of the open source community, co-creating the GUAC and FRSCA projects and co-leading the CNCF’s Secure Software Factory Reference Architecture whitepaper. He is an elected member of the OpenSSF Governing Board and Technical Advisory Council along with CNCF TAG Security Lead and an SLSA steering committee member.

Mar 17
Love0

Leading Tech Coalition Invests $12.5 Million Through OpenSSF and Alpha-Omega to Strengthen Open Source Security

By Blog

Securing the open source software that underlies our digital infrastructure is a persistent and complex challenge that continues to evolve. The Linux Foundation announced a $12.5 million collective investment to be managed by Alpha-Omega and The Open Source Security Foundation (OpenSSF). This funding comes from key partners including Anthropic, Amazon Web Services (AWS), Google, Google DeepMind, GitHub, Microsoft, and OpenAI. The goal is to strengthen the security, resilience, and long-term sustainability of the open source ecosystem worldwide.

Building on Proven Success through OpenSSF Initiatives

This new investment provides critical support for OpenSSF’s proven, maintainer-centric initiatives. Targeted financial support is a key catalyst for sustained improvement in open source security. The results of the OpenSSF’s collective work in 2025 are clear:

  • Alpha-Omega invested $5.8 million in 14 critical open source projects and completed over 60 security audits and engagements.
  • Growing a Global Community: OpenSSF grew to 117 member organizations and was advanced by 267+ active contributors from 112 organizations, working across 10 Working Groups and 32 Technical Initiatives.
  • Driving Technical Impact: The OpenSSF Technical Advisory Council (TAC) awarded over $660,000 in funding across 14 Technical Initiatives, strengthening supply chain integrity, advancing transparency tools like Sigstore, and enabling community-driven security audits.
  • Measurable Security Uplift: Focused security engagements across critical projects resulted in 52 vulnerabilities fixed and 5 fuzzing frameworks implemented.
  • Expanding Education: Nearly 20,000 course enrollments across OpenSSF’s free training programs, with new courses like Security for Software Development Managers and Secure AI/ML-Driven Software Development empowering developers globally.
  • Global Policy Engagement: Launched the Global Cyber Policy Working Group and served as a challenge advisor for the Artificial Intelligence Cyber Challenge (AIxCC), ensuring the open source voice is heard in evolving regulations like the EU Cyber Resilience Act (CRA).

AI: A New Frontier in Security

The security landscape is changing fast. Artificial intelligence (AI) accelerates both software development and the discovery of vulnerabilities, which creates new demands on maintainers and security teams. However, OpenSSF recognizes that grant funding alone is not the sole solution to the problems AI tools are causing today on open source security teams. This moment also offers powerful new opportunities to improve how security work is completed.

This new funding will help the OpenSSF provide the active resources and dedicated projects needed to support overworked maintainers with the triage and processing of the increased AI-generated security reports they are currently receiving. Our response will feature global strategies tailored to the needs of maintainers and their communities.

“Open source software now underpins the majority of modern software systems, which means the security of that ecosystem affects nearly every organization and user worldwide,” said Christopher Robinson, CTO and Chief Security Architect at OpenSSF. “Investments like this allow the community to focus on what matters most: empowering maintainers, strengthening security practices across projects, and raising the overall security bar for the global software supply chain.”

Securing the Open Source Lifecycle

The true measure of success will be execution. Success is not about how much AI we introduce into open source. It is determined by whether maintainers can use it to reduce risk, remediate serious vulnerabilities faster, and strengthen the software supply chain long term. We are grateful to our funding partners for their commitment to this work, and we look forward to continuing it alongside the maintainers and communities that power the world’s digital systems.

“Our commitment remains focused: to sustainably secure the entire lifecycle of open source software,” said Steve Fernandez, General Manager of OpenSSF. “By directly empowering the maintainers, we have an extraordinary opportunity to ensure that those at the front lines of software security have the tools and standards to take preventative measures to stay ahead of issues and build a more resilient ecosystem for everyone.”

To learn more about open source security initiatives at the Linux Foundation, please visit openssf.org and alpha-omega.dev.

Feb 26
Love0

OpenSSF Newsletter – February 2026

By Newsletter

TL;DR:

🇳🇱 Open Source SecurityCon Europe → Agenda live and registration open

🎙️ Securing Agentic AI in Practice → March 17 Tech Talk on AI/ML security in action

📖 Compiler Annotations Guide → Practical C/C++ hardening without rewrites

🏆 Security Slam 2026 → 30-day challenge to level up project security

🇪🇺 CRA in Practice @ FOSDEM → Turning regulation into actionable steps

📦 Package Repository Security Forum → Cross-ecosystem collaboration in action

🎙️ What’s in the SOSS? → CFP tips and a 4-part AIxCC deep dive

6 min read

Join Us at Open Source SecurityCon Europe 2026 in Amsterdam

Planning to attend KubeCon + Cloud Native Con Europe in March? Don’t miss OpenSSF’s co-located 1-day event! This gathering will bring together a diverse community, including software developers, security engineers, public sector experts, CISOs, CIOs, and tech pioneers, to explore challenges and opportunities in modern security. Collaborate with peers and discover the essential tools, knowledge, and strategies needed to ensure a safer, more secure future.

The agenda is live! Read the blog to learn what not to miss in Amsterdam and to see highlights from SecurityCon North America.

Read the blog | Register now | View the agenda

Mark Your Calendar For the Upcoming Tech Talk: Securing Agentic AI in Practice: From OpenSSF Guidance to Real-World Implementation

Tech Talk: Securing Agentic AI in Practice: From OpenSSF Guidance to Real-World ImplementationJoin us for the first OpenSSF Tech Talk of the year, focusing on agentic artificial intelligence (AI) security.

In this session, we will explore how the OpenSSF AI/ML Security Working Group is developing open guidance and frameworks to help secure AI and machine learning systems, and how that work translates into real-world practice. Using SAFE MCP and other solutions from OpenSSF member companies as examples, we will highlight community-driven efforts to improve the security of agentic AI systems, the problems they address, the design tradeoffs involved, and the lessons learned so far.

We will also feature OpenSSF’s free course, Secure AI/ML Driven Software Development (LFEL1012), which gives attendees a clear path to build practical skills and contribute to this rapidly evolving field.

Register and mark your calendar for March 17 at 1:00 p.m. ET. Additional speaker information will be shared soon.

Fill Out All The Margins 📖: OpenSSF Releases Compiler Annotations Guide for C and C++

OpenSSF has released a new Compiler Annotations Guide for C and C++ to help developers improve memory safety, diagnostics, and overall software security by using compiler-supported annotations. The guide explains how annotations in GCC and Clang/LLVM can make code intent explicit, strengthen static analysis, reduce false positives, and enable more effective compile-time and run-time protections. As memory-safety issues continue to drive a significant share of vulnerabilities in C and C++ systems, the guide offers practical, real-world guidance for applying low-friction hardening techniques that improve security without requiring large-scale rewrites of existing codebases. 

Read the blog

Security Slam 2026

Security Slam 2026 is a 30-day security hygiene challenge running from February 20 to March 20, culminating in an awards ceremony at KubeCon + CloudNativeCon Europe. Hosted by OpenSSF in partnership with CNCF TAG Security & Compliance and Sonatype, the event encourages projects to use practical security tools, including OpenSSF resources, to strengthen their security posture based on their maturity level. Participants can earn recognition, badges, and plaques for completing milestones, reinforcing a community-driven effort to improve open source software security at scale. 

Read the blog to learn more | Register now to receive reminders and instructions

EU Cyber Resilience Act (CRA) in Practice @ FOSDEM 2026: From Awareness to Action

At FOSDEM 2026, the CRA in Practice DevRoom brought together open source and industry leaders to turn the EU Cyber Resilience Act from policy discussion into practical action. Through case studies and panels, speakers shared concrete approaches to vulnerability management, SBOMs, VEX, risk assessment, and the steward role. 

Read the blog

Advancing Package Repository Security Through Collaboration

On February 2, OpenSSF convened the Package Manager Security Forum, bringing together maintainers and registry operators from major ecosystems to address shared challenges in package repository security. Discussions highlighted common concerns around identity and account security, governance and abuse handling, transparency, and long-term sustainability. The session reinforced that package ecosystem risks are interconnected and that improving security requires cross-ecosystem coordination, shared frameworks, and continued collaboration through OpenSSF’s neutral convening role.

Read the recap

Getting an OpenSSF Baseline Badge with the Best Practices Badge System

Is your open source project meeting the “minimum definition” of security? The OpenSSF has officially integrated the Open Source Project Security Baseline (OSPS Baseline) into its Best Practices Badge Program.

In our latest blog, David A. Wheeler explains how you can quickly identify and meet essential security requirements to earn a Baseline Badge.

What’s in the SOSS? An OpenSSF Podcast:

#50 – S3E2 Demystifying the CFP Process with KubeCon North America Keynote Speakers

Stacey Potter and Adolfo “Puerco” García Veytia share practical, behind-the-scenes advice on submitting conference talks, fresh off their KubeCon keynote. They break down how CFP review committees work, what makes an abstract stand out, common mistakes to avoid, and why authenticity matters more than polish. The episode also tackles imposter syndrome and encourages new and diverse voices to shape the future of open source through speaking.

#51 – S3E3 AIxCC Part 1: From Skepticism to Success with Andrew Carney

Andrew Carney from DARPA explains the vision and results behind the two-year AI Cyber Challenge (AIxCC), which tasked teams with building AI systems that can automatically find and patch vulnerabilities in open source software. Despite early skepticism, competitors identified more than 80% of seeded vulnerabilities and generated effective patches at surprisingly low compute costs. The episode looks at what comes next as these cyber reasoning systems move from competition to real-world adoption.

#52 – S3E4 AIxCC Part 2: How Team Atlanta Won by Blending Traditional Security and LLMs

Professor Taesoo Kim of Georgia Tech describes how Team Atlanta combined fuzzing, symbolic execution, and large language models to win AIxCC. Initially skeptical of AI, the team shifted its strategy mid-competition and discovered that hybrid approaches produced the strongest results. The conversation also covers commercialization efforts, integration with OSS-Fuzz, and how the experience reshaped academic security research.

#53 – S3E5 AIxCC Part 3: Trail of Bits’ Hybrid Approach with Buttercup

Michael Brown of Trail of Bits discusses Buttercup, the second-place AIxCC system that pairs large language models with conventional software analysis tools. The team focused on using AI for well-scoped tasks like patch generation while relying on fuzzers for proof-of-vulnerability. Now fully open source and able to run on a laptop, Buttercup is actively maintained and positioned for broader enterprise and community use.

#54 – S3E6 AIxCC Part 4: Cyber Reasoning Systems in the Real World

CRob and Jeff Diecks wrap up the AIxCC series by exploring how competition teams are applying their systems to real open source projects such as the Linux kernel and CUPS. They introduce the OSS-CRS initiative, which aims to standardize and combine components from multiple cyber reasoning systems, and share lessons learned about responsibly reporting AI-generated findings. The episode highlights how collaboration through OpenSSF’s AI/ML Security Working Group and Cyber Reasoning Systems SIG is shaping the next phase of AI-driven security.

News from OpenSSF Community Meetings and Projects:

Upcoming community meetings

In the News:

  • The OpenSSF was featured in a Technology Magazine Q&A. CRob discusses OpenSSF’s goals, OSSAfrica, the BEAR Working Group, Security Baseline, and much more. This conversation was also covered by AI Magazine. 

Meet OpenSSF at These Upcoming Events!

Connect with the OpenSSF Community at these key events:

Ways to Participate:

There are a number of ways for individuals and organizations to participate in OpenSSF. Learn more here.

You’re invited to…

See You Next Month! 

We want to get you the information you most want to see in your inbox. Missed our previous newsletters? Read here!

Have ideas or suggestions for next month’s newsletter about the OpenSSF? Let us know at marketing@openssf.org, and see you next month! 

Regards,

The OpenSSF Team

Oct 07
Love0

What’s in the SOSS? Podcast #41 – S2E18 The Remediation Revolution: How AI Agents Are Transforming Open Source Security with John Amaral of Root.io

By Podcast

Summary

In this episode of What’s in the SOSS, CRob sits down with John Amaral from Root.io to explore the evolving landscape of open source security and vulnerability management. They discuss how AI and LLM technologies are revolutionizing the way we approach security challenges, from the shift away from traditional “scan and triage” methodologies to an emerging “fix first” approach powered by agentic systems. John shares insights on the democratization of coding through AI tools, the unique security challenges of containerized environments versus traditional VMs, and how modern developers can leverage AI as a “pair programmer” and security analyst. The conversation covers the transition from “shift left” to “shift out” security practices and offers practical advice for open source maintainers looking to enhance their security posture using AI tools.

Conversation Highlights

00:25 – Welcome and introductions
01:05 – John’s open source journey and Root.io’s SIM Toolkit project
02:24 – How application development has evolved over 20 years
05:44 – The shift from engineering rigor to accessible coding with AI
08:29 – Balancing AI acceleration with security responsibilities
10:08 – Traditional vs. containerized vulnerability management approaches
13:18 – Leveraging AI and ML for modern vulnerability management
16:58 – The coming “remediation revolution” and fix-first approach
18:24 – Why “shift left” security isn’t working for developers
19:35 – Using AI as a cybernetic programming and analysis partner
20:02 – Call to action: Start using AI tools for security today
22:00 – Closing thoughts and wrap-up

Transcript

Intro Music & Promotional clip (00:00)

CRob (00:25)
Welcome, welcome, welcome to What’s in the SOSS, the OpenSSF’s podcast where I talk to upstream maintainers, industry professionals, educators, academics, and researchers all about the amazing world of upstream open source security and software supply chain security.

Today, we have a real treat. We have John from Root.io with us here, and we’re going to be talking a little bit about some of the new air quotes, “cutting edge” things going on in the space of containers and AI security. But before we jump into it, John, could maybe you share a little bit with the audience, like how you got into open source and what you’re doing upstream?

John (01:05)
First of all, great to be here. Thank you so much for taking the time at Black Hat to have a conversation. I really appreciate it. Open source, really great topic. I love it. Been doing stuff with open source for quite some time. How do I get into it? I’m a builder. I make things. I make software been writing software. Folks can’t see me, but you know, I’m gray and have no hair and all that sort of We’ve been doing this a while. And I think that it’s been a great journey and a pleasure in my life to work with software in a way that democratizes it, gets it out there. I’ve taken a special interest in security for a long time, 20 years of working in cybersecurity. It’s a problem that’s been near and dear to me since the first day I ever had my like first floppy disk, corrupted. I’ve been on a mission to fix that. And my open source journey has been diverse. My company, Root.io, we are the maintainers of an open source project called Slim SIM (or SUM) Toolkit, which is a pretty popular open source project that is about security and containers. And it’s been our goal, myself personally, and as in my latest company to really try to help make open source secure for the masses.

CRob (02:24)
Excellent. That is an excellent kind of vision and direction to take things. So from your perspective, I feel we’re very similar age and kind of came up maybe in semi-related paths. But from your perspective, how have you seen application development kind of transmogrify over the last 20 or so years? What has gotten better? What might’ve gotten a little worse?

John (02:51)
20 years, big time frame talking about modern open source software. I remember when Linux first came out. And I was playing with it. I actually ported it to a single board computer as one of my jobs as an engineer back in the day, which was super fun. Of course, we’ve seen what happened by making software available to folks. It’s become the foundation of everything.

Andreessen said software will eat the world while the teeth were open source. They really made software available and now 95 or more percent of everything we touch and do is open source software. I’ll add that in the grand scheme of things, it’s been tremendously secure, especially projects like Linux. We’re really splitting hairs, but security problems are real. as we’ve seen, proliferation of open source and proliferation of repos with things like GitHub and all that. Then today, proliferation of tooling and the ability to build software and then to build software with AI is just simply exponentiating the rate at which we can do things. Good people who build software for the right reasons can do things. Bad people who do things for the bad reasons can do things. And it’s an arms race.

And I think it’s really both benefiting software development, society, software builders with these tremendously powerful tools to do things that they want. A person in my career arc, today I feel like I have the power to write code at a rate that’s probably better than I ever have. I’ve always been hands on the keyboard, but I feel rejuvenated. I’ve become a business person in my life and built companies.

And I didn’t always have the time or maybe even the moment to do coding at the level I’d like. And today I’m banging out projects like I was 25 or even better. But at the same time that we’re getting all this leverage universally, we also noticed that there’s an impending kind of security risk where, yeah, we can find vulnerabilities and generate them faster than ever. And LLMs aren’t quite good yet at secure coding. I think they will be. But also attackers are using it for exploits and really as soon as a disclosed vulnerability comes out or even minutes later, they’re writing exploits that can target those. I love the fact that the pace and the leverage is high and I think the world’s going to do great things with it, the world of open source folks like us. At the same time, we’ve got to be more diligent and even better at defending.

CRob (05:44)
Right. I heard an interesting statement yesterday where folks were talking about software engineering as a discipline that’s maybe 40 to 60 years old. And engineering was kind of the core noun there. Where these people, these engineers were trained, they had a certain rigor. They might not have always enjoyed security, but they were engineers and there was a certain kind of elegance to the code and that was people much like artists where they took a lot of pride in their work and how the code you could understand what the code is. Today and especially in the last several years with the influx of AI tools especially that it’s a blessing and a curse that anybody can be a developer. Not just people that don’t have time that used to do it and now they get to of scratch that itch. But now anyone can write code and they may not necessarily have that same rigor and discipline that comes from like most of them engineering trades.

John (06:42)
I’m going to guess. I think it’s not walking out too far on limb that you probably coded in systems at some point in your life where you had a very small amount of memory to work with. You knew every line of code in the system. Like literally it was written. There might have been a shim operating system or something small, but I wrote embedded systems early in my career and we knew everything. We knew every line of code and the elegance and the and the efficiency of it and the speed of it. And we were very close to the CPU, very close to the hardware. It was slow building things because you had to handcraft everything, but it was very curated and very beautiful, so to speak. I find beauty in those things. You’re exactly right. I think I started to see this happen around the time when JVM started happening, Java Virtual Machines, where you didn’t have to worry about Java garbage collection. You didn’t have to worry about memory management.

And then progressively, levels of abstraction have changed right to to make coding faster and easier and I give it more you know more power and that’s great and we’ve built a lot more systems bigger systems open source helps. But now literally anyone who can speak cogently and describe what they want and get a system and. And I look at the code my LLM’s produce. I know what good code looks like. Our team is really good at engineering right?

Hmm, how did it think to do it that way? Then go back and we tell it what we want and you can massage it with some words. It’s really dangerous and if you don’t know how to look for security problems, that’s even more dangerous. Exactly, the level of abstraction is so high that people aren’t really curating code the way they might need to to build secure production grade systems.

CRob (08:29)
Especially if you are creating software with the intention of somebody else using it, probably in a business, then you’re not really thinking about all the extra steps you need to take to help protect yourself in your downstream.

John (08:44)
Yeah, yeah. think it’s an evolution, right? And where I think of it like these AI systems we’re working with are maybe second graders. When it comes to professional code authoring, they can produce a lot of good stuff, right? It’s really up to the user to discern what’s usable.

And we can get to prototypes very quickly, which I think is greatly powerful, which lets us iterate and develop. In my company, we use AI coding techniques for everything, but nothing gets into production, into customer hands that isn’t highly vetted and highly reviewed. So, the creation part goes much faster. The review part is still a human.

CRob (09:33)
Well, that’s good. Human on the loop is important.

John (09:35)
It is.

CRob (09:36)
So let’s change the topic slightly. Let’s talk a little bit more about vulnerability management. From your perspective, thinking about traditional brick and mortar organizations, how have you seen, what key differences do you see from someone that is more data center, server, VM focused versus the new generation of cloud native where we have containers and cloud?

What are some of the differences you see in managing your security profile and your vulnerabilities there?

John (10:08)
Yeah, so I’ll start out by a general statement about vulnerability management. In general, the way I observe current methodologies today are pretty traditional.

It’s scan, it’s inventory – What do I have for software? Let’s just focus on software. What do I have? Do I know what it is or not? Do I have a full inventory of it? Then you scan it and you get a laundry list of vulnerabilities, some false positives, false negatives that you’re able to find. And then I’ve got this long list and the typical pattern there is now triage, which are more important than others and which can I explain away. And then there’s a cycle of remediation, hopefully, a lot of times not, that you’re cycling work back to the engineering organization or to whoever is in charge of doing the remediation. And this is a very big loop, mostly starting with and ending with still long lists of vulnerabilities that need to be addressed and risk managed, right? It doesn’t really matter if you’re doing VMs or traditional software or containerized software. That’s the status quo, I would say, for the average company doing vulnerability maintenance. And vulnerability management, the remediation part of that ends up being some fractional work, meaning you just don’t have time to get to it all mostly, and it becomes a big tax on the development team to fix it. Because in software, it’s very difficult for DevSec teams to fix it when it’s actually a coding problem in the end.

In traditional VM world, I’d say that the potential impact and the velocity at which those move compared to containerized environments, where you have

Kubernetes and other kinds of orchestration systems that can literally proliferate containers everywhere in a place where infrastructure as code is the norm. I just say that the risk surface in these containerized environments is much more vast and oftentimes less understood. Whereas traditional VMs still follow a pattern of pretty prescriptive way of deployment. So I think in the end, the more prolific you can be with deploying code, the more likely you’ll have this massive risk surface and containers are so portable and easy to produce that they’re everywhere. You can pull them down from Docker Hub and these things are full of vulnerabilities and they’re sitting on people’s desks.

They’re sitting in staging areas or sitting in production. So proliferation is vast. And I think that in conjunction with really high vulnerability reporting rates, really high code production rates, vast consumption of open source, and then exploits at AI speed, we’re seeing this kind of almost explosive moment in risk from vulnerability management.

CRob (13:18)
So there’s been, over the last several, like machine intelligence, which has now transformed into artificial intelligence. It’s been around for several decades, but it seems like most recently, the last four years, two years, it has been exponentially accelerating. We have this whole spectrum of things, AI, ML, LLM, GenAI, now we have Agentic and MCP servers.

So kind of looking at all these different technologies, what recommendations do you have for organizations that are looking to try to manage their vulnerabilities and potentially leveraging some of this new intelligence, these new capabilities?

John (13:58)
Yeah, it’s amazing at the rate of change of these kinds of things.

CRob (14:02)
It’s crazy.

John (14:03)
I think there’s a massively accelerating, kind of exponentially accelerating feedback loop because once you have LLMs that can do work, they can help you evolve the systems that they manifest faster and faster and faster. It’s a flywheel effect. And that is where we’re going to get all this leverage in LLMs. At Root, we build an agentic platform that does vulnerability patching at scale. We’re trying to achieve sort of an open source scale level of that.

And I only said that because I believe that rapidly, not just us, but from an industry perspective, we’re evolving to have the capabilities through agentic systems based on modern LLMs to be able to really understand and modify code at scale. There’s a lot of investment going in by all the major players, whether it’s Google or Anthropic or OpenAI to make these LLM systems really good at understanding and generating code. At the heart of most vulnerabilities today, it’s a coding problem. You have vulnerable code.

And so, we’ve been able to exploit the coding capabilities to turn it into an expert security engineer and maintainer of any software system. And so I think what we’re on the verge of is this, I’ll call it remediation revolution. I mentioned that the status quo is typically inventory, scan, list, triage, do your best. That’s a scan for us kind of, you know, I’ll call it, it’s a mode where mostly you’re just trying to get a comprehensive list of the vulnerabilities you have. It’s going to get flipped on its head with this kind of technique where it’s going to be just fix everything first. And there’ll be outliers. There’ll be things that are kind of technically impossible to fix for a while. For instance, it could be a disclosure, but you really don’t know how it works. You don’t have CWEs. You don’t have all the things yet. So you can’t really know yet.

That gap will close very quickly once you know what code base it’s in and you understand it maybe through a POC or something like that. But I think we’re gonna enter into the remediation revolution of vulnerability management where at least for third party open source code, most of it will be fixed – a priority.

Now, zero days will start to happen faster, there’ll be all the things and there’ll be a long tail on this and certainly probably things we can’t even imagine yet. But generally, I think vulnerability management as we know it will enter into this phase of fix first. And I think that’s really exciting because in the end it creates a lot of work for teams to manage those lists, to deal with the re-engineering cycle. It’s basically latent rework that you have to do. You don’t really know what’s coming. And I think that can go away, which is exciting because it frees up security practitioners and engineers to focus on, I’d say more meaningful problems, less toil problems. And that’s good for software.

CRob (17:08)
It’s good for the security engineers.

John (17:09)
Correct.

CRob (17:10)
It’s good for the developers.

John (17:11)
It’s really good for developers. I think generally the shift left revolution in software really didn’t work the way people thought. Shifting that work left, it has two major frictions. One is it’s shifting new work to the engineering teams who are already maximally busy.

CRob (17:29)
Correct.

John (17:29)
I didn’t have time to do a lot of other things when I was an engineer. And the second is software engineers aren’t security engineers. They really don’t like the work and maybe aren’t good at the work. And so what we really want is to not have that work land on their plate. I think we’re entering into an age where, and this is a general statement for software, where software as a service and the idea of shift left is really going to be replaced with I call shift out, which is if you can have an agentic system do the work for you, especially if it’s work that is toilsome and difficult, low value, or even just security maintenance, right? Like lot of this work is hard. It’s hard. That patching things is hard, especially for the engineer who doesn’t know the code. If you can make that work go away and make it secure and agents can do that for you, I think there’s higher value work for engineers to be doing.

CRob (18:24)
Well, and especially with the trend with open source, kind of where people are assembling composing apps instead of creating them whole cloth. It’s a very rare engineer indeed that’s going to understand every piece of code that’s in there.

John (18:37)
And they don’t. I don’t think it’s feasible. don’t know one except the folks who write node for instance, Node works internally. They don’t know. And if there’s a vulnerability down there, some of that stuff’s really esoteric. You have to know how that code works to fix it. As I said, luckily, agent existing LLM systems with agents kind of powering them or using or exploiting them are really good at understanding big code bases. They have like almost a perfect memory for how the code fits together. Humans don’t, and it takes a long time to learn this code.

CRob (19:11)
Yeah, absolutely. And I’ve been using leveraging AI in my practice is there are certain specific tasks AI does very well. It’s great at analyzing large pools of data and providing you lists and kind of pointers and hints. Not so great making it up by its own, but generally it’s the expert system. It’s nice to have a buddy there to assist you.

John (19:35)
It’s a pair programmer for me, and it’s a pair of data analysts for you, and that’s how you use it. I think that’s a perfect. We effectively have become cybernetic organisms. Our organic capabilities augmented with this really powerful tool. I think it’s going to keep getting more and more excellent at the tasks that we need offloaded.

CRob (19:54)
That’s great. As we’re wrapping up here, do you have any closing thoughts or a call to action for the audience?

John (20:02)
Call to action for the audience – I think it’s again, passion play for me, vulnerability management, security of open source. A couple of things. same. Again, same cloth – I think again, we’re entering an age where think security, vulnerability management can be disrupted. I think anyone who’s struggling with kind of high effort work and that never ending list helps on the way techniques you can do with open source projects and that can get you started. Just for instance, researching vulnerabilities. If you’re not using LLMs for that, you should start tomorrow. It is an amazing buddy for digging in and understanding how things work and what these exploits are and what these risks are. There are tooling like mine and others out there that you can use to really take a lot of effort away from vulnerability management. I’d say that for any open source maintainers out there, I think you can start using these programming tools as pair programmers and security analysts for you. And they’re pretty good. And if you just learn some prompting techniques, you can probably secure your code at a level that you hadn’t before. It’s pretty good at figuring out where your security weaknesses are and telling you what to do about them. I think just these things can probably enhance security open source tremendously.

CRob (24:40)
That would be amazing to help kind of offload some of that burden from our maintainers and let them work on that excellent…

John (21:46)
Threat modeling, for instance, they’re actually pretty good at it. Yeah. Which is amazing. So start using the tools and make them your friend. And even if you don’t want to use them as a pair of programmer, certainly use them as a adjunct SecOps engineer.

CRob (22:00)
Well, excellent. John from Root.io. I really appreciate you coming in here, sharing your vision and your wisdom with the audience. Thanks for showing up.

John (22:10)
Pleasure was mine. Thank you so much for having me.

CRob (22:12)
And thank you everybody. That is a wrap. Happy open sourcing everybody. We’ll talk to you soon.