Skip to main content

Register for The Linux Foundation Europe Roadshow

search
Tag

CRA

Sep 23
Love0

What’s in the SOSS? Podcast #40 – S2E17 From Manager to Open Source Security Pioneer: Kate Stewart’s Journey Through SBOM, Safety, and the Zephyr Project

By Podcast

Summary

In this episode of What’s in the SOSS, CRob has an inspiring conversation with Kate Stewart, a Linux Foundation veteran who took an unconventional path into open source as a manager rather than a developer, navigating complex legal challenges to get Motorola’s contributions upstream. Now a decade into her tenure at the Linux Foundation, Kate leads critical initiatives in safety-critical open source software, including the Zephyr RTOS project and ELISA, while being instrumental in the evolution of SPDX and Software Bill of Materials (SBOM). She breaks down the different types of SBOMs, explains how the Zephyr project became a security exemplar with gold-level OpenSSF badging, and shares practical insights on navigating the European Union’s Cyber Resilience Act (CRA). Whether you’re interested in embedded systems, security best practices, or the evolving regulatory landscape for open source, this episode offers valuable perspectives from someone who’s been shaping these conversations for years.

Conversation Highlights

00:00 Intro Music & Promo Clip
00:00 Introduction and Welcome
00:42 Kate’s Current Work at Linux Foundation
02:18 Origin Story: From Motorola Manager to Open Source Advocate
06:38 Building Global Open Source Teams and SPDX Beginnings
09:45 The Variety of Open Source Contributors
10:57 Deep Dive: What is an SBOM and Why It Matters
17:05 The Evolution of SBOM Types and Academic Understanding
19:21 Cyber Resilience Act and Zephyr as a Security Exemplar
26:46 Zephyr’s Security Journey: From Badging to CNA Status
31:05 Rapid Fire Questions
32:19 Advice for Newcomers and Closing Thoughts

Transcript

Intro Music + Promo Clip (00:00)

CRob (00:07.862)
Welcome, welcome, welcome to “What’s in the SOSS?” the OpenSSF’s podcast where we talked to the amazing people that help produce, manage and advocate this amazing movement we call open source software. Today we have a real treat – someone that I’ve been interacting with off and on over the years through many kind of industry ecosystem spanning things. So I would like to have my friend, Kate Stewart, give us a little introduction, talk about yourself.

Kate Stewart (00:42.103)
Rob, glad to be here. Right now, I work at the Linux Foundation and have been focusing on what it’s going to take to actually be able to use open source in safety critical contexts. So obviously, security is going to be a part of that. We need to make sure that there’s no vulnerabilities and things like that. But it does go beyond that. And so that’s been my focus for the last few years. I’ve been working on a couple of open source projects, which is Zephyr and the other of which is Elisa and then helping with a variety of other embedded projects like Yocto and Zan and so forth. Trying to figure out what can we do to make sure that we actually get to the stage where we can do safety analysis at scale with vulnerabilities happening all the time with open source. And it’s a really good challenge. I’ve also been involved in SBOM World a fair amount over the years for fun. I think I was involved with the SBOM World before it was called the SBOM World actually with a project called SPDX, or software package data exchange is what it’s called now, and it’s now moved to being system package data exchange because data and models and all these other lovely concepts are part of the whole transparency that we’re going to need. And the transparency that we’re going to need is what we’re going to need for safety. So these things also come together with that theme in mind.

CRob (02:02.702)
Awesome, that’s an important space that we haven’t had a lot of folks talk about. Safety is just as important, if not more so, when we’re thinking about security and just kind of protecting people. So let’s dive into your background. What’s your open source origin story, Kate? How did you get into this crazy space?

Kate Stewart (02:18.967)
Well, I’m a little bit different than most. Okay. I got into this not as a developer. I got into this as a manager. Okay. So I basically was managing a team of developers doing bring up software back in the Motorola days. I was about 20 plus years ago now. And Apple had just finished pivoting away from the PowerPC. And so we needed to be able to prove that what we were doing in the silicon was actually going to work. And so this is part of embedded story and the enablement story to me. But what I ended up having to do is we went and looked at Linux. We went, OK, yeah, let’s work with Linux. Let’s work with the GCC tool chains, and let’s use that as our enablement platform. And then I had to, you know, we had our customers saying, OK, that’s fine, but until it’s upstream, we don’t believe it. So it was one of those sorts of ones, right?

It doesn’t, your port doesn’t exist unless it’s upstream as far as they were concerned. So we could get it running in our labs, but it’s all of a sudden I had to go and deal with the lawyers and figure out how can we get it so that we can actually contribute upstream. And I pretty much was doing, basically I went with the lawyers and when I was at Motorola and worked with them, convinced them, educated them, know, education, all that to get them so that they could understand, we’re not going to take a lot of risk.

CRob (03:18.866)
Mm-hmm.

Kate Stewart (03:47.313)
And it’s in our best interest to sell our chips. So we got that happening. And then the company spun out FreeScale for Motorola. And I got to do it all over again with a new set of lawyers. So I was managing the open source teams. And then we did a lot of work with the team in Austin. And then we started scaling that out worldwide to all the other open source teams in Motorola, or FreeScale at the time then, and managing teams.

CRob (04:00.05)
Hahaha

Kate Stewart (04:16.363)
Teams in China, teams in Israel, teams in Canada, and gosh, Japan and France. Anyhow, we had a good selection of teams around the world. And so making sure that we could actually do all this properly at scale was an interesting challenge. So that was my start in open source. And this is why you sort of see me in the licensing space, because I’ve been talking to lawyers a lot. And that’s kind of where SPDX started, is because we had to keep doing the same metadata over and over and over.

And so my colleagues at Wind River were looking at the same stuff. My colleagues at Monovista were looking at the same stuff. We had no way of collaborating. So it was a language to collaborate with. that if I go and scrub this curl package and pull all the licensing information out and sanitize it, I could share it with you type of deal and vice versa. So that’s kind of how SPVX started with being able to share the metadata about projects and make things transparent so that we could do the risk analysis properly. After that. you know, I basically went and got an opportunity to join Canonical and I was a Buntus release manager for two and a half years. So all of a sudden that was a whole different view of open source, right? I was coming from a nice little embedded space and at Canonical, I learned all about dependencies and all about how you had to make sure that your full dependency chain was actually satisfied and coherent.

CRob (05:23.664)
Nice.

Kate Stewart (05:39.447)
Or you were going to have problems. I also learned a lot about zero days and making the whole security story come together so that we could ship a distribution that people were coming, that was not going to cause problems for people. And so there’s about five, a bunch of releases I was released management for in that time period. And so that taught me a lot about open source at scale in the current environment. And after that, I went to, I was doing the roadmaps at Linaro.

CRob (05:51.43)
Mm-hmm.

Kate Stewart (06:09.399)
Product management, was the director of product management at Lenaro to figure out where did the army ecosystem want to collaborate? What topics do want to collaborate on? And so I was there for a couple of years and then I joined the Linux Foundation in 2015. So this will be my 10th year and it’s been a fun night. I know. So I’ve been, yeah, yeah, it is. they brought me in because the lawyers at the Linux Foundation knew me from the SPDX world. But after I joined, they sort of they had me point to a certain problem initially when I joined and we figured out a solution. But then they realized, oh, she understands embedded. Ooh. So they basically asked me to pull together the funding pool for real-time Linux. And I said to them that that was a big problem. And now, as of last year, we finally have everything upstream. It’s only taken eight years, but we finally got it. We preempt our taste at patches. And then after I was able to get that going well, went, hmm, we’ve got this RTOS called Zephyr that Intel is contributing up and we need someone to run it and start to build a program up. And so it was sort of like, okay, let’s figure out how we can deal with this one. So it was fun. We did a lot of surveys with developers, what were the big pain points in the IoT ecosystem? Back in 2015, I don’t know if you remember, but the big joke was what’s the S in IoT? Yeah, exactly. And so…

Focusing on security in Zephyr has been the focus pretty much since the day we launched the project. We had our first security committee meetings. And every time we found a new security best practice, we did try to apply it into Zephyr and to see where we are. I think we were the fifth project to get a gold badge out of the OpenSSF Badging Program. I think there’s a few more now, but not that many. We have gone to that level.

Kate Stewart (08:02.417)
And certainly Zephyr was there before Linux was there, just as side thing. So we actually got it.

CRob (08:08.274)
I’ll tell Greg next time I see him.

Kate Stewart
He knows it. I’ve teased him for many years. So nothing new there. But yeah, so we were trying to do best practices in Zephyr. And we’ve been working towards safety in Zephyr. Now it’s taken us a while to figure out a path that would work in the context of open source. But these were the two things that we were told back in 2015 by the developers that they wanted to see in open source Arctos. And so that’s what we’ve been focusing on. the project has grown really well over the years. We’re now the fifth most active project at the Linux Foundation in terms of project velocity, especially by CNCF, not us. So we have some degree of separation there.

And we’ve actually hit the top 30 of all open source projects now, so we’re the 25th. So it’s had a pretty good trajectory. And this just goes to show, if you try to do the best practices, it makes a project successful. And developers want to be there because they can build on it. So that’s kind of the origin story of where I am to today. I’ve been starting to work on it. I continue to work on Linux-related topics with the safety, with the Elisa project.

Kate Stewart (09:20.331)
And we’re busy trying to figure out similar paths, how we can go for certifications with Linux. So that’s been growing slowly. But both of these projects are ones that grow very slowly over time. And they just sort of creep up bit by bit by bit by step by step by step. And I’ve got some great, yeah, exactly, pedal by pedal by pedal. I’ve got some great board members in both projects who are very much engaged and have been doing a lot to help us move it forward. So I’ve been very lucky in that sense too. Yeah.

CRob (09:45.362)
Awesome.

That’s an amazing journey that you’ve described and touching on so many interesting different areas. I love like through these interviews, I get to hear how people got here and the variety of skills that people bring that you’re able to contribute very meaningfully to upstream is just amazing. It really makes me happy to hear that you have again, kind of a non-traditional open source background. That’s great.

Kate Stewart (10:09.729)
Yeah, like I say, whenever the academics go do surveys about open source origins, I always enjoyed being able to do that to say, it isn’t just developers that do open source and help open source. Realistically, it’s a community. And so it’s, know, everyone has different roles and different abilities to contribute in different ways. And as we all want to go after the same vision, the right thing happens. It’s pretty much what happened. It’s pretty much what I’ve seen anyhow.

CRob (10:31.89)
So where I first got to interact with you way back in days of yore was on this little thing that you alluded to, this small little thing called Software Bill of Materials. I joined the second NTIA call and you were one of the presenters in the room. So for our audience, could you maybe give me an elevator pitch on what’s an SBOM and why is it important?

Kate Stewart (10:57.835)
Well, so Software Bill of Materials is a way of exposing the metadata about the software you’re using in a transparent fashion. And so it’s basically putting in together the key elements of what are your components and how are they linked together? What’s the relationships between them? And understanding these relationships and how these components interact is what gives you the ability to decide if something is potentially at risk or not. Is it vulnerable or is it not vulnerable? What are the transitive dependencies? What’s happening? Realistically, there was a lot of simplifications that were made at that point in time in the initial S-BOM work. They’re starting to come back to bite us now. Just saying, one of which was the only thing was includes as a dependency. Well, realistically,

Kate Stewart (11:53.067)
You need to know something statically linked is something dynamically linked. How are these, how are these things interacting? and we’ve got, you know, things emerging right now in the whole AI front of, know, what training data we’re using, you know, what model on your models and how is this all assembled? And so these are things that we’d actually dealt with in SPDX a long time ago, but the SBOM community wasn’t ready to talk about it at that time. They’re starting to move in this direction now, but you’ll find, like I say, I’m having a lot of fun right now reading some academic papers, because I’ll be talking about some of these topics at the end of the month up in Ottawa at the MSR. So I’ll be doing a keynote there about some of this stuff. And I was looking at all these academic papers to see, OK, what do they think an S-BOM is, just so that I figure out where I have to talk to them about this stuff.

Kate Stewart (12:52.263)
One of the things that they’re seeing is that there’s a lot of the interpretations and the subtleties around the S-BOM is this data of components and the relationship between these components is very much like an elephant. It depends on which part of the elephant you’re coming in at it from and what you’re feeling with with your little line hands as to what you perceive it as.

Kate Stewart (13:18.999)
And we went through the, I guess it was in the CISA working group. I was a fly on the wall on the first one and opened my mouth, which is why I was on the second NTIA call. And that’s why I basically took over co-leading the formats and tooling working group under NTIA. And I was an active participant in the framing discussions. So I’ve been pretty much involved with it all the way through. then I was working when we first had the the SBOM Everywhere. So we had the SBOM Working thing in Washington. I was there. And that was sort of the start of figuring out, okay, well, we want to have the SBOM Everywhere sync to start focusing issues. And so when there was a hiatus between NTIA and CISA, that SBOM Everywhere group was keeping the discussion going in a reasonably collective way. And we’re sort of starting to head into that again with the NTIA SBOM Everywhere, pulling voices together, and understanding how this technology is evolving and where the strengths and weaknesses are and where the gaps are, filling the gaps. So I’ve been involved with the OpenSSSF, OpenSSF, SBOM Everywhere SIG under the tooling group, basically with Josh Breshers since it started, trying to make sure that we had the different voices coming in and we had the different perspectives available so that we could start to get a good lie of the land. And we started work on clarifying what is actually happening with the types of SBOMs? Because the type of data you have available for source SBOMs is what historically the legal focus focused on for license compliance, but still useful for the safety people. But then you have the build SBOMs where you say, OK, these are the pieces I’m putting together. And the question is, you capturing, these are all the source files that went into my build image, or are you just capturing these are the components that go into my build image?

Kate Stewart (15:14.559)
But these are build types of SBOMs. And they have different purposes, and they have different ability to reduce false positives. Specifically, if you don’t compile a piece of code into your package, you’re not vulnerable to it. there’s a lot of, I’d like to get rid of a whole bunch of these false positives in the security space. I really think we should be able to, if we actually get the right level of information. So we can take that and then what you actually configure may impact whether you have vulnerability or not when you deploy it. So what have you released and where are you deploying? And then what’s happening to the system as it’s running? Are you updating your libraries? Are there potentially interactions between your runtime libraries and what you put down as images? All these are different types of data that can legitimately be put into an SBOM. And there’s different levels of trust depending on where you are in the ecosystem, how much you actually how much the people who have put the data into the format actually understand the data and have confidence. Because there’s a lot of tools out there, which is the sixth type, which is an analysis SBOM. And these are ones that are looking at a different part of the life cycle or off to the side and trying to guess what’s going on. And the guessing what’s going on is if you don’t have anything else, that’s what you need to do. No question. But if you can get it precisely from the build flows and from your whole ecosystem as it’s being used, as it’s being deployed, as it’s being monitored, you’re going to have a lot more accuracy, which removes a lot of problems. So that type of concept that we sort of picked up on in OpenSSF, Seth Swansegan, has been picked up by CISUN. And we’ve got a short white paper out about the same thing. That concept hasn’t hit academia yet.

CRob (17:03.858)
We better get that over there.

Kate Stewart (17:05.55)
Uh-huh. so this is I’ll be having some fun. So the whole concept of the different data and like they’re trying to look at like all these SBOMs everywhere and how do you work with them, et cetera. And it’s sort of like, well, first question one is how authoritative is the people who produce this SBOM on you? Is it garbage in? Is it someone just filling in the fields and not really understanding what they’re filling in?

CRob (17:28.676)
Mm-hmm.

Kate Stewart (17:31.797)
Or does it come from a source that you trust and that you think actually knows what you’re talking about so you can build on it further and augment it and everything else? So yeah, these are the challenges I think that are kind of interesting for us to be playing with right now. And so I was part of, in SysI, working on the basically the tool quality was basically we were working on looking at tooling and looking at the formats and actually ended up doing a revision to the revision of the framing document. So we basically pulled together consensus on what the next additional sets of fields were going to be that got added into the tree. And then I think CISA has plans, or at least had plans, we’ll find out what happens this year, of basically going through a more formal process. But they’ve got the stakeholder input from us now. And that was the stage of where we thought, and realistically, the lawyers managed to show up in those meetings and convince them that, they did need to keep licenses in there now. Thank you. Because it’s another form of risk at the end of the day. It’s just risk forms. And so people creating these things and doing policies, the academics are busy trying to study this because it’s an interesting area for them. And so I think we’ll see a lot of innovation in the next few years and getting towards a stage where we can actually do product lines at scale.

Um, and being able to keep things safe, I think is where we need to aim for the whole SBOM initiatives and they need to be system-wide. They can’t just be software anymore. You need to know which data sets you’ve used because you can have poison data sets all over the place. can have, um, you know, bad training on your models might have an impact if you’ve not weighted things properly. These are all factors that when people are trying to understand what went wrong, they’re going to need to be able to look at.

CRob (19:21.458)
What I love about my upstream work is that I get to collaborate with amazing people like you on these really hard challenges and know software bill of materials has been won. Let’s move on to another interesting problem we all get to work with. The European Union last year released a new piece of legislation, the CRA, the Cyber Resilience Act.

Kate Stewart (19:45.565)
Yes.

CRob (19:47.43)
And it’s a set of requirements for predominantly manufacturers, but also a little bit for open source stewards that talks about the requirements for products with digital elements. through our parallel work, we worked with our, with Hillary Carter and the LF research team. And we did two study, we sponsored two studies. The first study, which was very sad news where we polled 600 people participating in upstream open source, whether that was a manufacturer, a developer, a steward like us. And the findings there were a little scary, where a lot of people are unaware and uncertain is the title of the paper. People don’t know about this law and the requirements that are coming down the pipe. But in concert with that, LF Research also released a paper that has, I feel, is some pretty spectacular news. The title of the other paper was Pathways to Cybersecurity, Best Practices in Open Source, and a project that’s near and dear to your heart was cited as one of the exemplars of an upstream project that takes security seriously and is doing the right types of things. So could you maybe talk a little bit about Zephyr and talk about like what your personal observations of what this report kind of detailed?

Kate Stewart (21:13.749)
Yeah, we’re going to have a gap. Also, there’s a lot of things that some of these projects have done right. Zephyr, for one, like I say, we’ve taken security seriously right since the project started. Literally, the first security committee meeting happened the day after we launched the project. it’s in our blood that we make sure that we’ve tried to do this right. We’ve been, you know, we started trying to figure out, and was funny enough, was part of the Core Infrastructure Initiative Open Source Badging started at that point in time roughly as well, that’s us. And so we looked at it going like, well, how do we improve our security posture as a project? Because we were coming at this from a community and we started using the badging program as looking at the criteria, filling it out, understanding how this all works. And we initially got to passing.

Yay, we got it. We’ve got our basic passing. And then David Wheeler got busy with his community, and he came up with a silver and a gold level on us. And we just kept increasing the resilience of the project at the heart of it all. So this was also really quite educational. So we started looking at that. And as we were going for that first passing, we realized, it. What’s involved with becoming a CVE, numbering authority, or CNA?

Kate Stewart (22:37.399)
And so I started working and reaching out to the folks behind that and understanding what’s involved. And so we ended up having myself and some of the security committee, we ended up meeting with them trying to figure out, okay, well, what does it take for project to be a CNA, not a company, but a project? And so in, think 2017, 2018, we actually got our CNA criteria. We fulfilled the CNA criteria and the project itself has been a CNA with a functioning P-SIRT.

CRob (23:06.066)
Nice.

Kate Stewart (23:06.557)
We’ve been plugged into the whole P-SIRT community and the first community and everything else for the last few years. And last year at first conference, I actually gave a talk about Zephyr because a lot of the corporation folks don’t realize that projects can do this too, if they have the will and the initiative behind their membership and their developers that they want to be this way. we’ve been looking at these things and tackling that.

CRob (23:14.514)
Excellent.

Kate Stewart (23:36.311)
And then we went after, we were sort of sitting on our laurels to a certain extent before we started really going after the silver. And the automation behind that badge kicked us out because some of the website lights weren’t working anymore. Yeah. So I went, oh, it kicked us out. We’ll take it seriously now. OK. This is good. It wasn’t just a paper exercise. There’s actually something that’s keeping you honest there behind the scenes. And that checking behind the scenes motivated us then to go after silver and then gold, and we finally got gold. then, so I can say we were about the fifth, I think the fifth to get gold, fourth or the fifth. I was really, you we were pretty happy. And we’ve maintained it since then. And every year we do a periodic audit of the materials. We started looking at the scorecard practices as a project. And so we started looking at last year and the security committee, actually, this is the amusing part here. The security committee was going, ah, this really doesn’t apply to people building things with Zephyr, et cetera, et cetera. And we were going like, oh, well, OK.

Kate Stewart (24:33.761)
And then when we had this little recent incident with that exposure, all of a sudden we’re talking on weekends and they realized, shit, we better take this all seriously. So we’re improving our posture there too now. So I think we went like from 50 something up to 78 and we’ve got plans for getting ourselves up to the hundred level type of deal. So we’re continuing to improve our posture and work on that sort of stuff there. So Zephyr takes security very, very seriously as you can tell.

CRob (24:52.914)
Excellent.

Kate Stewart (25:03.031)
It really is, you know, part of the reason I think we’ve been a successful project is the fact that we do have a strong security story. We’ve got our threat models. We’ve got, know, we’ve got the things that people are saying to be looking at and we keep putting them and we have, people that meet from our members and from the community and, you know, continue to refine our posture. So I think it’ll always be that way. And so whenever I can find a new practice, we’re starting to look at, can we apply it? How does it match to what we’ve already done?

And so I’m curious when the baseline stuff rolls out, what is it going to look like? And I suspect we’ve already hit there, but we might learn something. That’s always good. And the nice thing about doing it this way, which was applying the best practices badging and forth, let us put things in place. They’re serving us well for the CRA. And that’s probably why we showed up in this report, because

You know, lot of the things that we do with the US government, once the Europeans figure out, what database are they going to be throwing things at? Who is going to be the C-CERTS? We’ve been doing it already on the US side. We should be able to do it on the European side. It’s just a matter of figuring it out a little bit. Just basically, testing a few of our processes. But we’ve already been doing these types of things in one direction. We just have to broaden the reach a little bit more.

CRob (26:26.651)
Yeah, I think it’ll be a pretty easy pivot. You need to make some small adjustments, do some documentation potentially, but it sounds like you’ll be in a pretty good spot to fulfill that any obligations underneath that anyone that uses Zephyr will be in a great space to defend themselves like the Market Surveillance Authority or other groups.

Kate Stewart (26:46.037)
Right. And, you know, one of the things that the project did when we started looking at the criteria coming down, and the needs of the criteria was it was looking like, okay, they want to have this, this longer support period, like, you know, between five years up to 25 type of deer. And so the separate projects TSC actually in March voted to extend our LTS support from 2.5 years to five years. And we’ve got that we’re doing different things in different periods of time.

CRob (27:12.551)
Yay!

Kate Stewart (27:15.863)
Okay, so what we’re doing traditionally now for the first two years, two and a half years, and then we’re basically just focusing on security and critical updates after that. So it keeps the load reasonable for our maintainers and our release team members. And so that’s kind of how we’ve attracted. So, yeah, the TSE voted on it, it was approved. And so that’s our bit we could do to help shrink the gap between the steward obligations and the manufacturer obligations. So we’re trying to make it.

Kate Stewart (27:45.013)
Zephyr is friendly as possible for the manufacturers. Now we are going to have a challenge that I don’t see a clean story for yet. We went through some bulk vulnerability assessments and things like that, and we’ve changed some processes. And what we do is we have documented in the project, we will respond to any vulnerability reports in seven days. We will basically, you know, we’ll acknowledge them, you know, start working with whoever’s reporting things in, and then we will in 30 days have it fixed upstream.

And then in 60, we’ll have another 60 day embargo window. total 90 days of embargo window before we make it public. And we do this because in the embedded space, you know, it’s a lot harder sometimes to remediate things in the field. And so we wanted to make sure that, you know, the people who are trusting Zephyr as their RTOS, you know, would have a chance to work with their customers. Now the CRA, you know, and so the CRA is asking for, especially in severe vulnerabilities, some pretty different timelines. And so I’m really going to be interested in how that’s all going to boil itself out. The other thing that’s interesting is that the CRA is calling for us to notify our customers. Well, we don’t know who’s using Zephyr. So one of the mechanisms we put in place earlier, which was a product, like basically a vulnerability notification list for our product makers.

Kate Stewart (29:12.887)
So any of our members in the project or any people who can show us that they’re using a product that’s got Zephyr in it, we’ll add them to the notification list under embargo. And so we’re trying to handle it that way. But that’s going to be the best we’re going to able to do. We won’t be able to find this end user because we don’t have the line of sight. And now the cut, the manufacturer.

CRob (29:32.028)
But exactly, and that’s not necessarily your responsibility is the upstream. That’s the manufacturer somebody that’s commercializing this.

Kate Stewart (29:36.317)
Well, it’s one of those sections that applies a little bit to the stewards.

CRob (29:45.81)
It does, but the timelines are not identical.

Kate Stewart (29:47.147)
So that’s why we’re always going to do to whoever wants to let us know about them. Yeah, let us know about them. I think sanity will prevail, and we will not be subject to various other, some of the punitive stuff. But I think we can certainly look at trying to make sure that what we’ve done is as much as we can do with the data we have available to us. And hence, the more transparency we make into this ecosystem, the better. Speaking back to S-BOMs, you know, Zephyr actually, every build of Zephyr, you can get three SBOMs out with just a couple command line tweaks. You get sources SBOM of the Zephyr sources from the files you pulled from. Of course, the sources from the application you’ve used. So you get source SBOMs for each of those. And then you get a build SBOM that links back with cryptographic encryption to the source and to the, to the, to the source SBOMs and lets you know exactly which files made it in as well as which, you know, dependency links and components you may have pulled from your ecosystem. But that level of transparency is what we’re going to need for safety. And we have it today with Zephyr from following these best practices on security. So we’re in reasonable shape, I think, for the regulatory industries as well.

CRob (31:05.414)
Good. Well, I could talk about SBOM and CRA all day, but let’s move on to the rapid fire part of our interview.

Kate Stewart (31:14.134)
Okay.

CRob (31:17.66)
got a bunch of questions here. Just give me your first emotional response. First thing that comes to mind here. First question, VI or EMACs.

Kate Stewart (31:28.599)
VI.

CRob (31:30.546)
Nice, very good, very good. These are all potentially contentious somewhat, so don’t feel bad about having an opinion. What’s your favorite open source mascot?

Kate Stewart (31:34.638)
Tux

CRob (31:45.106)
Excellent choice. What’s your favorite vegetable?

Kate Stewart (31:50.625)
Carrots.

CRob (31:52.434)
Very nice. Star Trek or Star Wars?

Kate Stewart (31:58.455)
Star Trek.

CRob (32:00.541)
Very good. There’s a pattern. So everyone I’ve asked that so far has been trekkers, which is good. And finally, mild or spicy food.

Kate Stewart (32:09.611)
Depends. Probably, at this point now more mild. There’s certain things I like spicy though.

CRob (32:14.578)
Ha

CRob (32:19.42)
Fair enough. Well, thank you for playing along there. And as we wind down here, Kate, do you have any call to action or any advice to someone trying to get into this amazing upstream ecosystem?

Kate Stewart (32:33.303)
There’s lots of free training out there. Just start taking it. Educate yourself so that you can participate in the dialogue and not feel completely overwhelmed. Each domain has its own set of jargon, be it lawyers, licensing jargon, security professionals with their jargon, safety folks with all their standards jargon. Everyone talks about certain concepts a little bit differently. so taking the free training that’s available from the Linux Foundation and other places just so that you’re aware of these concepts. actually, before you start opining on some of these things, actually do your homework. I know that’s a horrible concept, but like I said, was reading on some of these papers, these academic papers, and it was pretty clear that they hadn’t done their homework in a couple of areas. So that was a bit sort of like, yeah, OK. So yeah, do your homework before you start to opine. But do your homework. Educate yourself. Do your homework. And then find the areas that are most interesting to you, because there’s so many areas where people need help these days and there’s a lot of things we can participate in. And you don’t need to be a developer to participate as well. Obviously developers make everything come together and make it all work, but there’s a need for people doing a lot of other tasks. And if you want to try and make things move forward, there’s lots of ways.

CRob (33:49.586)
Well, thank you for sharing your story and thank you for all the amazing contributions you’re making to the ecosystem. Kate Stewart, thank you for showing up today.

Kate Stewart (33:56.919)
Thank you very much for all the work you’re doing in OpenSSF. I think this is really important. Thank you.

CRob (34:03.718)
My pleasure. And to everyone out there, thank you for listening and happy open sourcing. We’ll talk soon. Cheers.

Jul 30
Love0

OpenSSF Newsletter – July 2025

By Newsletter

Welcome to the July 2025 edition of the OpenSSF Newsletter! Here’s a roundup of the latest developments, key events, and upcoming opportunities in the Open Source Security community.

TL;DR:

Submit Your Proposal: OpenSSF Community Day Korea

The Call for Proposals for OpenSSF Community Day Korea is closing Aug 3! If you have insights, tools, research, or community stories to share around open source software security, now is the time to submit your talk. The event takes place on November 4, 2025, in Seoul, South Korea, and brings together developers, researchers, and security professionals from across the open source and security ecosystems.

Whether your focus is on AI and security, vulnerability management, education, or tooling, we welcome submissions in a variety of formats, from quick 5-minute talks to extended 20-minute sessions. Deadline to submit: August 3, 2025, at 23:59 KST / 06:59 PST.

Share your expertise and help shape the future of open source security. We look forward to seeing you in Seoul!

Blogs:

New: Cyber Resilience Act (CRA) Brief Guide for OSS Developers

In our recent blog post, David A. Wheeler introduces the Cyber Resilience Act (CRA) Brief Guide for OSS Developers, a practical overview created by the OpenSSF to help open source developers understand and prepare for the EU’s new cybersecurity regulation. Although the CRA officially applies only within the EU, its global impact is significant due to the international nature of software distribution. The blog clarifies when the CRA does or does not apply to OSS, outlines potential risks for non-compliance, and highlights available resources including free training and community support to help developers build secure, compliant software. Read the full blog.

Recap: OpenSSF Community Day Japan 2025

OpenSSF Community Day Japan 2025 brought together developers, researchers, government, and industry leaders in Tokyo to advance open source software security. The event featured keynotes, technical sessions, and a live incident response exercise focused on secure development, tool adoption, and supply chain integrity.

Read the full blog for session videos, slides, and key takeaways.

Recap: OpenSSF Community Day North America 2025

OpenSSF Community Day NA 2025 brought together a diverse open source security community in Denver for a packed day of insights, tools, and collaboration. From real-world deployments of SBOM, Sigstore, and GUAC to securing AI pipelines and exploring the new AStRA control plane framework, sessions moved beyond awareness into action. 

Read the full blog for recordings, slides, key takeaways and ways to get involved.

On-Demand Webinar: Cybersecurity Skills, Simplified

The on-demand webinar Cybersecurity Skills, Simplified: A Framework That Works brings together experts from IBM, Intel, Linux Foundation Education, and OpenSSF to address a critical challenge: making cybersecurity a shared responsibility across all roles. The panel introduces the Cybersecurity Skills Framework, an open, flexible tool that helps teams identify, map, and improve security skills organization-wide. With insights on setting security OKRs, scaling training, and creating accessible learning pathways, this webinar offers practical guidance for anyone looking to strengthen their team’s security posture. Learn more.

What’s in the SOSS? An OpenSSF Podcast:

#35 – S2E12 Building India’s Open Source Security Community: From Developer Nation to Security Champions

In this episode of What’s in the SOSS?, host CRob sits down with Ram Iyengar, OpenSSF’s India community representative, to explore the evolving landscape of open source security in India. Ram shares his journey from professor to evangelist, the launch of LF India, and the challenges of inspiring a security-first mindset in one of the world’s largest developer populations. The episode covers everything from building local community momentum to hosting regional events and video series, offering listeners both practical insights and a personal look at the passionate effort behind India’s growing open source security movement.

#34 – S2E11 From Lockpicking to Leadership: Tabatha DiDomenico on Security, Open Source, and Building Community

In this episode of What’s in the SOSS? host Yesenia Yser sits down with Tabatha DiDomenico, open source security engineer, community leader, and president of BSides Orlando for a compelling conversation about her unconventional path into open source, the power of community, and the often-overlooked impact of DevRel. From her first experience with Netscape to shaping security strategy at G-Research and OpenSSF, Tabatha reflects on how curiosity, volunteering, and intentional advocacy have fueled her journey. Whether you are new to open source or a longtime contributor, this episode offers heartfelt insights, practical advice, and a powerful reminder: community is everything.

Education:

The Open Source Security Foundation (OpenSSF), together with Linux Foundation Education, provides a selection of free e-learning courses to help the open source community build stronger software security expertise. Learners can earn digital badges by completing offerings such as:

These are just a few of the many courses available for developers, managers, and decision-makers aiming to integrate security throughout the software development lifecycle.

News from OpenSSF Community Meetings and Projects:

  • The Security-Focused Guide for AI Code Assistant Instructions that is being developed by the Best practices and the AI/ML WGs is now in final draft, under PR here.
  • Zarf released version v0.58.0 including image push & pull and SDK enhancements.
  • OpenBao recently released v2.3.1 with support for namespaces, CEL for JWT authentication and PKI issuance, and SSH multi-issuer support. The community is making progress on per-namespace sealing, HSM/KMS backed key material, and horizontal scalability, and just kicked off a UI working group.

In the News:

Meet OpenSSF at These Upcoming Events!

Join us at OpenSSF Community Day Events in India, Japan, Korea and Europe!

OpenSSF Community Days bring together security and open source experts to drive innovation in software security.

Connect with the OpenSSF Community at these key events:

Ways to Participate:

There are a number of ways for individuals and organizations to participate in OpenSSF. Learn more here.

You’re invited to…

See You Next Month! 

We want to get you the information you most want to see in your inbox. Missed our previous newsletters? Read here! Have ideas or suggestions for next month’s newsletter about the OpenSSF? Let us know at marketing@openssf.org, and see you next month! 

Regards,

The OpenSSF Team

Jul 29
Love0

What’s in the SOSS? Podcast #36 – S2E13 From Compliance to Community: Meeting CRA Requirements Together

By EU Cyber Resilience Act, Podcast

Summary

In this episode of ‘What’s in the SOSS” CRob dives deep into the Erlang ecosystem with Jonatan Männchen (CISO, Erlang Ecosystem Foundation), Ulf Riehm (Product Owner, Herrmann Ultraschall), and Michael Winser (Alpha-Omega). This episode explores the critical importance of security in open source, particularly in light of regulations like the CRA. Hear how the Erlang community is proactively addressing security concerns by bringing in experts, fostering collaboration, and building trust. Discover why manufacturers are investing in upstream projects and how other ecosystems can learn from their approach. This conversation highlights the value of community, transparency, and the essential role of ‘stewards’ in the open source world.

Conversation Highlights

00:00 Welcome
00:57 Meet the Guests
02:56 Jonatan’s Journey into Erlang
06:16 The Alpha-Omega Connection
09:07 Ulf’s Perspective as a Product Manager
13:09 Funding Security in Open Source
18:58 Challenges in Implementing Security
24:54 Becoming a CNA and Normalizing Security
28:18 Jonatan’s role as CISO
32:01 Calls to Action & Advice
36:49 Wrap Up

Transcript

CRob (00:14)
Welcome, welcome, welcome to What’s in the SOSS, the OpenSSF’s podcast where we meet interesting people that are in and around the upstream open source ecosystem. My name’s CRob. I’m the chief security architect for the foundation, and I also do security stuff upstream to help protect that open source software we all know and love. And today I have an amazing collection of gentlemen here, and we’re talking about a very important topic. It’s about the value of bringing experts in.

So I would like to pass the microphone around. I’ll start off with Jonatan. Let’s introduce ourselves and kind of talk about what brought you here today to talk about this interesting topic.

Jonatan Männchen (00:57)
Yeah. Hi, I’m Jonathan Männchen. I’m the Chief Information Security Officer at the Erlang Ecosystem Foundation. And the reason I’m here today is that we’ve started implementing a lot of functionality in the security and in the compliance sector, mostly focused on the CRA. And based on that, I’ve met CRob and Michael, these lovely gentlemen in the Alpha and Omega call and was invited to come here and talk about it all.

CRob (01:31)
Ulf

Ulf (01:33)
Yeah, I’m a product owner with Herman Ultrasonics. We are a German machine builder, like a small company, 500 people only, not one of the big tech companies. And we have decided, arbitrary for a weird Swedish tech stack, including Erlang, to do our automation, to do our machine controls. And as a product owner, I had to make decisions whether how we would tackle security in the longer run. And that brought me here.

CRob (02:09)
Excellent. And our friend, Mr. Windsor.

Michael Winser (02:12)
Hi everyone. So I’m here for the free cookies. I was promised cookies. I think, you know, working in Alpha Omega, one of the surprising and the continuous benefits is that we end up finding community. find people and people find us and then that creates these connections. And so when Jonathan showed up in one of the public meetings and started chatting, I’m like, who are you? What are you doing? And we started talking more and that sort of led to more conversations and we’re still talking about things. that has spread to other parts of the airline community as well. And so the learnings continue. And for me, that’s just, it’s amazing what happens when you put people in a room and start talking together. So now here’s another room, let’s talk.

CRob (02:56)
Excellent. let’s start off. Jonathan, you’re here representing Erlang. Could you maybe talk to us about how you got into open source and maybe talk a little bit about what Erlang is all about?

Jonatan Männchen (02:56)
Mm-hmm. I think I started out quite the normal route, let’s say, just doing some side stuff from my corporate job, essentially. And as these things normally go, you kind of feel responsible for them and they grow and you get more and more of these kind of side projects going on. Some of them getting successful, others you decide to cut the loss at some point. And…

Yeah, I really started in the PHP ecosystem a long time ago, doing some pull requests on Symfony. And I published a library that does a SIP streaming from the server to the browser and that kind of thing. And around 10 years ago, I actually read a book on Elixir specifically and Phoenix, which a roommate at the time bought and I don’t think he ever read it himself, but I did. And yeah, I had to try it out. We had like the perfect project of like a, it was essentially like a bit, an online game essentially with money involved where we would play the game via web sockets and we had to have the state on the server to make sure people don’t cheat.

CRob (04:30)
Mm-hmm.

Jonatan Männchen (04:31)
And that was kind of like the perfect use case because that’s basically the first thing you read always about Erlang can handle that many millions of sockets at the same time. And yeah, kind of figured out at that point that basically I don’t have to wait for the unicorn project where this is the perfect solution, but rather in the end, it’s a technology

that is complete, you can build things with it. I don’t have to stick with PHP for the normal stuff. And yeah, over the time I got more more involved into Elixir itself, also with other open source projects. And I think around three years ago, I’m not quite sure, could be two, could be four. I got involved in the Erlang Ecosystem Foundation and the Security Working Group as well.

Working together with a lot of people trying to make Erlang secure. And maybe as a side note here, Erlang, Elixir, Gleam, and also a few other languages are all languages based on Erlang. So kind of like what’s Scala to Java, for example. And towards the end of last year,

I was talking a lot to Alistair, which is one of the board members of the foundation. And he raised for a long time that the CRA is a topic that we need to be very careful about. And the stars lined up, my last job was ending and in the end, yeah, everything lined up perfectly. And since the start of the year, I’m at the CISO trying to implement all of that.

CRob (06:16)
Awesome. So let’s talk about this new stage that you’re in. You mentioned that you and Michael and I met together at an Alpha and Omega community meeting. Can you, you and Michael maybe talk a little bit about how you two got introduced and how you discovered this amazing community that AO is nurturing.

Jonatan Männchen (06:40)
Yes. I mean, wait, where do I start? So yes, we haven’t really talked at FOSDEM, but I got to know you just from speaking at FOSDEM. But yeah, let’s start there. So I was at…

Michael Winser (06:40)
I think it starts with you, Jonathan. I don’t know how you came to the meeting.

Was it it FOSDEM? I gave a talk at. OK, yeah.

Michael Winser (07:04)
Yeah, so I’ll go. At FOSDEM, I had a couple of talks, one of which was in a room that was partly organized by the folks from the STA and talking about funding and open source. And as you might imagine, it was a crowded room. A lot of people, a lot of questions, lot, and you know,

Mirko and I Mirko’s from the STA Tried to put together a presentation even to sort of explain what we are and how we do things or whatever And in 15 20 minutes, we obviously compressed a lot of thoughts and time into that But it worked as intended right that we got lots of good questions and people who didn’t even know What we did or why or whatever sort of started coming out of the woodwork and and it’s been really great and John is over to you:

Jonatan Männchen (07:52)
Yeah, it was actually the day before. It was the FOSDEM Fringe event. I was not present at your talk. I knew that it happened. But it was the SBOM Fringe event where you were also speaking. I also didn’t… I mean, I read through a lot of the OpenSSF stuff on a high level of what the OpenSSF is doing. And I saw Alpha and Omega, but I didn’t really go into details there. just knew that it existed. yeah, you talking actually brought it up in my mind. And we, as the foundation, we are in this spot where we now have some financing, which basically just extends to myself. But really to implement all of this, we need more help than we currently have. And so I thought it would be good to reach out. And that’s also why I joined the call.

CRob (08:22)
Mm-hmm.

Michael Winser (08:49)
I remember now, and that of course was completely unplanned. I was at that event as just a participant, and then Philippe asked me to come up and just say a few words, and I babbled some stuff, and here we are. So it’s always the sort serendipity things that really drive interesting outcomes.

CRob (08:49)
Excellent.

Ulf (09:05)
Okay.

CRob (09:07)
This is a really interesting topic and let me pull Ulf in for a moment. As a product manager, kind of selecting components that are going to go into a product that your organization sells. How important is it to know that these upstream projects you’re relying on have support and do take security seriously?

Ulf (09:33)
Well, I’m here as an antidote to a poison, is vendor lock-in. So the bigger part of my life, I’ve been part of industrial automation and we were running factories for automotive supplies or plastics or whatsoever. And as part of this company, we were building machines and we were using open source, but we were using it in a, I wouldn’t call it un-moral, but in a weird way that we were just using it, you know, and didn’t, we didn’t take care about what you say, whether it is maintained or safe, it’s just there and you download it and you make a dependency and that’s it. And the antidote is number one, that at one day we stumbled over Alistair as well on a, on a … That was actually… What was that? It was in Berlin. Yeah, Elixir event in Berlin. And we realized that there’s a huge foundation behind it. And that was the cornerstone. And later when the CRA requirements came down to us and we started to wrap our minds how we would fulfill these requirements and make safe software for our customers, then only we realized how important these foundations may become to us. And we were lucky in a way that previously for other reasons, for reasons of resilience and reasons of resource management and reasons of development speed and whatever, know, we have chosen for Erlang slash Alexia stack. And so we were kind of enthusiastic about it, but we never choose it in the first place for security reasons. Then later, we realized that we are in front of a huge challenge of complying with these requirements, which are from you, but basically the United States are doing very similar stuff under different naming and many of them requirements, they overlap. And then we realized, lucky we are that we have chosen a pond rather than an ocean. And that pond is so concise and kind of personal and kind of streamlined, I would say. That gave us the confidence that if we use it to address these challenges, we would possibly have a very concise community to which we can reach out and meet real people, talking real talks and tackling real problems.

CRob (12:22)
Hmm.

Ulf (12:26)
So that is kind of how we ended up here. And this is also what made us finally, which convinced also my owner, we have a company owner and my CEO and also my development officer that we would fund such a foundation to a degree which is maybe not much in comparison to what probably Intel or Meta is doing, but you have to put it into relation to what our annual turnover is. And in that measure, it is a considerable amount of money and we are willing to continue to do so.

CRob (13:09)
Nice.

Michael Winser (13:10)
I just want jump in. I think you would be surprised comparing yourself to what other corporations are doing. And I just, want to start by celebrating the several things here. One is sort of the pragmatic taking control of your destiny approach, right? And it’s always, you know, it’s open source. There’s a lot of stuff that happens and it’s like free as in beer. It’s like someone shows up and gives you beer. But as I like to say, it’s really more like free as in puppies and they need care and they need love. And Organizations that understand that and make that investment Find out all kinds of interesting things such as you now actually have a lot more like you can train your puppies to go in the right direction and not not You know pee in the kitchen, for example Metaphorically, we’re going to stop with that particular direction But I think it’s also an example of how in a competitive landscape regulation even sometimes ham-fisted regulation, I would certainly not attribute anything to one regulation or the other, but regulation is hard. But any kind of regulation essentially creates better incentives and it rises. Like everybody has to pay a little bit more attention to these things because, you know, in a competitive landscape, every dollar you spend on feeding your supply chain and taking care of your puppies is a dollar you’re not spending on marketing or development or whatever. But, you know,

It’s your code, even though you’re not the ones writing it, it’s in your business, it’s in your product. And so the care of that investing in that has a return. So first of all, kudos to you and your organization. I think it’s amazing. and it’s a pattern I would love to see sustained and repeated as more organizations can find ways to do so. And I think you’ve also shown it’s not that hard. You just show up and say, we’d like to make sure that this gets done properly and things happen.

Ulf (15:04)
Yeah, and I would like to add that it becomes even a rational choice. There’s not, I mean, when we talk about puppies, there’s a lot of love and care and all of that, right? But you can also see the case I have been describing as a very rational choice, because especially if you look into the alternatives.

One alternative would have been we would have developed security by our own. Yeah. Okay. And, and obviously that, that is a monstrous task and we would have needed competences, which clearly we do not have. And it would have taken a lot of time probably and would have been expensive. So that has been ruled out in the first place. And the second option would have been that we would have outsourced it to some contractor.

Ulf (13:19)
I mean, there are specialist companies out there. You can tell them what to do. They have the competencies and they will do it in a proper timing and for a proper cost. But still there is a downside to that, which is trust. Because if we go to our customers and tell them about security and we tell them, the security we are selling to you is actually the one we bought from this other guy. And, and he’s a specialist, I tell you.

Then our customer would say, who’s that? And what is he doing exactly? And how do you know? And all of that good questions from a customer point of view, that’s a proper question. And then no matter whether he was doing wrong or right, to build trust is very difficult. In turn, if we kind of outsource that, it’s not a real outsourcing because we don’t have a mandate here, right? We are just funding it.

Ulf (14:09)
But if this is done by somebody else which we do not influence directly, there’s two benefits. There’s never a smell of influencing in turn. So we can tell them what they’re doing is trustworthy because we are not influencing them. There’s no conflict of interests. And also if they are doing it and we are not mandating them directly, they would look for a bigger community, which was foster a more resilient solution landscape. I’m very convinced that this would happen. And both of them mechanisms, I can go back to my customer and tell them, look, and because of these two mechanisms, you can trust them guys a lot more than you can trust either us or a contract that we have bought. So if you look at down that road, it’s probably a very rational choice to kind of outsource things to people you’re not influencing. It sounds contradictious in the first place, but it’s not that much contradictions if you think it to the end.

CRob (15:10)
And the behavior you’re describing – how a manufacturer gets value out of these upstream projects and you have taken the very conscious decision that we’re going to try to support them. That is exactly the behavior that the CRA has explicitly written in is they’ve asked manufacturers like if you’re using upstream components, you should give back and participate. And I really applaud you all for making that choice very early on.

Ulf (18:33)
Yeah and also look into CRA. You have three choices. You’re a consumer, a manufacturer or a steward.

Michael Winser (18:40)
Yeah.

Ulf (18:41)
I don’t want to be a manufacturer in key matters. I would love to be a steward, but I can’t. It’s not in our competencies. So to say, I love to be a steward, I can’t, so I’m going to fund one.

CRob (18:58)
Let me turn the next question to Mr. Windsor. Why is it so hard for a lot of projects to implement good security practices and how does funding help that?

Michael Winser (19:12)
I love this question. So somewhat Ulf talked about starts with competency. know, not everybody is a, you know, well, let’s start with the problem of software supply chain security, right? As I love to say, it’s like the Y2K problem without the same clarity of problem solution or timeline. Right. Everyone is still learning a lot about this and we have decades of technical debt. So expecting, you know,

Mary and Joe, software developers working on a cool open source project to have competency in all the risks that they are essentially carrying forward is unreasonable. It’s just not practical. And any solutions we do are not going to be magically by teaching everybody to become security engineers at the same time, any more than everybody knows how to do front end, back end, or use airline as a language or rust or whatever. There are competencies that take real time and energy to acquire.

And that’s a big deal. The other aspect is it actually goes back to the same competitive pressures that corporations are feeling at the of deepest end of the supply chain or the furthest out to the right end of the supply chain. Open source projects are, you know, like have different reward mechanism. At the end of the day, being used, being valuable is something people care about.

And a lot of the signal that they receive from their downstream dependencies, right, is somewhat abstract, but it’s about usage. How many people are using me? How many, you know, GitHub stars, which please do not use GitHub stars as an indicator of popularity. but, know, and so they’ll do things that people are asking them to do. And invariably, what do people want to do? Like I’m building some software and somebody has built a module that does something for me.

CRob (20:46)
Stars and likes.

Michael Winser (21:00)
If I can shift the work onto them, so could you add a feature that does X, right? Says every enterprise customer ever, and says every open source project. Software developers want somebody else to do the things that they’re not good at, right? So I’m using some HTTP client library. It does some really cool. There’s now an edge case on dealing with streaming over HTTP 3, blah, blah, blah, blah. Could someone do that for me, rather than me having to add that to my application code, which is trying to plug tab A into slot B and make an NCP talk to Zapier, for example.

Michael Winser (21:30)
And so that’s a big part, right? There’s a lot of pressure and signal towards adding new features. There’s a competency they already have, which creates a fluency and ease of work around the feature set they’ve developed. So you have this hard hill to climb of security of things I don’t really know about, an easy and rewarding hill to climb, which is things I do know about and people are asking for, right? Those choices are too easy, right? It’s too easy to go down the path of doing more of that.

And unfortunately, that problem is bigger than that because the people who are downstream who would benefit from the security and might benefit from the feature sets, they don’t know more about the security. They don’t know more about the code. So who is going to do that work? How’s it going to happen? And, you know, this is where I think what’s awesome about what Ulf and company have done, right, is saying, look, we need to bring some experts here. And what I love about it too is the point of leverage, right?

So you could go and look at all the supply chain things and fix all the individual pieces, or you can make it someone’s job in an entire ecosystem to reason and think about that ecosystem and to make changes that are going to benefit all of them. And that’s the alpha of Alpha Omega is all about that scale.

Ulf (22:45)
And we would not have done it if we would not have faith in the fact that it can be done in that ecosystem. And we have faith it can be done in that specific ecosystem. Yeah. Because it’s so streamlined and so concise and so complete in it’s so feature complete that it helps us a lot. Yeah.

Michael Winser (23:06)
I think that’s really key. And I think that the other thing that comes out of this, I think we’re starting to see these in other ecosystems and I fully expect them to become like significant factors in the airline ecosystem as well, which is you’re normalizing security. So when you think about software engineers and the set of skills that they all think are common, right? There’s a certain subset of things.

CRob (23:23)
Mm-mm.

Ulf (23:23)
Yes.

Michael Winser (23:31)
what we’re starting to do is to normalize a broader set of things around security concerns. So not everybody’s going to become a security expert, but if everybody’s aware of security and like, I should do this. it, mean, some of these things aren’t even about implementing more secure code, right you could probably talk for days on how maybe you should handle reports around vulnerability and just having process around vulnerabilities in your projects. And when somebody does tell you, whoopsie, you actually even have a process to handle that.

CRob (23:54)
Exactly.

Michael Winser (24:01)
That is a significant gap for an awful lot of open source projects.

CRob (24:04)
Mm-hmm.

Jonatan Männchen (24:05)
Which is, the way, also a gap we’re very specifically addressing. We’re in the process of becoming a CNA [CVE Numbering Authority]. We’re currently in the onboarding workflow, not done yet. But we’re actually becoming a CNA for every package that is in the package manager, if they’re not covered somewhere else. Just because we think that we have more tools available to do the correct decisions in the whole thing and also reach the

Michael Winser (24:13)
Yes.

CRob (24:26)
Nice.

Jonatan Männchen (24:34)
Correct people than MITRE ever could just because they’re not part of that ecosystem specifically. yeah, so we really want to cover this as a CNA and also build in all the vulnerability reporting into the default tooling so everybody gets the benefits of that.

CRob (24:41)
Exactly.

That’s awesome.

Michael Winser (24:54)
This is, this is, mean, this is a pattern we’re seeing more and more, right? And, know, there’s now documentation well written by other parts of the Alpha Omega family on how to be a CNA. This is what we did, how it worked out or whatever. It’s worth stating to the perhaps, you know, less CNA obsessed listener, right? That one of the things that happens here is that the community can have a more curated control over what is being reported as a vulnerability and the process gets centralized. And this is not to impugn our

CRob (25:20)
Mm-hmm.

Michael Winser (25:24)
Esteemed colleagues in the security research industry, right? But they have incentives to find vulnerabilities and want to push them out and like that and when you push them either straight up to MITRE or directly to the individual project there is none of that curation happening and this allows an Esteemed set of experienced people in the airline community to make sensible decisions about is this really a vulnerability? What severity it has and so forth and there’s still a dialogue and should be a dialogue with the researcher, but it’s not

Sort of like the problem is that there’s no dialogue with MITRE or it just happens. There you go. And then it’s very hard to undo that later on. And it drags around creating, you know, imperfect signal for people consuming things.

CRob (26:03)
Right. So, I see you as representing kind of a really exciting new trend that we’ve witnessed over the last few years, where communities are reaching out and bringing in subject matter experts to become this developer, security developer in residence, kind of having this role. From your perspective, and your role as CISO for Erlang, what do you see your role is in helping your community?

Jonatan Männchen (26:34)
I think the biggest part is to figure out what should we actually be doing. Because there’s lots of regulation from lots of different countries. Nothing is harmonized. And then even, for example, the open SSF, there is so many things in there just sifting through what does actually apply to us. And there’s other organizations than the open SSF as well. So just figuring out what should we be doing, I think is the biggest part.

CRob (26:39)
Mm-hmm.

Right.

Jonatan Männchen (27:04)
And yeah, I’ve started putting together essentially a roadmap of things that we want to implement. Also, there’s some stuff that I can directly tackle myself just because they’re in a size that makes sense for me to invest that in my time. For example, we just did the open chain certification for Elixir or the CV numbering authority, which is talked about.

CRob (27:22)
Very nice.

Jonatan Männchen (27:34)
And we also just implemented the best practices batch for Elixir as well. So there’s lots of different things going on and there’s lots of them, yeah, where I can just look at them, do them, get it done. But there’s also bigger ones like for example, implementing SLSA throughout the whole package manager, where we’re more at the point where we need additional help just because it doesn’t make sense for me to focus on that for that long time right now. And so.

CRob (27:53)
Mm-hmm.

Jonatan Männchen (28:04)
I’m trying to figure out a way of organizing all of that and getting the funding and figuring out what is exactly we’re trying to do. And yet just put together a plan that actually could work essentially.

CRob (28:18)
Michael?

Michael Winser (28:20)
I’m glad you mentioned SLSA and You and I should chat offline for some specifics but I’ve been working within the SLSA working group for a while and one of the members out Tom Hennan has created there’s one of the tracks we’re working on this the build track there’s a less developed thing called the build environment track which manages the sort of Security of the environment which run the Maturing nicely is something called the source track around dealing with the provenance of the source code and the environment in which the source was created, right? And so being able to say you have branch protection on and things like that, and there’s a set of requirements. Well, Tom has produced a very simple little workflow. There’s still in sort of prototype phase that makes getting to SLSA level three of source level three provenance, where you have this continuous from a date, point in time forward chain of trust for all the commits to your repo incredibly easy to achieve.

And so would love to work with you and the Erlang and the Elixir space and the package manager space to do that and then Connected back to trusted publishing depending upon the workflow from there to publishing into the package manager You can start to see an end-to-end provenance story. That is very interesting and You know last week I had a chat with some of your colleagues from Erickson who work on the OTP stuff and I was asking them about what what’s their interest to the package manager versus the other parts of the ecosystem and

They build from source, use the force, build the source. And so that eliminates a lot of tampering threats in the build space, but they still care about the provenance and authenticity of the source. And by the way, they also say they very much care about the health of the ecosystem as well. And so they’re to help out in various ways. So there are dots to connect there that I hope are, and this is part of what we’re funding at Alpha Omega, that reduce the toil for someone like you and your ecosystem to kind of take those next steps.

CRob (30:16)
And I bet as a product manager, Ulf, this would be a really compelling story if you knew that the components that you were putting integrating into your products had this pedigree and provenance that had that chain of custody and they were untampered with.

Jonatan Männchen (30:16)
Mm-hmm.

Ulf (30:31)
Absolutely, and that even if I knew that would be the case then still there’s tons of work to do for security so I’m offloading a part of the problems we are facing and still Previously we mentioned that or I mentioned that probably we do not have the competencies in security and Probably under rating our company. Of course, we have experts in that matter but not to that extent what Jonatan can do for us number one or the community can do for us, number two, or foundation can do for us, or CNA can do for us. And the processes you’re mentioning about making the correct ratings and making the correct proceedings in how to handle these vulnerabilities, all of that we can definitely not do. And still there’s tons of work to do to provide safe software or secure software to our customers from operating systems and good habits and proceedings in the pipelines and management of quality. All of that’s still down to us. And even there, we benefit from Jonatan providing best practices. Simply as that. It’s undisputed, right? Somebody calls out a best practice, it goes into our development rules, and here we go. So it’s simple. You don’t need to spend or wrap your brain around how to do that the best way. It’s a matter of trust.

At the end of the day, for us, it’s a matter of trust.

CRob (32:00)
Awesome. So as we wind down, I would like to talk about, you know, what is all your individual calls to action? You know, what if there are other communities, whether it’s a project or another language ecosystem, and they hear about this amazing story that the three of you are weaving together, you know, what advice would you give these communities and how they can enter in and become these, good stewards and good participants in these types of situations.

Jonatan Männchen (32:36)
Yeah, thinking a second what to say.

Michael Winser (32:39)
Why don’t I start? Because I’ve got the easiest thing to offer right up front. Whether you are an expert in coding, an expert in the problem space, an expert in the language or the package that you’re using in your business, the first and simplest thing to do is to engage, to contact the organizations upstream of yours and say, hello, my name is Michael, and I am benefiting from your work. I would like to make hello and say, how’s it going? Introduce myself so that when you have a problem later on, whether it is an audit finding out that your CRA compliance is at risk because of some practice or whether it’s a silly little bug or whether it’s a vulnerability has surfaced and you’re not sure whose fault it is or what to do or how to do something or what the importance of it is. If you already have a working relationship, even if it’s just purely social, if it’s just literally love in the human sense of like love is a verb, hello, how are you today?

I care about your work, right? You’re already so much better off than you would be otherwise. And so the first thing to do is to engage and to listen, and then you will have a very clear path of opportunities forward, or at least the connection when you need them.

Jonatan Männchen (33:53)
What I could add, a lot of people in an ecosystem don’t really look outside of that ecosystem. So it’s really important that you’re not trying to do everything by yourself. There’s lots of smart people from lots of different places that already thought about these things, but they haven’t thought about it in your specific programming language probably. But yeah, looking around what others are doing and actually connecting beyond the borders of your own ecosystem is probably one of the most important things to do.

Ulf (34:39)
And from a user perspective, of the other end of the food chain, whatever, I wish that more people would be honest about their usage of open source and their contribution and, know, distinguish clearly what is their added value with what they have developed and they willing to sell to their customers and what they have just, you know, grabbed as a base for what they want to offer as a customer value. And if that would be a more honest and a more transparent way of doing business, then automatically more people would join an initiative like we have been doing and that base would become a lot more resilient and even a lot.

And it will be worth the living, you know, for the people who are doing it. mean, currently, most of or many of them projects are maintained by enthusiasts and not for living. And sounds sounds wrong, kind of wrong. Yeah, I would like I can’t see why we should not distinguish between our added value and somebody else’s added value and make that very transparent. Transparency.

CRob (35:37)
Excellent. Well, gentlemen, I really appreciate your actions, both in your businesses and upstream and in your communities. And I thought this was a really insightful conversation. And I know we’ll be having more like this as items like the Cyber Resilience Act in Europe or legislation around the globe continues. This is going to be a matter of great importance that downstream has generated an unimaginable amount of value from the work of upstream. And there needs to be a way to be more participatory and to give back and to show that love that Mr. Windsor noted back to those developers that have given you so much. So gentlemen, thank you. I appreciate your time. And with that, happy open sourcing. That’s a wrap for us.