Podcast

00:00 – Welcome and Introduction to AICC
01:37 – OpenSSF’s AI Security Mission: Two Lenses
03:54 – Competition Highlights: What the Teams Discovered
07:43 – Real-World Impact: From Research to Production
10:44 – Lessons Learned: Working with Open Source Maintainers
13:13 – OSS-CRS: Building a Standard Infrastructure
14:29 – Breaking Down Walls: Post-Competition Collaboration
15:39 – How to Get Involved

• Jeff Diecks LinkedIn page
• Christopher “CRob” Robinson LinkedIn page
• AI Cyber Challenge (AIxCC)
• OSS-CRS Project
• OpenSSF AI/ML Security Working Group
• Cyber Reasoning Systems Special Interest Group (Slack)
• Get involved with the OpenSSF
• Subscribe to the OpenSSF newsletter
• Follow the OpenSSF on LinkedIn

CRob (00:09.408)
Welcome, welcome, welcome to What’s in the SOSS, the OpenSSF’s podcast where I get to talk to the most amazing people in the planet that are either involved or on the outskirts of open source software and open source security. Today, we have a treat. We get to talk to one of my dear friends and teammates, Jeff, and we’re gonna dive into a topic that I really don’t know a lot about today.

So Jeff, why you introduce yourself to the audience and kind of describe what you do for the foundation.

Jeff Diecks (00:44.686)
Yeah, thanks, CRob. And hello, I’m Jeff Diecks. I’m a technical project manager with OpenSSF. And I’ve been involved in open source for 20 plus years now. Goodness. And I am OpenSSF’s lead on the AI cyber challenge program that we work on. And CRob is sort of telling you the truth. He’s been on the three episodes prior to this where he’s learned plenty about AICC, but we’re here to talk a little bit more about this and wrap up the series today.

CRob (01:17.582)
Yeah, these words you use, AI, that isn’t something I hear a lot about. Wink. Could maybe you recap for us, like what is the OpenSSF doing around AI security? And then just maybe give a brief recap about the AI CC.

Jeff Diecks (01:37.028)
Yeah, for sure. So OpenSSF in the world of AI, we have our AI ML Security Working Group that looks at security and AI from kind of two lenses. The first is AI for security, which is what we’ll be talking about here today, projects that help you AI to help improve the security of projects, and security for AI, which is securing all this new world of AI things and all the lessons we’ve learned about securing software. AI is software too and it needs securing. We have a whole suite of projects and work that focuses on that too. Specific to AIxCC, again, it’s the AI Cyber Challenge. It was a two-year competition run by DARPA and ARPA-H. If you’re just hearing this episode first, I encourage you to go back to the first episode in this series with Andrew Carney from DARPA and ARPA-H for an overview of the program. And then we got into some good conversations with a couple of the team leads from some of the winning teams. But the purpose of the competition was to use AI and develop new systems to both find and fix vulnerabilities in open source software that are important to our critical infrastructure. An interesting part about the competition… written into the rules for any of the competitors accepting prize money, they were obligated to release their software as open source.

CRob (03:07.214)
Nice. That’s awesome. Well, yeah, again, I say that a bit tug-and-cheek, but there has been quite a lot of activity, whether it’s in the working group or specific to the follow-ons to the AICC competition, which is what we’re here about today to kind of put a bow on these conversations and help encourage the community to engage and go forward. So we talked about, we talked to a couple of the teams, we talked to Andrew, kind of gave us an overview of the program.

From your perspective and your engagement with the community, we have a new cyber reasoning special interest group within the foundation. So what have the teams been up to subsequently since the August close of the competition?

Jeff Diecks (03:54.414)
Yeah, there’s really two parts of this and I’ve had the great honor of meeting with and speaking with a lot of the teams and learning about what they’ve been doing. But we first started with just conversations about, you know, their experiences with the competition and what they learned, similar to the couple of episodes that we did. And what was really interesting is, you know, every single team, there’s something of value that came out of their system. They excelled in a, you know, at least a specific area.

They were all finding bugs in real world software. Just a couple of highlights from some of the other teams. Team Theori, which was among the three winners, the third place winner, they had a unique approach. Their system, unlike the others, did not use fuzzing. It used pure LLM AI. so just an interesting variation and you potential there for that system to be super flexible because it doesn’t come with some of the requirements of projects already being set up with fuzzing. So interesting to see what becomes of their system there. And then in a couple of other cases, it was really interesting. Teams that have systems that are extremely capable, but for one reason or another,

CRob (05:08.142)
Yeah.

Jeff Diecks (05:18.096)
There was maybe a specific part of their system that just didn’t work well with the scoring mechanisms of the competition itself. So we had a team that was one of the best at generating patches for the issues that it found that was most effective. But as these things go, there was kind of a late change in the architecture of their system during the competition. the part of their system that was supposed to submit all these patches that got generated into the competition for scoring didn’t function correctly and didn’t submit everything. So they didn’t get credit for all their great work. But we’ll talk in a little bit about well, but it’s still a capable system. And now we can use it not for a competition, but for real stuff that’s potentially even more valuable. There was another that it was great at generating proof of vulnerabilities.

CRob (06:13.87)
Mm-hmm.

Jeff Diecks (06:15.328)
And, but they had made some assumptions based on the competition infrastructure and the system just didn’t perform well within the confines of the competition. But what was interesting about the architecture of their system was they would generate a potential result that they thought might have been a finding from part of their system. And then they would submit that potential result out to several other LLMs and have them feedback.

CRob (06:24.718)
Mm-hmm.

Jeff Diecks (06:45.176)
a verdict on whether they thought it might be effective or not. We kind of made the joke. It was like doing the poll of getting eight out of nine dentists to agree and decide on the submission. So those are some highlights from the competition. But you asked about what the team’s been up to in recent months. So that’s been really interesting. DARPA has kind of extended the incentives from their program and they’re offering

CRob (06:48.846)
Hmm.

Jeff Diecks (07:13.552)
incentives and rewards for the teams now taking these systems and using them in the real world against real open source projects and demonstrating that they’re effective there. And if they can demonstrate that, they earn additional reward money, which is encouraging the adoption and the transition of this research into real world usage. So we’ve had some interesting findings there and a few examples there.

CRob (07:27.374)
Awesome.

Jeff Diecks (07:43.248)
We’ve got Team 42 that we’ve been working with. They’ve focused a lot on their system seeming to be very effective in working with the Linux kernel and specifically some of the out-of-tree subsystems. They’ve found and reported several bugs and had some of them accepted, accepted patches. And actually later this week we’ve scheduled and we’re doing a consultation for that team with a kernel maintainer.

to give them feedback and help their research move forward and any guidance they can give on how to make their system more effective. So we’re looking forward to that conversation.

CRob (08:23.36)
Excellent.

Today, so we have a mixture of projects that are being donated. They’re all open source, but some of them are being donated like here, for example. Where would we go from here? What is the group’s thoughts about these, broadly the cyber reasoning systems and kind of what other interests or ideas are floating around to keep the momentum?

Jeff Diecks (08:52.91)
Yeah, there’s a few things. So one, OpenSSF is involved with the teams and we’ve formed, as you mentioned, the special interest group, the cyber reasoning system special interest group as the kind of continued home from the competition for all the teams to continue collaborating together and working there. some interesting developments so far, we’ve hosted having the teams present to one another.

of their work with real open source projects and examples of bugs they’ve found in the process they’ve been following and how they’ve been getting received from open source projects with what they’ve been submitting. So for example, Team Fuzzing Brain, who has donated their CRS systems for OpenSSF to host and support.

They’ve been working against a bunch of projects, but specifically they shared some examples of their work with the CUPS project where they found some bugs, they reported them, they’ve had some accepted patches, and they’ve gotten great feedback from the CUPS maintainers who are very appreciative of their work, both finding bugs and submitting patches, but also helping to generate and expand the fuzzing.

CRob (09:56.408)
God.

Jeff Diecks (10:17.028)
know, harness coverage of the project itself, which the systems are pretty capable of. So we’ve been learning a lot about the reporting process because it’s one thing to have these capable systems, but you know, it’s the world where like you and I and everybody else are a bit, you know, skeptics of just, you know, pure AI things, right? So we’re, working our way through and kind of learning from one another about

CRob (10:19.82)
very nice.

Jeff Diecks (10:44.078)
What’s the best way to keep humans in the mix and how are projects receiving these things? What are some lessons learned? So for example, we had a conversation in a SIG meeting where we’re talking about the patch submission process and some of the projects were kind of reacting. It was perhaps a bit too aggressive to just go ahead and introduce yourself by submitting a patch to the system.

CRob (10:46.51)
Mm-hmm.

Jeff Diecks (11:12.784)
right into the pull request queue of a project. And the group suggested maybe for the next go-round, it’s maybe a more polite way to introduce yourself by opening an issue, reporting how it was found, what it was found, all the supporting information, and then attaching a patch to be considered versus just, hey, here’s a PR. By the way, it came from AI.

So some interesting.

CRob (11:43.022)
Right. And we’ve heard a lot of feedback from upstream about their disinterest in that approach.

Jeff Diecks (11:50.772)
Yeah, well, and was a big focus of the scoring of the competition itself. That was among the feedback that we gave consistently for a couple of years of make sure you’re incentivizing the development of systems that don’t make life more difficult for maintainers, but hopefully make things easier and think about how these things will be received, not just technical capabilities.

But you mentioned, you know, donated projects and, you know, the one that I think is of real interest and, you know, for folks to follow along. So Team Atlanta led the way development of a project that we’re calling OSS-CRS, and bundled in with it, they intend to have something called CRS Benchmark.

And what these are for is it intends to be a standard infrastructure for building and running and evaluating all these CRSs and being able to kind of mix and match and use different parts of different ones for, you know, kind of a combined solution.

So, you know, if we think of a future where we’ve got, you know, a system like we talked about that’s most effective at generating patches, but we’ve got a different one that’s best at finding vulnerabilities.

CRob (12:58.755)
Wow.

Jeff Diecks (13:13.488)
And the hope is that through this standard interface, folks can leverage and kind of fine tune things to get the best performance and the best results out of a combination of systems rather than just relying upon a single one. So you can just think of it the way it’s intended to run. If you just imagine yourself at a command line prompt and you issue a OSS dash bug find dash CRS build.

then give it a configuration and a project, a compatible project, and that’ll build a system to run against. And then you can issue, thing, OSS bug find CRS run config project and the name of harness, and it’ll go off and do its thing. So again, you’re specifying which configurations you’re wanting to use, which subsystems you want it to.

CRob (14:01.666)
Mm-hmm.

Jeff Diecks (14:14.01)
pull from. So they’ve got an interesting roadmap. You know, we’re talking with them and, you know, hoping to bring our community and perspective to help support that project and its development, you know, and adoption into the real world.

CRob (14:29.176)
And I remember us talking around DEF CON last year. And I think the competition and the prize money are great. That was very exciting. But I’m most excited about this kind of phase we’re in now, where we’re seeing the teams with that ethical wall down between them from the competition. Now they’re actually able to talk and collaborate and share ideas. I’m really excited to see the community come together, helping support these students on these ideas.

Jeff Diecks (14:58.916)
Yeah, and that’s been the interesting part and part of why this it’s taken us a bit to get this whole podcast series out through the course of the competition for competition integrity reasons. You know, we were advising the competition organizers, but we weren’t interacting with the teams themselves. So we had to go through a whole process after the finals to introduce ourselves to all of the individual teams and let them know that we’re here and about and you know, the things we offer to help.

support them in the further development of the system. that’s been an interesting few months of lots of great conversations and seeing these teams come together within our working group and special interest group.

CRob (15:39.842)
So you’ve inspired me. I sure would like to know more on how to get involved. How can I do that?

Jeff Diecks (15:41.488)
Ha

Well, if your Monday afternoons at 1 p.m. Eastern are free, we have two different meetings that basically are in that time slot on alternating weeks. Our full AI ML security working group meets again on Mondays at 1 p.m. Eastern time on a biweekly basis. And on the alternating weeks, the cyber reasoning system special interest group meets in that time slot. You can find them.

CRob (16:09.326)
Peace.

Jeff Diecks (16:11.844)
You know, both of those meeting series on our community calendar at opensf.org slash calendar.

CRob (16:18.958)
Well, I want to thank you for helping shepherd and guide the folks in the competitions. We’re seeing some great results come out of this. And I’m really excited to see what our community and these amazing students kind of come up with on how to further the use of AI to help improve security on things. Yeah. And with that, we’ll say this is a wrap.

Jeff Diecks (16:42.308)
Sounds good. Thanks, CRob, and we’ll see you in the meetings.

CRob (16:48.911)
I for one welcome our new robot overlords and I wish everybody a happy open sourcing. Have a great day.

In the third episode of our AI Cyber Challenge (AIxCC) series, CRob sits down with Michael Brown, Principal Security Engineer at Trail of Bits, to discuss their runner-up cybersecurity reasoning system, Buttercup. Michael shares how their team took a hybrid approach – combining large language models with conventional software analysis tools like fuzzers – to create a system that exceeded even their own expectations. Learn how Trail of Bits made Buttercup fully open source and accessible to run on a laptop, their commitment to ongoing maintenance with prize winnings, and why they believe AI works best when applied to small, focused problems rather than trying to solve everything at once.

This episode is part 3 of a four-part series on AIxCC:

• 1. From Skepticism to Success: The AI Cyber Challenge (AIxCC) with Andrew Carney
• 2. From Skeptics to Believers: How Team Atlanta Won AIxCC by Combining Traditional Security with LLMs
• 4. Cyber Reasoning Systems: The Real-World Journey After AIxCC

00:04 – Introduction & Welcome
00:12 – About Trail of Bits & Open Source Commitment
03:16 – Buttercup: Second Place in AIxCC
04:20 – The Hybrid Approach Strategy
06:45 – From Skeptic to Believer
09:28 – Surprises & Vindication During Competition
11:36 – Multi-Agent Patching Success
14:46 – Post-Competition Plans
15:26 – Making Buttercup Run on a Laptop
18:22 – The Giant Check & DEF CON
18:59 – How to Access Buttercup on GitHub
21:37 – Enterprise Deployment & Community Support
22:23 – Closing Remarks

• Michael Brown’s LinkedIn page
• AI Cyber Challenge (AIxCC)
• Trail of Bits
• Buttercup GitHub Repo
• OpenSSF AI/ML Security Working Group
• Cyber Reasoning Systems Special Interest Group (Slack)
• Get involved with the OpenSSF
• Subscribe to the OpenSSF newsletter
• Follow the OpenSSF on LinkedIn

CRob (00:04.328)
And next up, we’re talking to Michael Brown from Trail of Bits. Michael, welcome to What’s in the SOSS.

Michael Brown (ToB) (00:10.688)
Hey, thanks for having me. I appreciate being here.

CRob (00:12.7)
We love having you. So maybe could you describe a little bit about your organization you’re coming from, Trail of Bits, and maybe share a little insight into what your open source origin story is.

Michael Brown (ToB) (00:23.756)
Yeah, sure. So Trail of Bits is a small business. We’re a security R &D firm. We’ve been in existence since about 2012. I’ve personally been with the company about four years plus. I work there within our research and engineering department. I’m a principal security engineer, and I also lead up our AIML security research team. So Trail of Bits, we do quite a bit of government research. We also work for commercial clients.

And one of the common threads in all of the work that we do, not just government, not just commercial, is that we try to make it as public as much as we possibly can allow. So for example, sometimes, you know, we work on sensitive research programs for the government and they don’t let us make it public. Sometimes our commercial clients don’t want to publicize the results of every security audit, but to the maximum extent that our clients allow us to, we make our tools, we make our findings, we make them open source. And we’re really big believers in

that the work that we do should be a rising tide that raises all ships when it comes to the security posture for the critical infrastructure that we all depend upon, whether we’re working on hobbies at home and whether we’re building things for large organizations, all that stuff.

CRob (01:37.32)
love it. And how did you get into open source?

Michael Brown (ToB) (01:42.146)
Honestly, I’ve just kind of always have been there. So realistically, you know, the open source community is where a lot of the research tools that I started out my research career. That’s where you found them. So I started off a bit in academia. I got my undergrad in computer science and then went and did something completely different for eight years. And then when I kind of

Uh, you know, for context, I joined the military. flew helicopters for like eight years and basically nothing in computing. But as I was starting to get out, um, the army, I, know, I was getting married, about to have kids. I kind of decided I wanted to be, you know, around the house a little bit more often. um, you know, I started getting a master’s degree at Georgia Tech. They’re offering it online. And then after I did that, I went to go, um, do a PhD there and also work for their, um, uh, their applied research arm, Georgia Tech Research Institute.

So lot of the work that I was doing was, you know, cutting edge work on software analysis, compilers and AI ML. And a lot of the stuff that I, you know, built the tools that I did my research on, they came from the open source community. They were tools that were open sourced as part of the publication process for academic work. They were made publicly and available open source by companies like Trail of Bits before I came to work with them as the result of government research projects.

So, honestly, I guess I don’t really have much of an origin story for when I got there. I kind of just landed there when I started my career in security research and just stayed.

CRob (03:16.814)
Everybody has a different journey that gets us here. And interestingly enough, you mentioned our friends at Georgia Tech, which was a peer competitor of yours in the AICC competition, which you all on Trail of Bits team, I believe your project was called Buttercup. And you came in second place. You had some amazing results with your work. So maybe could you tell us a little bit about the…

Michael Brown (ToB) (03:33.741)
Yeah, that’s correct.

CRob (03:43.15)
What you did is part of the AI CC competition and kind of how your team approached this.

Michael Brown (ToB) (03:51.022)
Yeah. So, um, you know, at the risk of sounding a bit like a hipster, um, I’ve been working at the intersection of software security, compiler, software analysis, AI ML for, you know, basically, um, almost my entire, uh, career as a research scientist. So, you know, dating back to the earliest program I worked on, uh, for, DARPA was back in 2019. And, um, so this was, this was before the large language model was a predominant form of the technology or kind of became synonymous with AI. So.

CRob (04:04.719)
Mm.

Michael Brown (ToB) (04:20.792)
For a long time, I’ve been working and trying to understand how we can apply techniques from AI ML modeling to security problems and doing the problem formulation, make sure that we’re applying that in an intelligent way where we’re going to get good solid results that actually generalize and scale. So as the large language model came out and we started recognizing that certain problems within the security domain are good for large language models, but a lot of them aren’t.

When the AI cyber challenge came around, we always approached this and this, I was the lead designer, my co-designer, Ian Smith. And I, you know, when we sat down and make the, the original concept for what became Buttercup, we always took an approach where we were going to use the best problem solving technique for the sub problem at hand. So when we approached this giant elephant of a problem, we did what you do and you have an elephant and you’ve got to eat it, eat it one bite at a time.

So each bite we took a look at it and said, okay, you we have like these five or six things that we have to do really, really well to win this competition. What’s the best way to solve each of these five or six things? And then the rest of it became an engineering challenge to chain them together. Our approach is very much a hybrid approach. This was a similar approach taken by the first place winners at Georgia Tech, which by the way, if you’ve got to be beat by anybody being beat by your alma mater, it takes a little bit of this thing out of it. So, you know, we came in first and second place. It’s funny, I actually have another Georgia Tech PhD alumni.

CRob (05:33.832)
You

Michael Brown (ToB) (05:42.926)
on my team who worked on Buttercup. So Georgia Tech is very well represented in the AI cyber challenge. So yeah, we’ve always had a hybrid approach. The winning team had a hybrid approach. So we used AI where it was useful. We used conventional software analysis techniques where they were useful. And we put together something that ultimately performed really, really well and exceeded even my expectations.

CRob (05:45.458)
That’s awesome.

CRob (06:07.56)
I can say I mentioned in previous talks, I was initially skeptical about the value that could have been derived from this type of work. But the results that you and the other competitors delivered were absolutely stunning. You have converted me into a believer now that I think AI absolutely has a very positive role can play both in the research, but also kind of the vulnerability and operations management space. What

Looking at your buttercup. What is unique about your approach with the cyber reasoning system with the buttercup?

Michael Brown (ToB) (06:45.39)
Yeah, so it’s funny you say that we converted you. I kind of had to convert myself along the way. There was a time in this competition where I thought, you know, this whole thing was going to kind of be reliant on AI too much and was going to fall on its face. And then, you know, at that point, I’d be able to say like, see, I told you, you can’t use LLMs for everything. But then it turns out, you know, as we got through there, we use LLMs for two critical areas and they worked much better than I thought they would. I thought they would work pretty well, but they ended up working to a much better degree than I thought they actually would. you know, what makes Buttercup unique?

CRob (06:49.852)
Yeah.

CRob (07:00.678)
You

Michael Brown (ToB) (07:15.69)
is that, like I said, we take a hybrid approach. We use AIML for very good problems that are well-suited for AIML. And what I mean by that is when we employ large language models, we use them on small subproblems for which we have a lot of context. We have tools that we can install for the large language model to use to ensure that it creates valid outputs and outputs that can carry on to the next stage with a high degree of confidence that they’re correct.

CRob (07:30.076)
Mm-hmm.

CRob (07:43.912)
Mm-hmm.

Michael Brown (ToB) (07:45.934)
And then in the places where we have to create or sorry, in one of the places where we have to use conventional software analysis tools, those areas are very amenable to the conventional analysis. So, you what I mean by this? good example is, for example, we needed to produce a proof of vulnerability. We have to have a crashing test case to show that when we claim a vulnerability exists in a system, we can prove through reproduction that it actually exists. Large language models aren’t great.

at finding these crashing test cases just by asking it to look at the code and say, hey, what’s going to crash this? They don’t do very well at that. They also don’t do well at generating an input that will even get you to a particular point in a program. But fuzzers do a great job of this. So we use the fuzzer to do this. But one of the things about fuzzers is they kind of take a long time. They’re also more generally aimed at finding bugs, not necessarily vulnerabilities.

CRob (08:36.808)
Mm-hmm.

Michael Brown (ToB) (08:42.702)
So we use an AIML, or Large Language Model based accelerator, or a C generator, to help us generate inputs that were going to guide the fuzzer to either saturate the fuzzing harnesses that existed for these programs more quickly. They would help us find and shake loose more crashing inputs that correspond to vulnerabilities as opposed to bugs. And those things really, really helped us deal with some of the short analysis and short.

processing windows that we encountered in the AI cyber challenge. So was really a matter of using conventional tools, but making them work better with AI or using AI for problems like generating software patches for which there really aren’t great conventional software analysis tools to do that.

CRob (09:28.018)
So as you were going through the competition, which went through multiple rounds, are there anything that surprised you or that you learned that, again, you said your opinion changed on using AI? What were maybe some of the moments that generated that?

Michael Brown (ToB) (09:45.226)
Yeah, so there I mean there were a couple of them. I’ll start with one where I can pat myself on the back and I’ll finish with one where I was kind of surprised. So first, we had a couple of moments that were really kind of vindicating as we went through this. Our opinion going into this was that large language models, couldn’t just throw the whole problem at it and expect it to be successful. So going into this, there was a lot of things that we did that we did

CRob (09:49.405)
hehe

Michael Brown (ToB) (10:14.774)
two years ago when we first started out that, you know, that’s like two years ago, it’s like five lifetimes when it comes to the development of AI systems now. So there were some things that we did that didn’t exist before that became industry standard by the time we finished the competition. So things like putting your LLM queries or your LLM prompts in a workflow that includes like validation with tools or the ability to use tools.

CRob (10:29.298)
Mm-hmm.

Michael Brown (ToB) (10:43.062)
That was something that was not mainstream when we first started out, but that was something that we kind of built custom into Buttercup when it came particularly to patching. And then also using a multi-agent approach. know, a lot of the, you know, I don’t know, a lot of the hype around AI is that, know, you just ask it anything and it gives you the answer. You know, we’re asking a lot of AI when we say, here’s a program, tell me what vulnerabilities exist, prove they exist, and then fix them for me.

And also don’t make a mistake anywhere along the way. It’s way too much to ask. we found particularly with patching, were, back then, multi-agent systems or even agentic systems or multi-agentic systems were unheard of. were still just, we were still using chat GPT 3.5, still very much like chatbot interactions, web browser interactions.

CRob (11:16.564)
Yeah.

Michael Brown (ToB) (11:36.438)
integration into tools was certainly less widespread. So we had seen some very early work on archive about solving complex problems with multiple agents. So breaking the problem down for it. And we used this and our patcher ended up being incredibly good. was our most important and our biggest success on the project. Really want to shout out Ricardo Charon, who’s our leader, the lead developer for our patching agent.

CRob (11:47.976)
Mm-hmm.

Michael Brown (ToB) (12:06.414)
or for a patching system within for both the semi finals and finals in the ICC. He did an incredible job and we really built something that I, like I said, I regard as our biggest success. So, know, sure enough, and as we go through this two year competition, now all of a sudden, you know, multi agentic systems, multi agentic tool enabled systems, they’re all the rage. This is how we’re solving these challenging problems. And also a lot of this problem breakdown stuff that has made its way baked into the models now, the newer thinking and reasoning models from

Anthropic and open AI respectively. They you you can give it these large complicated problems and will do it will first try to break it down before trying to solve it. So you know we were building all that stuff into our system before. It came about so that that’s an area where you know, like I said, we learned along the way that we had the right approach from the beginning and it’s really easy to go back and say that that’s what we learned that we were right. So on the other side of this I will have to say, you know, I’ll reiterate I was really surprised at how well.

CRob (12:53.639)
Mm-hmm.

Michael Brown (ToB) (13:04.11)
language models were able to do some of the tasks we asked it to do. Part of it’s how we approach the problem. We didn’t ask too much of it. And I think that’s part of the reason why the large language models were successful. an area that I thought where it was going to be much more challenging was patching. But it turned out to be an area where, a certain degree, this is kind of an easier version of the problem in general because open source software, which are the targets of the AI cyber challenge, they’re ingested into the training.

CRob (13:08.924)
Mm.

Michael Brown (ToB) (13:31.404)
data for all of these large language models. the models do have some a priori familiarity with the targets. So when we give it a chunk of vulnerable code from a given program, it’s not the first time it’s seen the code. But still, they did an amazing job actually generating useful patches. The patch rate that I expected personally to see was much lower than the actual patch rate that we had, both in the semifinals and in the finals. So even in that first year window,

CRob (13:33.64)
Mm.

Michael Brown (ToB) (13:58.63)
I was really kind of blown away with how well the models were doing at code generating, code generation tasks, particularly small focused code generation tasks. So I think, think large language models are kind of getting a bad rap right now when it comes to like, you know, trying to vibe code entire applications. They’re like, gosh, this, this code is slop. It’s terrible. It’s full of bugs and stuff. Well, well, you did also ask you to build the whole thing. You know, if I asked a junior developer to build a whole thing, they probably also put together some.

CRob (14:07.366)
Yeah.

CRob (14:17.233)
Yeah.

CRob (14:26.258)
Yeah.

Michael Brown (ToB) (14:26.71)
and gross stuff. But when I ask a junior developer to give me a bug fix, much like the large language model, when I ask it for a more constrained version of problem, they tend to do a better job because there’s just fewer moving parts. So yeah, those are are two things I took away. One that, you know, like I said, I get to pat myself on the back for another that was actually surprising.

CRob (14:46.012)
That’s awesome. That’s amazing. So now that the competition is over and what does the team plan to do next beyond this competition?

Michael Brown (ToB) (14:57.098)
Yeah, so I mean, look, we spent a lot of our time over the last two years. A lot of I wouldn’t quite say blood. I don’t think anyone would bled over this, but we certainly had some tears. We certainly had a lot of anxiety. you know, we put a lot of we put a lot of ourselves into Buttercup. And so, you we want people to use it. So to that end, Buttercup is fully available and fully open source. know, DARPA made it a contingent, a contingency of participating in the competition that

CRob (15:09.917)
Mm-hmm.

Michael Brown (ToB) (15:26.892)
you had to make the code that you submitted to the semi finals and the finals open source. So we did that along with all of our other competitors, but we actually took it one step further. So the code that we submitted to the finals is great. It’s awesome, but it runs that scale. It used $40,000 of a $130,000, I think, total budget. And it ran across like an Azure subscription that had multiple nodes.

countless replicated containers. This is not something that everyone can use. We want everyone to use it. So actually in the month after we submitted our final version of the CRS, but before DEF CON occurred, where we figured out that we won, we spent a month making a version of Buttercup that’s decoupled from DARPA’s competition infrastructure. So it runs entirely standalone on its own, but more importantly, we scaled it down so it’ll run on a laptop.

CRob (16:18.696)
Mm-hmm.

Michael Brown (ToB) (16:25.154)
We left all of the hooks. We left all of the infrastructure to scale it back up if you want. So the idea now is that if you go to trail of bits.com slash buttercup, you can learn about the tool. have links to our GitHub repositories where it’s available and you can go, you can go download buttercup on your laptop and run it right now. And if you’ve got an API key, that’ll let you spend a hundred dollars. We can run a demo to show you that, we can find and patch a vulnerability and live.

CRob (16:51.496)
That’s easy.

Michael Brown (ToB) (16:53.164)
Yeah, so anyone can do this right now. So if you’re an organization that wants to use Buttercup, you can also use the hooks that we left back in to scale it up to the size of your organization, the budget that you have, and you run it at scale on your own software targets. So even users beyond the open source community, we want this to be used on closed source code too. So yeah, our goal is for, you you asked what we’re gonna do with it afterward. We made it open source, we want people to use it.

And even on top of that, we don’t want it to bit rot. So we actually are going to retain a pretty significant portion of our winnings of our $3 million prize. We’re going to retain a pretty significant portion of that. And we’re going to use it for ongoing maintenance. So we’re maintaining it. We’ve had people submit PRs that we’ve accepted. They’re tiny, know, it’s only been out for like a month, but you know, and then we’ve also made quite a few updates to the public version of Buttercup afterwards. So it’s actively maintained.

There’s money from the company’s putting its money where its mouth is. We’re actively maintaining it. The people who built it are part of the people who are maintaining it. We are taking contributions from the community. We hope they help us maintain it as well. And yeah, we’ve made it so anyone can use it. I think we’ve taken it about as far as we can possibly go in terms of reducing the barriers to adoption to the absolute minimum level for people to use Buttercup and leverage AI to help them find and patch vulnerabilities at scale.

CRob (18:16.716)
I love that approach. Thank you for doing that. How did you fit the giant check through the teller window?

Michael Brown (ToB) (18:22.574)
Fortunately, that check was a novelty and we did not actually have a larger problem than the AICC itself to solve afterward, which was getting paid. So yeah, we did have the comically large check, you know, taped up in our booth at the AICC village at DEF CON and it certainly attracted quite a few photographs from passersby.

CRob (18:26.716)
Ha ha ha!

CRob (18:31.964)
Yeah.

CRob (18:37.864)
Mm-hmm.

Michael Brown (ToB) (18:47.736)
I don’t know. think if you get on like social media or whatever and you look up AICC, if you see anything, there’s probably lots of pictures of me throwing a big smile up and you two thumbs up underneath the check that random people took.

CRob (18:59.464)
So you mentioned that Buttercup is all open source now. So if someone was interested in checking it out or possibly even contributing, where would they go do that?

Michael Brown (ToB) (19:07.564)
Yeah, so we have a GitHub organization. We’re Trail of Bits. You can find Buttercup there. You can also find our public archives of the old versions of Buttercup. So if you’re interested in what, like, the code that actually won the competitions, you can see what got us from the semifinals to the finals. You can see what won us second place in the finals. And you can also download and use the version that’s actively maintained that’ll run on your laptop. And all three of them are there. Their repository name is just Buttercup.

We are not the only people who love the princess bride. So there are other repositories named butter. Yeah. There are other, there are other repositories named butter cup on GitHub. So you might have to sift it a little bit, but yeah, basically it github.com slash trailer bits slash butter cup, I think is like 85 % of the URL there. don’t have it memorized, but, but yeah, you can find it publicly available and, along with a lot of other tools that, trailer bits has made over the years. So we encourage you to check some of those out as well. A lot of those are still actively maintained.

CRob (19:39.036)
That’s what it was.

Michael Brown (ToB) (20:03.72)
have a lot of community support. believe it or not at last, I counted like something like 1250 stars. buttercup is only like our fifth most popular tool that, that trail of bits has created. So, you know, we, we were quite, we were quite notable for creating some binary lifting tools that are up there. we have also some other tools that we’ve created recently for, parser security, analysis like that, like graftage.

And then also some kind of more conventional security tools like algo VPN, like still rank above buttercup. So as awesome as buttercup is, it’s like, it’s like only the fifth coolest tool that we’ve made as voted on by the community. So check out the other stuff while you’re there too. believe it or not, buttercup isn’t, isn’t, isn’t our most popular, offering.

CRob (20:51.56)
Pretty awesome statement to be able to say. That’s only our fifth most important tool.

Michael Brown (ToB) (20:53.966)
Yeah.

Michael Brown (ToB) (20:58.444)
I don’t know, you know, like personally, I’m kind of hoping that maybe we move up a few notches after people get time to like go find it and, you know, and start it. But, you know, we, we’ve, we’ve made some other really significant and really awesome contributions to the community, even outside of the AI cyber challenge. So I want to really, really stress all of that stuff is open source. We, you know, we, we aren’t just doing this because we have to, we actually care about the open source community. We want to secure the software infrastructure. We want people to use the tool and secure the software for, you know, before they, before they, you know,

Get it out there so that you we we we tackle this like kind of untackable problem of securing this massive ecosystem of code.

CRob (21:37.606)
Michael, thank you to Trey Labitz and your whole team for all the work you do, including the competition runner-up Buttercup, which did an amazing job by itself. Thank you for all your work, and thank you for joining us today.

Michael Brown (ToB) (21:52.802)
Yeah, thanks for having me. You one last thing to shout out there. If you’re an organization, you’re looking to employ Buttercup within your organization. Don’t be bashful about reaching out to us and asking about use cases for deploying within your organization. We know we’re happy to help out there. That’s probably an area that we focus on a little bit less in terms of getting this out the door for average folks to use or for individuals to use. So we’re definitely interested in helping to make sure Buttercup gets used.

Like I said, reach out to us, talk to us if you’re interested in Buttercup, we want to hear.

CRob (22:23.44)
Love it. All right. Have a great day.

Michael Brown (ToB) (22:25.678)
All right, thanks a lot.

Join co-hosts CRob and Yesenia for a special season finale celebrating OpenSSF’s fifth anniversary and recapping an incredible year of innovation in open source security! From launching three free educational courses on the EU Cyber Resilience Act, AI/ML security, and security for software development managers, to the groundbreaking DARPA AI Cyber Challenge where competitors achieved over 90% accuracy in autonomous vulnerability discovery, 2025 has been transformative. We reflect on standout interviews with new OpenSSF leaders Steve Fernandez and Stacey, deep dives into game-changing projects like the Open Source Project Security Baseline and AI model signing, and the vibrant community conversations around SBOM, supply chain security, and developer education. With nearly 12,000 total podcast downloads and exciting Season 3 plans including AI Cyber Challenge competitor interviews, CFP writing workshops, and expanded global community initiatives in Africa, we’re just getting started. Tune in for behind-the-scenes insights, friendly competition stats on our most popular episodes, and a sneak peek at what’s coming in 2026!

00:00 – Celebrating OpenSSF’s Fifth Anniversary
02:52 – Educational Growth and New Initiatives
05:51 – Community Voices and Leadership Changes
08:45 – The Role of Community Manager
11:44 – Open Source Project Security Baseline
14:47 – AI and Machine Learning in Open Source
17:47 – Software Bill of Materials (SBOM) Discussions
20:34 – Podcast Highlights and Listener Engagement
22:26 – Looking Ahead to Season Three

• Yesenia Yser on LinkedIn
• Christopher Robinson on LinkedIn
• Cybersecurity Skills Framework
• OpenSSF’s Free Courses:
  • LFD 125 – Security for Software Development Managers
  • LFEL 1001 – Understanding the EU Cyber Resilience Act
  • LFEL 1012 – Secure AI/ML Driven Development

• OpenSSF What’s In The SOSS Podcast Episodes:
  • Podcast #27 – S2E04 Enterprise to Open Source: Steve Fernandez’s Journey to the OpenSSF
  • Podcast #29 – S2E06 Showing Up Fully: Meet OpenSSF’s new Community Manager, Stacey Potter
  • Podcast #25 – S2E02 Empowering Security: Yesenia Yser on Open Source, AI, and Personal Branding
  • Podcast #44 – S2E21 A Deep Dive into the Open Source Project Security (OSPS) Baseline
  • Podcast #36 – S2E13 From Compliance to Community: Meeting CRA Requirements Together
  • Podcast #40 – S2E17 From Manager to Open Source Security Pioneer: Kate Stewart’s Journey Through SBOM, Safety, and the Zephyr Project
  • Podcast #45 – S2E22 SBOM Chaos and Software Sovereignty: The Hidden Challenges Facing Open Source with Stephanie Domas (Canonical)
  • Podcast #23 – Kusari’s Michael Lieberman Talks GUAC, SLSA and Securing the Open Source Supply Chain
• OpenSSF Blog
  • OpenSSF at Black Hat USA 2025 & DEF CON 33: AIxCC Highlights, Big Wins, and the Future of Securing Open Source
  • From Beginner to Builder Series By Ejiro Oghenekome and Sal Kimmich 
    • Understanding OpenSSF Community and Working Groups
    • Your First Code Contribution
    • Free OpenSSF and Linux Foundation Education Courses
• OpenSSF Guides:
  • Visualizing Secure MLOps (MLSecOps): A Practical Guide for Building Robust AI/ML Pipeline Security
  • Cyber Resilience Act (CRA) Brief Guide for Open Source Software (OSS) Developers
• OpenSSF Annual Report
• Talks/Videos:
  • Open Source SecurityCon NA 2025: Lightning Talk: AIxCC Results and New Open Source AI Projects To Help Secure Open Source Software – Jeff Diecks (OpenSSF)
  • KubeCon CloudNativeCon North America (Atlanta) Keynote: Supply Chain Reaction: A Cautionary Tale in K8s Security – Stacey Potter (OpenSSF) & Adolfo “Puerco” García Veytia
• Email us if you are interested in being on the Podcast

CRob (00:05.428)
Welcome, welcome, welcome to What’s in the SOSS. Today I’m joined with my co-host, Yesi, and we got a really great recap for everybody. We’re gonna be talking about the whole last year’s season of What’s in the SOSS, some of the amazing people that she and I got to interview. Yesi, I’m excited to actually get to talk with you today.

Yesenia (00:13.58)
Hello.

Yesenia (00:30.318)
I know, I got co-host and I never got to co-host with you and here we go. But today’s exciting because it’s not just celebrating everyone’s impact and everything awesome that’s been done in the open source community, but this year’s actually OpenSSF’s fifth year anniversary. That was amazing. I just found out. I was like, whoa, good episode.

CRob (00:47.44)
Wait!

CRob (00:53.646)
Yeah, some of us have been around a whole five years, so it’s not quite a surprise, but hey. That’s right. So I mean, kind of looking back over the last year, we had so many amazing things that both our community did and then like we’ve highlighted through the podcast. You know, let’s you know, we had a whole section where we worked with our.

Yesenia (00:58.798)
But at least we’ve made it longer than COVID. That’s fine.

CRob (01:18.704)
Linux Foundation education team on a whole cybersecurity skills framework to try to help coach new people into the profession and try to help identify skills that employers would want to hire. And I know this has been talked about a little bit in the bear working group, right?

Yesenia (01:36.598)
Yes, it’s something that we’re also using to consider as we bring in more contributors that are newer to this space. This is like a really good framework and a functional structure of how we can bring in these folks and help them scale up as well as helping these open source contributors.

CRob (01:53.368)
Right, and as we’re upskilling, you know, the crew and the back was really busy. We issued three whole new courses this year. All three exactly.

Yesenia (02:02.318)
Free courses and across the different very important spaces because who isn’t talking about CRA and AI? right there. Like it’s right there for you. got an hour long video on each. You got a nice little badge at the end. And for our software development managers, we can also talk about security. So those are, you know, three new courses if you’re checking out on how to expand your education.

You have the LFD 125, which is your security for software development managers. Two on my bucket list because they impact my work, which is understanding the EU Cyber Resilience Act. That’s LFEL 1001. wonder, my binary math is a little rustic, but curious what that converts to. And then our secure AI ML driven development. This one, I know a few people in the…

The BEAR working group that I’ve taken it and good feedback and BEARRRR!. But not even these new courses, but just the group in general. We have new LFS, new OpenSSF members joining us.

CRob (03:14.96)
That was pretty cool. And I think you actually got the opportunity to interview Stacey when she started, right? She’s our new Community Manager.

Yesenia (03:23.456)
Yeah, if you haven’t worked with the open-ended stuff, you haven’t met Stacey’s great community manager, really there. I wanted to say a word, but we’re on live, so I can’t. But she’s really driving. It’s good episode too. She got on our podcast, shared a little bit of her background. And I know she works closely with the Bear community, helping drive a lot of the operations. But we also had a new general manager. You got to interview.

CRob (03:51.384)
Right, yeah. Yeah, my new boss Steve, Steve Fernandez joined us around the first quarter and he brings with us a real kind of business and corporation focused background. So he’s really helped kind of mature a lot of the stuff we do around here and enhanced the scope of the services that we offer the community.

Yesenia (04:13.006)
And there was one more. I don’t know, I can’t put my finger on it. There was one more new member. Hmm.

CRob (04:16.75)
Hmm

Well, we did have a new co-host this year. Hello?

Yesenia (04:22.306)
That’s right, it’s me! Yes! Sound more now!

CRob (04:28.747)
Very exciting. Yeah. And overall, the podcast kind of focused on current topics, new and interesting projects like our security baseline. We had a couple talks around CRA and I know that we are, we’ll kind of save this a little bit as a teaser for next year, but we did several talks talking about AI.

Yesenia (04:49.902)
And there we did also talk on the AIxCC, which, you know, they’re going ahead and pushing security into the future with their autonomous vulnerability discovery. I know working in my past that that autonomous vulnerability discovery is such a complex, huge issue that I’m excited somebody’s driving deeper into that and working with OpenSSF.

CRob (05:12.752)
And I think I mentioned to you in some of the podcasts, I came into the whole AIxCC competition incredibly skeptical that I was unsure of the value that AI tools would bring into this space. But after we got the results, I was just floored. The fact that like the top team had over a 90% accuracy rate in finding and writing a fix for vulnerabilities.

Yesenia (05:39.078)
Wow.

CRob (05:41.836)
The second place team was only in the high 80% success ratio…only. Yeah, like there’s some amazing stuff and that really kind of convinced me that this is there’s some value in this space. And I think there’s, I’m really looking forward to some of the collaboration with around the cyber reasoning systems and a lot of the new things we’re doing in the AI space right now.

Yesenia (05:59.663)
Do know if they’re continuing it for next year?

CRob (06:06.116)
The competition isn’t continuing, but we will be continuing to work with DARPA and ARPA-H and the different competitors. We’ve already lined up. You’ll see some podcasts coming out in the early next year where we’re talking to the different competition teams. And several of those groups already are working to donate their software to the OpenSSF to help continue to grow a community and continue the development and refinement of these systems. There’s going to be some amazing stuff out of the AIML Working Group next year.

Yesenia (06:24.087)
Nice.

Yesenia (06:34.68)
Yeah, because I can just imagine with the percentage of the intrigue, just the research and the technical architecture of how they designed this to be able to produce such results. I know it’s going to be a huge impact into our open source and the security overall. But it’s for one year, you know, we had educational growth, governance, maturity, policy collaboration, our supply chain security. That’s one of my favorite words. I got earrings for it. It’s sharper.

CRob (06:51.631)
Yeah.

Yesenia (07:04.864)
And then you got AI, know, the preflow of that that’s come in. It’s really hit the open source in a real way. And I’m excited and I love that the podcast is capturing how we’re evolving in these spaces with the voices from our community.

CRob (07:19.172)
Mm-hmm. So let’s talk about those community voices a little bit. You mentioned that I had the opportunity to kind of talk with Steve, our new General Manager, and it was interesting that, know, Steve spent some time in his podcast, which was the title was Enterprise to Open Source, kind of talking about Steve’s journey. And he really kind of focused in on how his decades ofbeing a consumer of open source really is forming his current role as a steward of open source right now.

Yesenia (07:56.931)
Yeah, after listening to that, it was, it was, it’s understanding of why he got the position considering his background in the space, like, and just since he started the changes that’s happened in open source and the growth of what is, you know, Steve’s vision from where he bridges enterprises’ risks mindset with that of open source. Like that is something we definitely need to consider when it comes to it, because one of their major consumers is our enterprises.

And I know he’s played a big role in the Baseline and maturing that foundation of it. From listening to the episode, I know he talks about like those decades of consuming and then stepping into this and really calling security a hidden greatness, which is the work that you only notice when it’s missing or you get impacted, right? And this is for even the everyday person is like, you won’t realize that you need that security privacy until youknow, credit cards are stolen, right? So, but for him really coming in and turning those enterprise pain points into what is OpenSSF roadmap this year, and the greater is really helping organizations ship safer software.

CRob (09:09.88)
I agree. Now let’s talk about showing up fully. This was the interview you did with Stacey, our new Community Manager. What highlights would you like to share out of that conversation?

Yesenia (09:19.874)
This one, I love this, this is one of my favorites. Stacey came in and she’s had this background of becoming a community manager for open source communities. And she really kicked the ground running and was pushing that train. Like she’s behind that train moving it. But her real focus was around belonging, that authenticity, the inclusion and connecting BEAR with DevRel. And even though they’re two different working groups under us.

We have a very similar mission, just a different scope. So being able to come in as a community member and really ground how much community work is underpinned and all the technical work. I’ve seen her show up fully to the calls to not just Bear, but the other working groups and just making sure that she drives that community first mindset. And she connects with the maintainers, the members, the newcomers, and just making sure everyone’s being heard and felt. So,absolutely love that and you know there’s so much more into that 

CRob (10:24.24)
She also does a pretty amazing keynote.

CRob (10:29.968)
You’ve got to watch the video. It’s amazing.

Yesenia (10:45.43)
and I didn’t get to see the keynote but I you gotta watch it I’ve heard so many with her and Puerco

CRob (10:53.368)
And that’s, I think another interesting thing kind of pivoting around the community manager role. We have so many things going on across all the technical initiatives and working groups. It’s hard to kind of keep track of all of it. And that’s why having this role of that community manager is so important to be that connective tissue between our folks in the community that are contributing with staff, with the Board and the TAC. So it’s really important to have that, that role to help keep us balanced and focused.

Yesenia (11:22.306)
Yes. And let’s not forget, the podcast wouldn’t be the podcast without Stacey sitting here, listening to us, editing, publishing it. Big kudos if you ever see Stacey and you do her podcast. Please let her know she’s working really hard behind the scenes. She’s listening to us right now. So tons of kudos to her.

CRob (11:39.352)
Absolutely. Well, thinking about, thinking about what came up next is, the Open Source Project Security Baseline was a big effort for us, both in our community and within the whole broader LF. We did a, yeah, yeah. And we did a great podcast with two of the maintainers, Eddie Knight and Ben Cotton. And the title was a Deep Dive into the Open Source Project Security Baseline. And, you know, I thought that was.

Yesenia (11:53.932)
You helped push a lot of that.

CRob (12:08.752)
Pretty amazing little chat because both Eddie and Ben approached this project from the perspective of an upstream maintainer. We want to do whatever we can to remove work and burden from upstream and allow them to focus on creating amazing software and not necessarily have them have to worry about a compliance checklist, so to speak.

Yesenia (12:33.998)
And what I know with the Baseline, it ties together several projects.

CRob (12:39.596)
Yeah, have the Baseline itself is the catalog, which is the brains of the whole operation. And that details a list of requirements that should be done in the course of software development, publication and consumption. And then we have the orbit working group, which actually is the kind of the home for the baseline. And the ORBIT working group has a series of software projects.

That help try to automate or enable a lot of these different techniques. So we have things like around managing, making policy-based decisions in your CI pipeline, like a minder or a Gemara. We have a security insight spec that’s all part of the Orbit Working Group. And that’s a way for people to express how they are achieving some of these requirements. So like, for example, if you’re a project and you make, you issue SBOMS.,

You can make a security insights file to tell people how to find your SBOM. So they don’t have to come continually emailing you asking you for more information.

Yesenia (13:47.119)
And I heard a very quotable famous quote come out of this podcast, which was, oh, we got to put this on a t-shirt. “Give maintainers a way to show their security work, not just promise it.” Because that’s a huge thing. You’re working on these projects day and night in the stereotypical basement. And no one really cares unless they’re impacted, not in that sense. But it’s nice that we could show.

Have a way for maintainers to show their security work, give themselves a kudos and acknowledgement for the hard work putting it together.

CRob (14:19.279)
Right.

CRob (14:23.21)
And that’s where I’m very excited. And this kind of ties in with Steve’s vision and strategy is that projects like Baseline or SLSA, these are things that help downstream, meet your boardroom expectations. But all of these things are created and curated by the community. So again, we try to wherever possible focus in on the maintainer experience and making things easier. And I just love thatkind of dual purpose that we’re trying to help both up and downstream at the same time.

Yesenia (14:56.824)
Yeah. And then this year we also going back into those educational pieces, like some other episodes we talked to it was, you know, David Wheeler’s new AI ML Development Software. We have the Cybersecurity Skills Framework that we talked about earlier. And from there, we had that conversation with Sarah. think you interviewed Sarah on the AI competition model signing. What was your takeaway from that?

CRob (15:25.168)
Yeah, that was really great. So that was right as, so we have an AI ML Working Group is one of our technical initiatives and they’ve been around for about three years. And it was a little bit of a slow start where they did a lot of talking and evaluating and kind of setting up liaison relationships with the, there’s a whole cast of characters that are involved in AI security in the upstream ecosystem. And when I talked with Sarah, it was right after they’d had two publications.

The first was an AI Model Signing Project where they were leveraging a Sigstore and In-toto to help consumers understand, here is a signed model or a signed artifact. On this day, theycreated this artifact and it’s been untampered with since. So again, it’s trying to help provide more information into the pipeline so people can make risk-based decisions.

Yesenia (16:02.837)
interesting.

CRob (16:21.774)
And then right after that, they also released a white paper and of talking about how to integrate DevSecOps practices into machine learning and LLM development. And that’s been a really important artifact where it’s helped us realize, recognize that there are a lot of people involved in creating air quotes here, AI stuff, whether it’s an application, you’re training a model, you’re trying to go to market with something. There’s a lot of personas that are involved and onmost of them aren’t classically trained software engineers or cybersecurity practitioners. So the white paper kind of highlights these other people that participate in this creation process and talks about some techniques that are both old, you know, from AppSec, what we’ve done for 25, 30 years that have worked well, that could be applicable in the AI space. But then they also talk about some new ideas because these technologies are a little different and it does requiresome new ways of thinking, of being able to interrogate the different gizmos, whether it’s GPUs or eGenTech. So each technique requires some a little bit different tools to help protect them.

Yesenia (17:33.487)
Yeah, I’m glad you brought up the white paper because I was about to be like, I read the white paper. It was actually a good piece of knowledgeable guidance and information on how to Model Sign that I’m bringing into my own industry. It’s a good read, you know, and then we have other reads like the CRA compliance that we had a conversation with Alpha-Omega and the Erlang group. Those are also two good episodes to watch or to listen to.

Yesenia (18:03.522)
When it comes to the CRA. But, you we’ve talked about Baseline, we talked about GUAC, we’ve talked about SLSA, but the other card on, you know, the other bingo card for 2025 is SBOM. What episodes do we have on that?

CRob (18:14.746)
That’s right.

What episodes did we have on software bill of materials? 

CRob (18:51.608)
Right, we did do several things around SBOM. We had the opportunity to talk with Kate Stewart, who’s been a leader within the software build material space almost since the beginning. She represents SPDX, which is one of the two tools that most people use to create software builds materials, with the other one being Cyclone TX that our friends over at OWASP care take. And that was really interesting kind of talking about Kate’s perspective of the evolution of these things.

And then more recently, I had the opportunity to talk with the chief security officer of Canonical, my former coworker, Stephanie Domas. And we talked about a bunch of different things. And SBOM was kind of wrapped up in that conversation and talking about just challenges within the current regulated space that both commercial entities like an.

Yesenia (19:26.445)
Ooh.

CRob (19:41.546)
Canonical will face, but also upstream open source maintainers as well. So really engaging conversations around supply chain and software bill materials. The GUAC conversation was also really good and kind of important. That’s a very useful tool to help you get wisdom out of your SBOMS. Wisdom.

Yesenia (20:00.601)
Wisdom. Word of the day. It’s awesome. Considering it’s OpenSSF’s fifth year, just this year’s reflection on podcasts, we’ve really covered on multiple areas of the community, has been working on. And just my favorite thing about this whole thing is the little competition that’s going on against these podcast episodes where our guests have come in and asked, what’s my number? What’s my view? So as of today’s recording, we have the Mike Lieberman’s talk on GUAC SLSA and securing open source at 611. GitHub’s Mike Hanley, transforming department of NO . at 406.

Yesenia (20:55.886)
Eric Brewer and the future of open source software at 370, Vincent Danen and the Art of Vulnerability Management at 328. I’m so glad my dislexia, is not switching these numbers. And lastly, we have Sonatype’s Brian Fox and the Perplexing Phenomenon of Downloading Known Vulnerabilities at 327. So if you want to help these folks out,

Yesenia (21:25.644)
Give it a listen and let’s see if we can change the top episodes by the end of the year.

CRob (21:31.024)
It’s kind of a curious peek behind the scenes where guests will come in and do their podcast. And they’re very interested. It’s not vanity, but people like to hear that their work is valued. And so there is very healthy competition and some little bragging rights that Mr. Lieberman will kind of say, well, I have the most downloaded open as a podcast. So it’s just kind of fun, like a friendly little healthy competition. And again, and focusing in on some of these key areas of supply chain security, application development, software build materials and such.

Yesenia (22:06.19)
Yeah, it’s crazy to see that we’ve, across all the episodes, been about 11,800 total downloads and just 6,000 in 2025. So big thank you to our listeners, our supporters for that. I think it’s the first year of this podcast or second.

CRob (22:24.526)
Second, the second. And that actually kind of gives us our segue towards the end here. We’re talking about a lot of things that happened during 2025. And we are about to publish our annual report where you can kind of dive in and double click on some of these details. We’ll provide a link as this podcast is published that you can look at the report that will link into things like our five-year anniversary or our work with DARPA on AICC or all these amazing things around the baseline. So that’s, I’m really excited to kind of share that annual report with everybody that touches on a lot of the topics that Yacenya and I have talked through and many others. And that kind of moves us on. We’re going on to bigger and better things. 2026 is going to be season three.

CRob (23:17.88)
And I think we’ve got some really interesting topics kind of queued up.

Yesenia (23:21.464)
Are we gonna share? Are we gonna share? Are we gonna be nice to our listeners?

CRob (23:24.944)
I think everyone’s on the nice list. We can share that with them. Yeah, you’re going to see us starting off the year with kind of a full court press around AIxCC and AI and ML security topics. We have a bunch of work queued up with some of the cyber reasoning system competitors. We’ll talk with some of the competition organizers, again, talking more with our community experts around these very important topics and maybe unveiling some new projects that our AI team has in the hopper. That’s going to be very exciting. We’re going to have some very special guests from around the world of open source and public policy and research. And we’re going to have some very recognizable names that may have been in the show or a part of our community’s orbit we would love to reengage with and talk more with.

CRob (24:21.358)
So thinking about, you’re going to see multiple series of episodes around the AIxCC competition in particular. We’re going to be focusing in on industry and research stars. So we’re going to try to find some well-known voices out in the research community, joining some of our maintainers and kind of talking about some big picture conversations in the ecosystem. And then you’ll see many more things around our education efforts.

Would you like to talk about some of the stuff I know that Bear’s preparing to do?

Yesenia (24:51.822)
For Bear, we have very exciting things for next year. Not associated with that sports team. We have the next mentorship for the summer that we’re going to be producing. We’re working towards those details. We’re working with a group out in Africa for having an open source, what is it called? Open Source Security and Software Africa Group for the primary focus on doing speaking engagements, holding meetups and conferences in Africa. Cause there’s a huge community group there that have nowhere really to go. with global restrictions and visas, they’re very limited. So helping them kind of grow that out, share out some tips and tricks that we’ll be sharing on social just to drive more awareness into these projects and to these teams. And of course our community office hours, which have also had a lovely set of community members that have come in and shared their journeys, education pieces and blogs that have been recently produced like, Sal and Ejiro have produced about newcomers into the open source. We’re working on getting part three released, but you can find part one and two out in OpenSSF’s blog main page.

CRob (26:39.576)
Excellent. And I’m also excited that we’re going to be doing some special education segments on the podcast around how to write a good call for papers abstract. And then how to build your first conference talk, which is something that again, a lot of these newcomers haven’t had experience with that. Some of us that have been around the block a little bit can help share some of the wisdom we’ve earned over the last couple of Right.

Yesenia (26:46.83)
Yes.

Yesenia (27:02.358)
My try on era.

CRob (27:07.512)
And with that, I want to thank you for coming on board and being our co-host. You’ve really brought in a nice set of energy and a fresh perspective when you’re talking with our community members. And I wanted to remind everybody, as we are preparing for season three, if you have ideas or suggestions for topics, please email marketing at openssf.org. We would love to hear your episode pitches, your CFP stories, if you want to do some demos or have case studies.

Yesenia (27:12.119)
Absolutely.

CRob (27:35.822)
Or you just have just general projects that help the broader OpenSSF mission of improving the security of open source software for everybody forward. So thank you again, Yesenia. It’s been a pleasure. I’m looking forward to another exciting year of talking with you again. All right. Happy open sourcing, everybody.

Yesenia (27:50.776)
Thanks, CRob. To the next episode.