Mike Hanley soundbite (00:01)
Core to everything that we do is this thesis that good security really starts with the developer. Internally, for our engineers, that means the developers who are Hubbers. But it’s also all of our customers and all of the developers and all of the open source maintainers and communities that are using GitHub.
Omkhar Arasaratnam (00:18)
Welcome to What’s in the SOSS? I am this week’s host, Omkhar Arasaratnam. I’m also the general manager of the OpenSSF. Joining us this week is Mike Hanley. Mike is the CSO and SVP of engineering at GitHub. Prior to joining GitHub, Mike was the vice president at Duo Security where he built and led the security research, development and operations function. After Duo’s acquisition by Cisco, for many billions of dollars in 2018, Mike led the transformation of Cisco’s cloud security framework and later served as CISO for the company. And when he’s not talking about security at GitHub, Mike can be found enjoying Ann Arbor, Michigan with his wife and eight kids. Hold on. Do we need a pull request, Mike?
Mike Hanley (01:02)
I think we do. I think we need to update that to nine kids.
Omkhar Arasaratnam (01:04)
Well, congratulations to the Hanley family on a ninth.
Mike Hanley (01:08)
Thank you, Omkhar I appreciate it.
Omkhar Arasaratnam (01:11)
I’ve got to ask you a question that every parent has probably asked you. How do you transport your family around? I mean, logistically, I’m curious.
Mike Hanley (01:21)
We have one of those amazing vans that looks like we’re running a summer camp. We have a Ford Transit, and it’s all-wheel drive, because I’m in Ann Arbor, Michigan, so we’re a four-seasons kind of area, so we need to make sure we can get around in the snow. But it’s great. It’s just got the side door, so when we get to school, everybody just sort of files out in a line, and we throw the door shut, and we’re off to the races. So Ford Transit, 12 seats. I got space for one more. (Laughter) So we love that thing. It’s great for getting around with the whole fam.
Omkhar Arasaratnam (01:48)
That’s amazing, a feat of engineering. Speaking of engineering, you’re the CSO and SVP of engineering at GitHub. Can you walk me through your role overseeing both of these teams and what it means for you, what it means for how you think about secure software development at GitHub?
Mike Hanley (02:02)
Yeah, so I initially joined GitHub a little over three years ago to be the first chief security officer. So bringing all the security programs together and then sort of writing the next chapter of that story. And about a year after I got to GitHub, so a little over two years ago, I said yes to the opportunity to also take on running all of engineering at GitHub. And having those two hats may seem a little bit unique on the outside, but I think it’s actually very natural for us at GitHub because core to everything that we do is this thesis that good security really starts with the developer and internally for our engineers, that means the developers who are Hubbers at GitHub building everything related to GitHub.
But it’s also all of our customers and all the developers and all the open source maintainers and communities that are using GitHub. We want security to start with them as well and start with in a way that works for them as well. So secure in their accounts. So things like the 2FA work that we’ve done, but also that it’s easy for them to get security value into the projects and communities that they are a part of.
Omkhar Arasaratnam (03:05)
You’re the home for the world’s developers, and GitHub has this unique role in helping to secure the software supply chain, which is an area that can be really difficult for leaders to understand in depth and breadth. How are you advising organizations that are on their journey to better understand their supply chain? What would you advise security leaders or developers tackling this issue?
I mean, as, I consider myself a software engineer who’s been doing security for a long time. Not every security leader has a background like you or I, where it comes from a place of software engineering. I’m curious how you articulate the complexity to those that may be more oriented towards risk or infrastructure, things of that nature.
Mike Hanley (03:53)
Yeah, I think the biggest danger, Omkhar, in terms of organizations or teams understanding their supply chain risk is they think too small or they have a too narrow of a lens. It’s like, well, my supply chain is secure because I know or have an attestation as to what totally composes the software that I’m actually building, packaging and publishing. And while that’s good, and certainly it’s something that organizations should do — and if they’re doing it great pat on the back to them because frankly, they’re already a leg up over a lot of other places that are doing nothing — but it’s important not to stop there.
It’s…we’ve seen this actually with if you look at a lot of the mainstream incidents that have kind of hit the news in the last few years, of course you’re very familiar with all these, but they’ve involved things like attacking build systems. Some of them have been back doors in software
Some of them have been insider threats, and you sort of have this range of potential attack vectors on the security of your software. So I think you need to consider things like, of course, having an inventory of what’s there and doing, you know, sort of the bread and butter vulnerability management that you would, and dependency management that you, that you would or should be doing as a best practice.
But it’s also considering, you know, how do I trust or how do I assess the organizations that are actually producing the software that I depend on? Are they secure in their persons? How do you think about the accounts and communities that are actually contributing to that work? Do I understand all the third-party integrations and tools that I’m using, of which many organizations I would suggest don’t have a full inventory of many of those? When we look at third party integrations on GitHub, there are a vast sort of sea of those that customers and developers and organizations use.
Having an understanding of like, what’s plugged in where? What do they have access to? The people who develop and operate that integration or the service I’m integrating with, what’s their security look like? And I think it’s really just understanding this very broad kind of network of effects that happens goes well beyond just, like, the idea that somebody could potentially commit a backdoor, which is obviously an important thing to assess or that there might be a vulnerability and a dependency that you have. These are actually important things to assess, but your threat model needs to be much, much broader than that.
And I think for every organization that’s really worried about their code getting backdoored, good, they should be thinking about that. But they also need to make sure they actually go look and say like, what third-party applications of our developers authorized sort of full and unfettered access to our build systems on? If you haven’t done that, you need to make sure that you’re looking at some of those things. And this has really actually informed a lot of the decisions that we’ve tried to make at GitHub in the course of the last few years.
I mean, the one that I think is what I hope will be one of the most impactful things that’s happened in supply chain security in the last several years was actually driving two-factor authentication adoption. And you’ll remember the sort of package hijacking attacks that happened on the NPM ecosystem in late 2021. And that was a really interesting set of learnings for us because the adversary incentives are so clear. If I can just get one piece of code to run a Monero miner on this package that’s in the top 500 on NPM, I’m going to get at least 10,000 downloads an hour out of that.
And so the incentive is very high to go attack those accounts, but the vast majority of maintainers and developers that we found at the time actually weren’t using 2FA. And it’s sort of interesting to look and say, well, in the IT world, if you started a corporate job tomorrow, you would get a managed laptop, an account, and something that was enrolled in 2FA, because that’s now a best practice and a standard in IT. Yet we don’t have, or we haven’t quite caught up with those types of standards and best practices in the broader sort of developer ecosystem.
So I think the 2FA work that we did — while it was hard to get many, many millions of developers enrolled in 2FA — that’s the kind of thing that just raises the bar for everybody and it substantially increases the attack cost of a supply chain attack because you’re kind of crowding out account takeover and sort of these other low hanging fruit that would most commonly be exploited by a bad guy.
Omkhar Arasaratnam (07:38)
I think that’s a great point and thanks for all the great work that you all have done in order to increase some of the security basics like 2FA more broadly. I want to jump back to your role at GitHub. And as I mentioned before, as somebody that personally identifies as a software engineer who’s been doing security for a really long time, you know, software engineers the world over have oft complained about the security department coming in and saying, you can’t do this, blah, blah, blah. And then their velocity slows down and they can’t do the cool thing because security person said no. Both teams roll up to you. How do you balance that? How do you, how do you see that impacting your day to day? Is that a, is that a good thing? Bad thing?
Mike Hanley (08:19)
What you described is what I often call the Department of No. And really, I think a modern security team, especially in a modern software-driven organization, of which the vast majority of companies are at this point, you have to be the Department of Yes And. Which is, yes, we want to do the new thing. We assume that our engineers, our finance people, our marketing people have the right goals for the company. And we want to figure out how to make that particular endeavor successful and contemplate the risk management needs that we have as a company.
We want to make sure the good thing happens while making sure that the bad thing does not happen. I think for us internally, having both teams under one roof while there’s traditionally this separation that you mentioned, that separation comes with challenges because it actually, in many cases, encourages the security team to sort of sit in the ivory tower and come down and say, well, Omkhar, you can’t do that because reasons. And usually that engagement happens
at the 11th hour, right? I mean, it’s very rare that you get sort of negative security feedback early in a model like that.
And I find that by having the teams together, I mean, literally all the security leaders are sitting in the same calls as the engineering leaders because they all are part of the same team. And we have this notion internally that really everybody in the company is on the security team. And obviously, that means something specific for engineering because they’re doing development work that impacts the security of the company, the security of our stakeholders. They’re building things to make sure that we can realize good security outcomes, especially for open source maintainers and our paying customers. So they kind of have a threefold mission there where they’re helping with security.
But it’s also true for the finance people who report a gift card scam from our CEO or the HR and recruiting folks who are looking out for fake potential employees who are trying to scam their way into the company. So that idea that everybody’s on the security team is really a cultural approach for us to make sure that everybody’s a part of that broader network. And so this is basically the antithesis of this idea that humans are the weakest link. In fact, we view them as the strongest part of our security program and actually an enhancement of all the tooling that we have by getting everybody engaged.
But specific to engineering, I think it’s great because it actually makes sure the incentives are tightly aligned, right? Like I’m responsible for making sure that the website is running and secure all the time. Like those are the two things that are effectively my set of responsibilities to the business and they are not in any way intention. In fact, if you look at the vast majority of our investment profile, it is actually going toward those two things. Now we’re actually doing a lot of net new features and we’re building new things all the time and we’ve got you know experimentation on bets that we want to place in the future, but the overwhelming amount of our work goes to to those two priorities that I mentioned a minute ago.
You know we’re really geared toward like making sure that security is a good experience for everybody not not a bad one, which doesn’t mean we don’t say no from time to time. But I think you minimize the number of no’s by having security as deeply engaged in what’s happening at the edge as possible, because the security team can’t actually be everywhere all at once. Our security team is not equivalent to the number of engineers that we have in the company. I don’t know. I’ve yet to meet anybody who can say that. And I don’t think I will in this lifetime. Certainly not with the jobs reports that we see that suggest there’s a vast shortage of security talent and expertise. And certainly you and I see this when we’re thinking about how to help open source maintainers. It’s just not out there. It doesn’t exist en masse. And it doesn’t exist in sort of broadly available sense.
But if you’re really leaning into the idea that we want to help the engineers and you recognize that security team is not going to be a part of great security work all the time, you actually want to make sure the engineers are the superheroes from a security standpoint or the finance folks are the superheroes from a security standpoint and you shift that mindset toward one that’s actually trying to drive that outside the security team. This actually works really, really well for us, and I think creates a really tight coupling between those two teams. And it also allows us to, I think, focus on like higher order concepts.
So for example, the idea not just that we do security, but that we do it well, and we do it in a way that’s actually a great experience. So we talked a little bit about 2FA, spending the time with the design team and the product teams to figure out what is doing this well and in such a way that it’s not alienating, that it’s actually a good experience, that people actually adopt it at scale without feeling like it’s being foisted on them in a counterproductive way.
Because when you end up in that scenario, people find a way to get around your security controls, whether they’re your internal developers or stakeholders that you’re affecting with an initiative like that. So really having everybody together under one roof driving those common goals, I think has actually been like very, very healthy for both the company’s internal security objectives, but also for our ability to affect broad change in the open source ecosystem as well with things like that 2FA initiative that we did.
Omkhar Arasaratnam (12:44)
I think you’re spot on. Humans are ingenious. Humans will want to do the thing that provides them the least friction. Let’s make the security, the secure thing, the thing that provides least friction. Your story about everyone’s accountability being security. It’s not just the security team, it’s finance, it’s engineering, et cetera, reminds me and sounds a little corny, but 20 years ago when I was just getting into security, I was at this conference and everybody had this lanyard that said, I’m a firewall. And, you know, 20 years later, it seems kind of corny, but the idea was security was everyone’s accountability and it didn’t need to be backstopped by SecOps. If somebody at the frontline could stop that upstream, the benefit was even larger.
Now, if we switch gears a little bit and think about everyone’s favorite topic, AI. How do you think AI will impact open source security, and what ways do you see it helping us to secure the software supply chain?
Mike Hanley (13:42)
My view on this is very bullish and it is summed up as this. I think AI will be two things. One is it will redefine the idea of shifting left in software development, which is, you know, we’ve been talking about this for more than 10 years at this point. The idea obviously that you move security value as far left as possible. What that’s meant to date has been, you’re generally getting feedback sort of at CI/CD time, right? Like after you’ve written the code, submitted it for review and it’s going through testing, that’s typically when you’re getting feedback. So that’s left obviously of boom in most cases where boom is your thing has been deployed to production, you find a vulnerability or you get breached.
AI is basically saying when you are in the editor, bringing your idea to code through the keyboard or other input device, at that moment, you don’t just have security expertise, you actually have an AI pair programmer who has a vast range of knowledge and expertise in a variety of topics, accessibility, security, scalability, the particular language or framework that you’re using, right there with you, giving you more secure suggestions, helping you check your work, and literally just paired with you the entire time.
So I think that is the realization of what we really want with shift left, because you can’t actually go any further left than when you’re bringing your idea to code in the editor. And I think that’s the highest value place to add security value and it’s also the best experience for developers because you are in the flow doing the work in that moment, not, hey, I committed my changes up, I’m waiting for tests to run while I go get lunch and then I come back or maybe even it’s the next day depending on how slow your build systems are.
The next day I get feedback and I gotta go, what was I thinking at that moment when I did this? So that shift left and adding security value right in the idea in real time is huge and that is uniquely available to us because of the advances in AI, certainly with GitHub Copilot, but there’s others out there as well that are trying to do similar things. So that’s one.
The second is AI gives us an opportunity not just to help address vulnerabilities in new and existing code, right? So this idea that as I’m writing it, I can get higher quality, more secure code in real-time, but also I’m adding value to things like my CI/CD. So it’s, I’m getting better suggestions from automatically getting suggested fixes from some of the tooling that I have now, instead of just giving, hey, there’s a problem here. It’s there’s a problem here and here’s how you should fix it. So there’s tons of value there, but, but it also enables the idea to retrospectively fix things.
And I think this is one of the like grand challenges that we’ve had frankly in, in security generally, but especially in open source security Omkhar as you know, like a lot of the building blocks that we have for everything that we do and experience are maintained by a small number of people who don’t necessarily have robust access to security resources. And in many cases, fundamental building blocks of the internet are unmaintained at this point. Major projects that people depend on are unmaintained. And that is a huge challenge for the whole ecosystem.
But the idea that, particularly through AI and agents, that we might actually be able to at scale
ingest process and refactor or fix bugs at scale in open source projects to me is super exciting, super compelling. It’s a way to supercharge, not just your internal developers who are writing first party code, but actually help supercharge open source developers or open source communities and really empower them to go do this work. They’re like, they may not be incentivized to do, they may not have the resources to do, they may not have the expertise as part of their community of maintainers and their project to go do. That to me is super compelling.
And you know, it’s interesting when you say like, well, we’re dependent on these things, if only we could figure out what to do about it. And often, we talk about, well, we can deploy people to go help. And yeah, that’s good. That helps for the short term. But that doesn’t scale. And it doesn’t necessarily help with these massive excavations that it takes to go back into projects that have 10, 20, or more years of history behind them. So I’m excited that AI can actually help us get the leverage that goes beyond what we can do with human scale, especially where we have tactically deployed things or sort of volunteer firefighter resources that can go help with a small number of things. AI is going to give us the lift, I think, to go scan and support and help fix and really renovate, maybe is a nice way to put it, some of the projects that we depend on for the next 10, 20, and 30 years.
Omkhar Arasaratnam (18:02)
That’s a great point, Mike. And it’s also why we look forward to programs like the AI Cyber Challenge from DARPA and seeing the work that’ll come out of that. Switching gears a bit, we’re going to go into rapid-fire mode. So I’m going to ask you a series of questions, give you a couple of possible answers. And of course, it’s up to you whether to choose one of them or say, hey, Omkhar, you got it wrong. Actually, this is how I feel. So are you ready to go?
Mike Hanley (18:30)
I’m ready, go.
Omkhar Arasaratnam (18:32)
Spicy vs. mild food?
Mike Hanley (18:35)
It’s basically always spicy for me.
Omkhar Arasaratnam (18:37)
Always spicy. I knew we got along good for a reason. Now this is a controversial one. We talked about bringing AI to the IDE. So let’s talk about text editors, Vim, VS code or Emacs?
Mike Hanley (18:48)
My tastes have changed over time. I would say I’m more of a Vim guy now, but at times I’ve actually just used Nano, which I don’t know that that’s a popular answer. (Laughter) I’m not proud of that, to be clear, but at times that was the preferred editor in a prior life.
Omkhar Arasaratnam (19:03)
Why does Nano bring such shame? (Laughter) All right, tabs or spaces?
Mike Hanley (19:09)
I’m generally a tabs, tabs kind of guy. Keep it, keep it simple.
Omkhar Arasaratnam (19:12)
Awesome. Alright, closing things out, Mike, you’ve had a wonderful illustrious career so far. What advice do you have somebody for somebody entering our field today? Be it from a software engineering or security perspective?
Mike Hanley (19:26)
I think for either one, find an opportunity to meet people and get involved and do something, have something tangible. And the great thing about open source is this is actually one of the very best ways to build your resume because your work is easily visible. You can show people what you’ve done. You see a lot of resumes, a lot of experiences, a lot of schools.
But what can really differentiate, I think, is when you can say, like, here’s a thing that I did, that I built, and I learned something unique from that. And you don’t always necessarily get that from just going through things like coursework. When you’ve really had to, like, duct tape something together and deal with the messy reality of getting things to work the way you want them to work outside of a sandbox, I think there’s a lot of value and sort of real-world knowledge that comes from that that is impossible to…there’s no compression algorithm for experience is a way I’ve heard this put previously.
And just hacking on the weekends with some of those projects or finding some people who you want to work on was for the sole sake of just learning how to do something new is incredibly valuable and it’s a great way to stand out.
Omkhar Arasaratnam (20:25)
That’s great advice. A mentor of mine once said, well, you should never test in prod learning and prod is a, is a useful lesson.?last question for you. What’s your call to action for our listeners?
Mike Hanley (20:38)
I think it’s similar. It’s getting involved in some meaningful way. But I think going back to some of the things that we talked about earlier, Omkar, just to give a slightly different answer to that, would be ask some hard questions in your organization about your understanding of open source security, and particularly your supply chain. I do think when I think about 20-plus years in the security space, it is one of those things that stands out to me as unique in that many organizations in security still don’t do the basics. I mean, it is 2024, and we still have to tell people to turn on two -factor authentication. So that is what it is, I would say.
But it’s also true that not every organization does a great job with inventory management and with configuration management, or even just sort of mapping out their broader ecosystem and infrastructure. And I think just going back and saying, like, do we really understand our supply chain? Do we really understand our dependencies? Do we really have, like, provenance of our assets, our inventory, our vulnerability management?
So I think again, in the context of open source security, like really just going back and saying, like, do we really know how we use this stuff and challenge maybe what the assumptions have been in the previous weeks and months and years? I guarantee you that whatever you’re looking at is far too narrow. And I think that can be an important conversation. It can really help your organization up-level its supply chain because again you might find healthy initiatives out there that are low-hanging fruit that you can take advantage of, so that’d be maybe a call to action is to have that conversation in your next security team meeting.
Omkhar Arasaratnam (22:02)
Great advice from Mike Hanley, the CSO and SVP of engineering at GitHub. Mike, thanks so much for joining us on What’s in the SOSS? and we look forward to having you on again soon, perhaps.
Mike Hanley (22:13)
Thank you Omkhar, great to be here.
Announcer (22:15)
Thank you for listening to What’s in the SOSS? An OpenSSF podcast. Be sure to subscribe to our series of conversations on Spotify, Apple, Amazon or wherever you get your podcasts. And to keep up to date on the Open Source Security Foundation community, join us online at OpenSSF.org/get involved. We’ll talk to you next time on What’s in the SOSS?
