Securing the Supply Chain in WordPress

Tom:
Okay, well, we’re back. We’re joined by John Blackbourn, who leads the WordPress core security team. I’ll ask you to give us a bit of background, John, on yourself, just to introduce yourself. And then, I think to start, I know we’ve got WordPress 3.8 coming down the pipe, and there’s some interesting security stuff headlining that release. I think that would be a good place to start. Tell us a bit about that.

John:
Sure. Yeah, that’s right. So I work as the Director of WordPress Security at Human Made, and primarily that role means that I am working in the capacity of the WordPress core security team lead, with my time sponsored on that team. I’ve been involved with WordPress for a long time now—longer than I thought, actually. I think it’s past 18 years now. Yeah, I was looking back through some of my old tickets—it’s 18 years now—and in recent years, I’ve taken much more of an interest in the world of web application security. So this role fits me well. And yeah, everything is going pretty well so far, especially with the WordPress 6.8 release, which is due to be released next month, in the middle of April, I believe.

Tom:
I realize I just said 3.8, so that’s me dating myself. It’s a long time ago that we had 3.8.

John:
So I’m—

Tom:
Glad you corrected that.

John:
Yeah, 6.8.

Tom:
6.8, okay.

John:
Yeah, so I can talk about some of the security enhancements in 6.8 if you like.

Tom:
Yeah, that would be great. I know one of them actually was a pretty old ticket that finally got closed. So again, dates—those of our listeners who were around when that was filed—

John:
And some of the listeners may have even been part of that ticket. You never know.

Tom:
Yeah, true.

John:
Yeah. So one of the headline changes for 6.8 will be a change to the way that passwords in the database are stored. It’s a change to the underlying mechanism for how they’re hashed and then stored in the database. They will be switched from an older mechanism called PHPass to an industry-standard mechanism called Bcrypt. This is widely used—it’s used across many other open-source PHP web applications you can think of, and in the general wider software ecosystem, Bcrypt is the standard for securely storing passwords in a database. That’s what we’ve switched to in 6.8.

I guess there are two main advantages to this change. One is that it increases the computational cost of trying to crack a password if the hashed password was exposed in a data breach or if someone got access to your database. A fundamental aspect of securely storing a password is ensuring that nobody can reverse-engineer it to gain access to your password and then either gain access to your site or, in cases where users reuse passwords between services—which, of course, they certainly shouldn’t be doing—allow that breach to lead to further exploits.

Another advantage of the change is that it future-proofs the whole password storage mechanism in WordPress. So in the future, WordPress can upgrade to more secure algorithms as they come along. As computing power increases, some of the ways that algorithms are used can be tweaked so that they are slower and more secure. This change has really enabled all of that future-proofing, so we won’t have to go through a long process of changing the storage mechanism again in the future.

Tom:
So it’s a more foundational set of improvements than just simply switching it out.

John:
That’s right. Yeah. It’s bringing us back up to the state of the art in how passwords are handled in PHP applications.

Tom:
It seemed to me also like there was quite a lot of work done to make sure that the switchover happened pretty transparently to users, but also to site administrators or IT admins who may have a large user base and would be concerned about having to do password resets for their users.

John:
Exactly. I mean, I said it was a headline feature, but ironically, it’s a feature that will be completely invisible to end users. Users won’t need to change their passwords, they won’t get logged out when they upgrade to 6.8, and the upgrade to the storage mechanism will happen invisibly. The next time you sign in using your password, the old storage mechanism will be replaced with Bcrypt, and it will just work. So yeah, kind of ironic really, that it’s been treated as a large change, but it’ll be completely invisible to end users. That was one of the goals—users don’t have to do anything.

Brad:
This was a ticket that was first introduced 13 years ago, back around version 3.4, so you weren’t far off, Tom. When you said 3.8, there we go. It’s pretty neat when you see a ticket that old, with that many people involved over the years, go quiet, then get picked up again. To actually see it cross the finish line must be a great feeling, especially for you and the security team who contributed to getting this into core.

John:
Definitely. Yeah. And of course, some people will question why it took so long to get the change in. The answer to that is really that it had lots of dependencies. Back when the ticket was originally opened, there was nowhere near enough widespread support in PHP to make the change. WordPress still supported older versions of PHP, so we couldn’t introduce the change. As time went on, newer versions of PHP were introduced, and the older ones disappeared, allowing us to make the change. The change itself was fairly comprehensive—it wasn’t straightforward. But yeah, definitely pleased to get that one closed. A 13–14-year-old ticket.

Brad:
Yeah. So let’s talk about your approach to security around open source. Open source security has some extra layers, right? Since everybody has access to the code, bad actors can be looking for vulnerabilities or ways to inject malicious code. You’re really up against the world. What’s your approach from an open-source mentality for tackling that?

John:
Yeah, so there are two parts to it. One part is the private work that happens behind the scenes—handling security vulnerability reports through the HackerOne bug bounty program. That’s all done in private because we’re dealing with potential exploits. The team handling that consists of volunteers as well as people like me, who are sponsored by their employers.

Then, there’s the public side, which is what you mentioned, Brad. One big advantage of open source is that it gets reviewed by many people, including security experts. Hardening work that isn’t related to vulnerabilities can be done in the open. For instance, I believe WordPress 6.8 has around 10 additional tickets that improve security but don’t necessarily fix vulnerabilities. That work was done publicly, as was the Bcrypt change, which had contributions from cryptography experts.

One cool thing about this change was that industry bodies like the Open Worldwide Application Security Project (OWASP) updated their documentation around password storage as a result of WordPress making this change. That had a wider effect on documentation for anyone implementing web application security.

Karim:
That’s really cool. So speaking of working in the open, there’s an upcoming hackathon at CloudFest, and you’re working on one of the projects there. Tell us about it.

John:
Yeah, that’s right. Some of your listeners might have heard me talk about this on Bob’s podcast last week, where we did a good intro into the hackathon at CloudFest happening later this month. I’m co-leading a project on securing the supply chain for open-source software with Nestor from Patchstack.

John:
Yeah, we are going to be working on a few things related to the supply chain, not only for WordPress but for open-source software in general. That means, for example, looking at software published via GitHub and ensuring that workflows and tools used to publish those packages are secure. These workflows operate in a very privileged manner—if someone were to hack into them, they could alter how software is built and introduce vulnerabilities. There’s a whole suite of tooling available for securing the supply chain, but we haven’t necessarily made full use of it in the WordPress ecosystem, or it hasn’t been well integrated.

So, there are some cool things we could do, such as improving WP-CLI to verify packages before installation. If someone installs a plugin, maybe we could verify its supply chain integrity. We’ve got a bunch of ideas, but we only have two days, so we’ll need to be focused. I’m looking forward to seeing what we can accomplish.

Tom:
How many people typically get involved in these hackathons? Do you have a sense of the turnout?

John:
This will be my first time at the CloudFest Hackathon, so I’m not entirely sure. I get the impression that there are a handful of teams, each with a small group of people, but inevitably, we’ll all check out what the other teams are working on and maybe even contribute. It seems like a fairly informal and collaborative event. I believe it’s happening the weekend after next, in March, so it’s coming up quickly.

Tom:
Yeah, it’s getting close. A bit of a follow-up question, maybe touching on what we discussed earlier—security five or ten years ago was often seen as a weakness of WordPress or open source in general. People would say, “Oh, anyone can see the code, so they’re going to find vulnerabilities.” But I feel like that perspective has completely flipped. One of my favorite stats is from Red Hat’s State of Enterprise Open Source report, which says that around 90% of IT leaders now believe open source is more secure than proprietary software.

That shift seems to align with what you said earlier—the fact that more eyes are on open-source code actually leads to more security contributions and fundamentally more secure code. Have you seen security priorities change in WordPress over the last decade?

John:
Yeah, it’s interesting. When people say, “Anyone can look at the code, so they can find vulnerabilities,” you can only interpret that as a good thing, right? Because the alternative is that the vulnerability still exists, but nobody knows about it because the code isn’t public. That doesn’t mean the vulnerability isn’t there.

Of course, some people argue, “Well, if you can’t see the code, you can’t exploit it.” But that’s not necessarily true. There are plenty of ways to find vulnerabilities, even without direct access to the source code. For instance, in programs like the Patchstack bug bounty program, a lot of the security issues discovered aren’t found by reading the code—they’re found through penetration testing and other security assessments. The fact that the code is open source just makes it easier to verify and fix those vulnerabilities.

So yes, I think the mindset has changed over the last decade, especially as open source has continued to grow in both enterprise and mainstream software. People are realizing that just because something is hidden doesn’t mean it’s secure.

Tom:
And actually, it’s arguably more risky if you’re using proprietary software. Like you said, just because the vulnerabilities aren’t visible doesn’t mean they don’t exist.

Brad:
The Schrödinger’s cat situation for code—if a vulnerability exists, but you can’t see it, does it actually exist?

John:
Exactly. There are lots of arguments you can make in favor of open-source security. Some people might say, “Well, proprietary software is developed by expert teams at large multinational companies, so it must be secure.” But that’s also true for major open-source projects. WordPress, for example, has contributors from some of the biggest tech companies on the planet, like Google. They dedicate a lot of time and resources to WordPress, and the experience of the developers working on it is no different from those in proprietary software companies.

The same applies to other large open-source projects. They’re backed by major organizations with highly experienced contributors.

Tom:
And we’ve also seen a rise in bug bounty programs, where even big proprietary platforms are now crowdsourcing vulnerability detection.

John:
Yeah, exactly. Many of these programs are extremely successful. If you look at platforms like HackerOne, you can see the bounty amounts paid out—they run into the hundreds of thousands or even millions of dollars for large organizations. And these aren’t necessarily open-source projects; they’re just recognizing that vulnerabilities will be found regardless of whether the code is public or not.

Brad:
Yeah, thinking back to what Tom mentioned about how the security landscape has shifted over the years—it’s changed so much. Those of us who have been in WordPress for a long time, especially on the sales side, used to get a lot of security-related questions. Ten years ago, one of the biggest concerns was, “I’ve heard WordPress is insecure. I’ve heard it gets hacked a lot.” But now, that’s barely a concern anymore. The conversation has shifted because security has improved so much, thanks to people like you, John, and the entire security team.

It’s made our jobs easier—especially in enterprise sales. When big organizations see that NASA, Microsoft, and Google are using WordPress, they understand that these companies wouldn’t be using insecure software. So, a huge thank you to you and everyone who has contributed to improving WordPress security over the years.

John:
Thanks, Brad. Yeah, there have been so many contributors over the years working on security. That Bcrypt ticket alone had at least 30–40 contributors involved. It’s definitely a team effort.

Karim:
For folks who are going to be at CloudFest, how can they find you?

John:
So, I think it’s an early start on Saturday morning. If you have a CloudFest ticket, you’ll need a separate ticket for the hackathon—that’s not included with the main event. I’m not sure if there are still tickets available, but you can check the CloudFest website.

We’ll start with an introduction to the teams, and then we’ll break off and start hacking, collaborating, and working on our projects. From what I understand, people will be able to move between teams, but ideally, we want folks to focus on the projects they sign up for.

Karim:
And if they’re not at the hackathon but want to connect with you at CloudFest or afterward, how can they reach you?

John:
That’s right—on Monday, there’s a dedicated WP Day at CloudFest, and I’ll be around. As for socials, I’m mainly on LinkedIn these days. If you search for my name, John Blackbourn, you’ll find me there. Feel free to connect, and we can meet up if needed. Otherwise, I’ll be around on Monday after the hackathon.

Tom:
Alright, thanks very much, John. Great talking to you today.

John:
Cool. Thanks for having me.