Baruch Sadogursky (Chief Sticker Officer at JFrog) joins Natalie & Johnny to lament the current state of dependency management in Go and other languages. They discuss the problems dependency managers face, possible technical mitigations like SBOMs, people problems that will never be solved by tech, and take questions from listeners in the #gotimefm channel of Gophers Slack.
Baruch Sadogursky: There were and still are a lot of efforts in trying to do that… And it looks like - this is from our experience as well; we tried to come up with – again, back a couple of years ago when Go Modules only appeared, we heard this vision of “Hey, we will have some kind of a network of central module registries that will communicate with each other, trying to establish whatever truth really is, and make sure that the rest of them are not allowed”, and everything. This is where we actually started Go center. That was one of those registries. But with time, in a matter of the next years, somehow this vision morphed to “Hey, Google will maintain this central module repository, and whatever Google says is the right thing, is the right thing.” So this is where the usability of Go Modules kind of disappeared, because there was no more alternatives to whatever golden modules Google will give you.
But then it was a very useful experience for us to really try and evolve this idea of negotiating what’s [unintelligible 00:17:34.25] And I would say there are now two most important aspects of this system. The first is how do we know which module is the right module, in terms of “Hey, I have two of them, they’re named the same, they have the same version, but their checksum is different. Which one of them is lying, and which one of them is authentic?” That’s the first one. And the second is chain of custody of those modules that we decided they are the right ones. So I decided this module is authentic, and we can talk in a second why… And then, okay, how do I make sure that what I have now in my machine is really the same module, and the papertrail of the decision that it was the authentic module is actually preserved and cannot be hacked?
[18:30] That brought us to envisioning a system that everybody will be able to use in order to guarantee those two, and it’s called Pyrsia. Actually, Pyrsia - that’s an interesting story of the name… It’s actually a system that ancient Greeks used to convey messages with flames between each other. So this is why you have the torch.
The idea is a decentralized P2P network that will provide a consensus of what modules are authentic, and then make sure that they cannot be tampered with, with some kind of blockchain-backed ledger. I know that we lost two thirds of the audience right now, after I mentioned blockchain, but whoever stayed - you did the right thing; it has nothing to do with cryptocurrencies or Web 3.0. It’s just a ledger that you cannot tamper with; that’s the interesting part of it over there.
So basically, what we say is “Okay, everybody now can build modules from source, and publish them wherever they like.” The system will communicate and decide if that’s the golden build by comparing it to the build that the original producer of this module creates. Let’s say I have a library now that I wanna publish, and I create a build that because it’s my library, I know is the right build. Now, everybody can have their own CI producing the same module from my sources, because my sources are obviously open source; as long as the build produces the same result, your end result is as good as mine.
Now, we record it, so no one can hack into that and pretend that another result is the right one, and then we distribute it through a P2P network, because now we don’t care where it comes from, as long as we can guarantee that it’s the right results. So now Natalie builds it on her machine, and she’s behind a firewall and the connection to whatever central repository [unintelligible 00:20:36.12] but you see the next door, in the same intranet with her, in the same company, and as long as we verify that the build that Natalie produced is as good as this golden one that was produced by the original maintainer of the library, you can use whatever she built without downloading it again, outside of the firewall, and getting tons of approvals and whatnot.
So this looks like, from our perspective, the solution to some of the supply chain issues, because it guarantees the authenticity, and it protects the supply chain itself, because we verify on your machine that the package you have is exactly the package you need to have, regardless of where it actually came from. And you also have this network of the Go registry, npm registry, JFrog Artifactory, Maven Central, you name it, the sources that provide infrastructure for massive distribution and scale… And again, we don’t need to trust them in terms of “Are they giving us the right packages?” because we can verify ourselves. And also, in case they are down, we can also rely on the P2P network to get the modules in case they are not available.