Baruch Sadogursky (Chief Sticker Officer at JFrog) joins Natalie & Johnny to lament the current state of dependency management in Go and other languages. They discuss the problems dependency managers face, possible technical mitigations like SBOMs, people problems that will never be solved by tech, and take questions from listeners in the #gotimefm channel of Gophers Slack.
Baruch Sadogursky: There were and still are a lot of efforts in trying to do that⦠And it looks like - this is from our experience as well; we tried to come up with ā again, back a couple of years ago when Go Modules only appeared, we heard this vision of āHey, we will have some kind of a network of central module registries that will communicate with each other, trying to establish whatever truth really is, and make sure that the rest of them are not allowedā, and everything. This is where we actually started Go center. That was one of those registries. But with time, in a matter of the next years, somehow this vision morphed to āHey, Google will maintain this central module repository, and whatever Google says is the right thing, is the right thing.ā So this is where the usability of Go Modules kind of disappeared, because there was no more alternatives to whatever golden modules Google will give you.
But then it was a very useful experience for us to really try and evolve this idea of negotiating whatās [unintelligible 00:17:34.25] And I would say there are now two most important aspects of this system. The first is how do we know which module is the right module, in terms of āHey, I have two of them, theyāre named the same, they have the same version, but their checksum is different. Which one of them is lying, and which one of them is authentic?ā Thatās the first one. And the second is chain of custody of those modules that we decided they are the right ones. So I decided this module is authentic, and we can talk in a second why⦠And then, okay, how do I make sure that what I have now in my machine is really the same module, and the papertrail of the decision that it was the authentic module is actually preserved and cannot be hacked?
[18:30] That brought us to envisioning a system that everybody will be able to use in order to guarantee those two, and itās called Pyrsia. Actually, Pyrsia - thatās an interesting story of the name⦠Itās actually a system that ancient Greeks used to convey messages with flames between each other. So this is why you have the torch.
The idea is a decentralized P2P network that will provide a consensus of what modules are authentic, and then make sure that they cannot be tampered with, with some kind of blockchain-backed ledger. I know that we lost two thirds of the audience right now, after I mentioned blockchain, but whoever stayed - you did the right thing; it has nothing to do with cryptocurrencies or Web 3.0. Itās just a ledger that you cannot tamper with; thatās the interesting part of it over there.
So basically, what we say is āOkay, everybody now can build modules from source, and publish them wherever they like.ā The system will communicate and decide if thatās the golden build by comparing it to the build that the original producer of this module creates. Letās say I have a library now that I wanna publish, and I create a build that because itās my library, I know is the right build. Now, everybody can have their own CI producing the same module from my sources, because my sources are obviously open source; as long as the build produces the same result, your end result is as good as mine.
Now, we record it, so no one can hack into that and pretend that another result is the right one, and then we distribute it through a P2P network, because now we donāt care where it comes from, as long as we can guarantee that itās the right results. So now Natalie builds it on her machine, and sheās behind a firewall and the connection to whatever central repository [unintelligible 00:20:36.12] but you see the next door, in the same intranet with her, in the same company, and as long as we verify that the build that Natalie produced is as good as this golden one that was produced by the original maintainer of the library, you can use whatever she built without downloading it again, outside of the firewall, and getting tons of approvals and whatnot.
So this looks like, from our perspective, the solution to some of the supply chain issues, because it guarantees the authenticity, and it protects the supply chain itself, because we verify on your machine that the package you have is exactly the package you need to have, regardless of where it actually came from. And you also have this network of the Go registry, npm registry, JFrog Artifactory, Maven Central, you name it, the sources that provide infrastructure for massive distribution and scale⦠And again, we donāt need to trust them in terms of āAre they giving us the right packages?ā because we can verify ourselves. And also, in case they are down, we can also rely on the P2P network to get the modules in case they are not available.