Johann Gyger: [58:14] Yeah, so besides supply chain tools, I already mentioned Sigstore, which is really amazing work. It still has some rough edges, I have to say, but it’s getting there. And I think Cilium is doing a pretty good job as well. They are starting to solve the service mesh problem from a different perspective, which might be another game changer… Because services meshes - they have been around; there are quite some solutions, like Linkerd, and Istio, and Consul from HashiCorp, but they’re not being widely adopted, in my opinion. I might be wrong, but I think they add another layer of complexity onto Kubernetes, for instance.

Cilium solves this at the kernel level. They’re replacing the IP table stuff that’s going on with eBPF, which is a lot more flexible, and they have a lot of – how do I put it…? New possibilities are opening up to solve problems.

Gerhard Lazu: Yeah. HashiCorp Vault, with integration to 1Password Connect. Yes.

Gerhard Lazu: That’s something which I don’t know, to be honest. I’ll have some people from HashiCorp in a future episode to talk about this, like "How should we use Vault? What are the options?" Because that’s exactly what I don’t know. I know it’s time to move from our current setup to something better… I mean, we’ve been talking for a while to improve it. It was never high enough on the list, but I think the time is right for us to have a better source of truth, and I think I would prefer it to be managed. Again, this goes back to the conversation which I had with Kelsey - the database should be managed, everything documented… And I think secrets - we should have a service for that, for sure.

Sid Sijbrandij: HashiCorp, Confluent, Elasticsearch… And we’re following in the footsteps of even a pure open source company in the name of Red Hat. So I do think that if you make something that’s used by developers, by IT people, open core is becoming a more and more important way to do that, to the point where it’s getting harder and harder for 100% proprietary companies to even get significant traction for their offering.

Gerhard Lazu: HashiCorp configuration language, ACL.

Andrea Luzzardi: Yeah. But now it’s kind of niche to do that, whereas before that was standard. By default, you would clone AMIs, manage AMIs… There were actually a few cool tools from HashiCorp, like Packer, to actually create your AMIs in more programmatically. But I think given the information we had, that’s the best we could do. So I’d probably do exactly the same thing.

Nabeel Sulieman: [24:16] If I was going to do this too in KCert itself, what I would have to do is– I intentionally named KCert ‘KCert’, with the K, because it’s Kubernetes only, right? It doesn’t actually deal with anything else. But theoretically, we could abstract the storage interface that KCert uses. And it doesn’t have to use Kubernetes primitives. It could instead fetch your certificates and save them to a vault, or whatever. It would actually not be that hard of a change to do it. But I think what ends up becoming tricky is, “Okay, well, where should I store it? Should I store it to Azure Key Vault? Should I support AWS’s version of that? Should I support HashiCorp‘s K-Vault?” There’s just so many options there, and it gets harder to do.

Kelsey Hightower: Azure, AWS, Google Cloud. You go there. API documentation and GCloud command line tool, and Terraform support. So they have all the layers. They say “Listen, we don’t know if you’re gonna write your own tool or not.” Maybe you work at HashiCorp and you need to see all the API calls and all the return responses, and the rules about authentication. There’s all the docs. Now, maybe you are a developer. So - okay, we support Golang, we support gRPC, we support all of these things. So here’s the SDK to make that even easier for you to interact with the API. And then maybe you don’t care about any of that, and now you could just use Terraform, and we have a module that lets you represent those things as just something native, in Google Cloud in particular, whereas Amazon has CloudFormation. We even have tools that just let you directly declare what you want, and then we will hold the promises on the config. We tend to have all of these things because we have no idea what your entry point will be, or if you need custom tooling.

Tom Pansino: It’s a lot of pieces, but none of it is particularly proprietary. So we just need to clear that all with our legal team, and stuff. But we’re using Terraform, which is of course HashiCorps’s infrastructure as code language, so the idea being that you want to have the state of your infrastructure captured just as well as your development code is. And our goal is to make it so that basically everything that we push in this layer of this account structure, this platform layer is delivered via Terraform.

So we’ve got our repo inside of our repository, we have a configuration file, which has an array with objects that represent each of our accounts, or all of our domains. It’s really the domains, it’s not the accounts. So it lists out the name of each domain and the types of environments we want to create for each, so a development or a dev, stage or a prod, all of those things. And then when you run it, the Terraform is structured with various modules. If you’ve worked with Terraform, this is going to sound really familiar, but various modules and things, that one of them I think is– there’s an account module, basically, for Amazon. And so it’s going to provision a new Amazon account, it’s to go and do things like set the support plan to be the correct tier of support that we ask all of our production accounts to have, which is I think the four-hour response time or whatever it is, or the one hour. I forget exactly what we use. And so then in addition, it’s also going to New Relic and it’s setting up accounts in New Relic and provisioning API keys there.

We do a lot of work with– and many people who have worked at larger companies are probably familiar with a concept of like a service account user, or a machine user. We call them faceless users. It’s a user who has no face, is how I was taught; sitting in a chair. And so the point is to say, “I want to connect my GitHub actions back to Amazon to let it do deployments.” So we provision a user and a password for that, and we put that password in GitHub, and in a secrets manager, so that people can start using it in their CI flows. So yeah, it basically runs through a GitHub Action, CI/CD, and does a deploy, and then all of these resources, several thousand of them, spring into existence and start doing what the teams need them to do.

Shawn Wang: Oh, yeah. The origin actually goes back to AWS, actually… So our CEO was the tech lead for what became AWS SQS. This is old school. [laughs] This is like 15 years ago. And then a tech lead for AWS Simple Workflow Service, which is kind of like the predecessor to AWS Step Functions, which most people know today. And then CTO left to eventually wind up at Uber, where they created an open source version of this for Uber’s needs… So it runs everything from like a lot of Uber Eats, to Uber’s driver onboarding.

[36:13] Imagine when you onboard a driver you have to go check things like their criminal background, their driving record, whatever. 17 things need to be true before you say “Alright, I’ll proceed to the next step.” Imagine if you could write all of that in a single function and say “Alright, just block until all these things are true. And then continue – just let me know when it’s freed up.” It’s a very nice programming model for that.

So they open sourced it at Uber four years ago, and then it got a lot of traction at places like Airbnb and Stripe and Netflix, all of whom have been public – even HashiCorp as well… And Mitchell Hashimoto is one of our advisors. One of the fun things that he said, which I really loved, was that if Temporal didn’t exist, then he would have had to write it. So I think we kind of scooped one from HashiCorp, which is pretty fun.

Dan Mangum: No, I believe that you’ll have a dependency there on HashiCorp Terraform plugin SDK, and then you’ll have a replace statement at the bottom of the go mod, that indicates you want to replace that dependency that’s in your require with the fork there that Hassan has.

Zac Smith: Yeah, well - we started in 2014, July. We started relatively small - a couple people, me, Jacob, Aaron, a guy named Aaron Welch and a few original technical founders… And we started with a small seed round that we put in with some friends and family, and said “We’re gonna figure out if we can get to the end of the year.” Because infrastructure is an expensive business; not only do you have to fund doing the work and the software, you have to, like, buy stuff. So we had to, like, buy stuff.

So we started that, and I think by when we got our first product on the market - and I remember, because it was like January of 2015 when we finally had something that you could consume… So we really compressed, a lean startup, six-month kind of thing. Ramen noodles, late nights etc. etc. And we finally got it out, and we were working off of fumes by that point, and just trying to see if we could prove that we could do this.

And we finally got it out - I remember one of our first customers, a guy named Mitchell shows up on our website, and signs up for HashiCorp. He was like, “This is cool, I can deploy automatic bare metal with an API.” I’m like, “That’s exactly what we’re doing.”

Jerome Hardaway: Yeah. David, and how they work together, and built this; and I have another junior dev with them, who through the process of this he ended up getting full-time at his job. He was on a contract, an internship, came through, did this… I made him – I was like, “You are the engineering manager. You’re not allowed to touch a single line of code, unless you’re pairing with somebody. And you report to me. I’m the client.” Because you know, I wanna put the fear of God in people. That’s what I do. I feel like that’s what 80% of my job is, is scaring the s***t out of people. Like, “Okay, Jerome’s serious as f***.”

So we’re gonna be talking about that, and we’re gonna show the things that are like – I’m gonna show you our entire roadmap that I have, that I’ve built in secrecy from people. And it’s crazy, because I saw how GitHub was building their stuff, and I was like “So we can be this crazy? I’m gonna be as crazy. This is what I’m gonna be.”

[01:51:45.04] And just like – I’m gonna bring the award up, because it’s here, and I’m super-excited to go and look at it right now… And then we’re just gonna talk – oh, there’s just two new board members… One of them is a super – like, people absolutely love Taylor Desseyn. He’s the hottest recruiter on the market right now. He’s coming onto the board of VWC. People beg to be in his Rolodex when it comes to jobs. The dude is at VACO, and he’s on fire.

And Melanie Sumner. Melanie is – she’s one of those rare frontend talents. She’s at HashiCorp now, building their design systems, but she was at LinkedIn… And she’s one of those talents that when it comes to accessibility – she’s an accessibility expert that can actually write code. And you would be amazed how rare that is in this market, having an accessibility expert who can write scalable, performant framework code. There’s Melanie and there’s Marcy, and that’s pretty much all I know who are true accessibility experts with coding jobs. Marcy Sutton and Melanie Sumner. Melanie was a Navy veteran, and Taylor - he’s just a dude who’s been around veterans all his life, active duty military, and he’s like “I cannot wait to give back.” And he’s helping us with our processes, with 1.5x-ing our juniors/entry-level devs, which is what we wanna do.

So that’s just in a nutshell what we’re gonna have. We’re gonna bring some other troops to talk…

David Flanagan: Yeah, Brian Ketelsen and I are the creators and maintainers of CUE Blocks. We’re both huge fans of CUE. We think it’s just a great language for defining schema, applying constraints, and even doing some basic comprehensions and mathematics with them.

So it’s not a Turing-complete programming language, but they are starting to add more query APIs and other things to bring it in line with some of that.

So I really like Dagger… I have done an episode with Solomon on Rawkode Live, where we dug into Dagger and we did some deployments. I think it’s a really good tool, and I love seeing CUE used in this way. It’s very similar to TerraForm, in the regards of that you have to have something that understands the abstract form. The HCL, the Yaml, or even the CUE is just compiling down to Yaml at the end of the day anyway.

You’re still constrained in that you can’t do a lot of conditional logic. Loop logic does exist in CUE, and you can do some things like that… But then modifying things within the loop gets a bit difficult, because you’ve only got access to that array count. So it depends on your use case. But I think Dagger is great, and that they’re moving beyond, into like where Boundary is as well. I’m not sure if you’re familiar with HashiCorp’s Boundary…

Carolyn Van Slyck: Yeah, it tells you upfront everything you’re going to need. So you won’t be surprised 30 minutes in that you need a GitHub token, and then you’re like “Oh, shoot, I don’t have it.” It gives you all that info of what you need upfront… And it also gives teams a way to store that securely. So you could have a team, like HashiCorp Vault, or pick a cloud, pick a vault, it doesn’t matter - you’ve got a place where you can put your secrets and share it as a team, and then you could associate it and say that “In the dev environment we all use this set of credentials when we install things.” So you don’t have to run around and try to find all of them yourself.

[24:01] Some things you don’t reuse between people… For example, if I actually had AWS creds, I had Carolyn [unintelligible 00:24:08.01] Johnny, you wouldn’t be using my AWS creds; you’d be prompted to find your own from somewhere, like look deep inside yourself and find some creds… [laughs] But otherwise you can share a whole bunch of other information with each other and reuse it across team members, so it’s easier for one person to step in for another without having to, again, have all this operational knowledge - I think that’s what it comes down to - and reducing how much you need to know about how the sausage of your software was made.

Aaron Schlesinger: Throw Node in there… [laughter] Throw everything, I guess. Throw everything in there. But what language can you pick up that doesn’t need a VM, that has high-level concurrencies/primitives built in the language, that can do networking in a couple lines of code, that’s got really good support for built-in high-level networking protocols… You know there’s not a lot of other stuff out there. And if you’re three engineers - or however many engineers it was - at Google and you’re trying to build Kubernetes, or… Shout-out to Nomad, because I saw some Nomad in the chat there… If you’re HashiCorp and you’re trying to build a distributed system abstraction layer, you kind of need those things, and you’re gonna get up and running with something like that faster with Go than probably with C++, or Java… Not sure about Node; maybe Node will get you there just as fast… Although you don’t have access to multi-core on Node, so there’s a trade-off there as well.

[56:08] So I think Go really caught on because it has most of this stuff right out of the box that you need… And it was one of the only languages that had it at the time when a lot of these things were coming up.

Jacquie Grindrod: Yeah, so I started learning it – this was actually my third attempt learning it. I’ve tried to learn it at previous jobs a few other times, and it just really didn’t feel like there was enough support or time to dive into it. This time I started learning it on a live stream. Straight from installing Go, I started live streaming, I went to the document page, and was like “I don’t even know how to run this, but here we go. Let’s get started.” And that was in preparation for my interviews at HashiCorp, because I learned that all of their products are basically in Go.

So I did all that livestreaming, and I was solving Exercism challenges on live stream, which was a bit more of a dive-in than I thought it was going to be initially, because you went straight from Hello World to “I got [unintelligible 00:04:03.08]” which had a lot of things I just wasn’t ready for, like the custom stuff… I was gonna have it open on my other screen to reference that, but…

[04:13] It was a pretty big dive, and it felt like pair-programming with basically the internet, with whoever wanted to show up, which was pretty terrifying, but it was really nice, because it ended up being a lot of people that already know Go, and who are really friendly… And they end up being really good teachers, because they kind of come by and try to guide you the right way, instead of telling you the answer, which wasn’t what I expected from that experience… So it was really nice.

David Yakobovitch: Right. If I look at an ML engineer as someone who has software experience in building applications, and a data engineer as someone who could already work with cloud systems or distributed systems… And often the bootcamps and the masters programs just don’t give enough there. That’s why you wanna look at capstones for that.

But I think the challenge is there’s so much information to cover and pack into it that data science has just become this term that encompasses the industry. And how I look at it is, simply put, what used to be big data became predictive analytics, became data science, which is now the AI industry. It’s constantly evolving… But the truth is when you look at data science roles, 60% to 80% of the work is still in the data. It’s cleaning data, it’s labeling data, it’s getting it all set up…

I featured actually just at the beginning of August on the HumAIn Podcast Mark Sears, who runs CloudFactory, which is one of the big data labeling companies between and Africa. They have 10,000 just labeling data. I think the reality that a lot of data scientists don’t know until they join the company is you’re not playing with algorithms all day. Maybe you might, but even an ML engineer - you’re gonna be working with APIs, and setting up pipelines and systems before you get to start training and testing and working with other teams to see those results.

So I definitely see a specialization occurring in the field. In fact, I’m calling now a new subfield emerging in data science, which we’re starting to see in some trend reports, called Data Science as a Service. Similar to how we saw Infrastructure as a Service, with things from like HashiCorp, Ansible and Terraform, and a lot of deployment options for the cloud, we’re gonna start seeing that (and we already are) in Data Science as a Service. We’re seeing companies like Neptune and Spell, and [unintelligible 00:19:00.20] and other ones, which have all just recently raised their series A’s, that they’re helping deploy systems.

You even see the founders of Anaconda - a couple of them branched off and launched Saturn Cloud, which is launch these systems in Dockers containers, and now do your data science. Paperspace and Brooklyn got notorious for that, and has been doing a phenomenal job partnering with companies like FastAI, and Insight Data Science Fellowship as well.

Kyle Daigle: Yeah, Jess is 1,000% cooler than I am, and so unfortunately I will leave that to her, but hopefully she can come on. Basically, with Actions there’s a couple of moving parts. You mentioned the visual editor - we make it easy for you to go in, and you can drag and drop and basically say “On a particular event” - and this is generally like a webhook event; so when a pull request is opened, when an issue is opened, when something is pushed, when a ref is deleted, all those sorts of things - I would like you to run an action.

In the visual editor it’s easy to see that you can also parallelize actions. So “I would like you to run these two actions.” Then as each action completes, it will go down the line, and so on. So you can create this interesting dependency list where you can parallelize to two separate actions - “I’d like you to run CI over here, and I would like you to check dependencies. Then if both of those pass, I want you to package this and push it up to the registry”, for example. So the visual editor makes that very clear.

[27:48] Under the hood, we use workflow files, and the visual editor ultimately goes down to that. The workflow file is ultimately a subset of HCL, the HashiCorp language - if you’re familiar with any of those products, it’s a very similar syntax; it’s not completely it, but it’s a subset. We chose that in part because we wanted to make a workflow file that was very clear to understand, that didn’t have a lot of cruft; it was human-readable, which was the primary concern… So we looked at all the obvious suspects - YAML, JSON and so on, but ultimately landed on this, because we thought it was honestly aesthetically-pleasing, which we thought would be a very important part of how easy it would be to feel these things out as human beings. So that’s the first piece here.

Now, skipping from the workflow file to an individual action, an individual action is ultimately just a Docker container. That container can do basically anything you can fit inside it. You get a single CPU, with some memory, that will take care of what your action is trying to accomplish.

If you look in the open source examples of what actions can do, usually you see the Docker file saying where we’re actually going to run this thing, what it does, so on and so forth. Any Docker container should work; there’s some limitations around exactly what we let you do with Docker, but we’re working with the community to make sure we’re making it as broad as it’s possible. Each one of those gets a couple of things for free; each action gets a GitHub token for free, and it also gets a checkout of the repository that you are working in for free. In this way we’re trying to make it very simple for you to do things like “Run these tests. Go look at these files in my repo, and then call a GitHub API and create an issue” etc.

The Actions environment is ultimately a VM where we’re running these actions on your behalf, and we’re using that VM boundary primarily for the security concerns that we have. We do some interesting things with Docker to make it a little bit more secure and so on, but ultimately, at the end of the day all of this code is going into a VM for us to run for you.

In between all this we have a couple of services that help us do this. GitHub is generally notoriously a monolithic Rails application, but in this instance we have the launch editor itself, which is the drag-and-drop and workflow file piece as its own service… Ultimately, the tool that actually receives these events and decides whether a workflow needs to be run as its own separate service, and then ultimately the tool that is running the code and sort of orchestrating and scheduling it is a separate service as well.

So that’s kind of the very quick overview of how a workflow file visually, or a workflow file-wise triggers these actions based on events, and then ultimately these actions are all run inside of a VM, which has access to the GitHub token, the repo, and then also some secrets if you wanted to put some outside tokens, and we support all that and keep it encrypted as well.

Adam Stacoviak: Including a world with big numbers. We’ve seen the headlines on Changelog.com this week in the news feed - billion-dollar valuations, hundreds of millions of dollars invested into new companies or companies that are now unicorns, HashiCorp being an open core model type company that’s just taken on a new round of gigantic funding… So there’s clearly lots of money at play here, and it’s a new world for open source every single day.

So where do we go from here? Clearly, we’ve had a great conversation, that’s led from not only Dgraph as a tech and how it applies, graph databases 101, on through how they could be used… You’re clearly super-smart; you’ve had to relicense, you’ve been through a journey… What do you suggest may be the next step? Maybe not here today, because we’re getting out of time, but what are some suggestions for you to continue this conversation in ways that are meaningful, that can get to meaningful change? Do we have a conference about it, do we do a Sustain-like unconference, or just kind of a gathering? How can this best be approached by the right people in ways that are not vicious and attacking, but in ways that are meant to actually get to change? What do you suggest?