In this episode, we will be talking to Russ Cox, who joined the Go team at Google in 2008 and has been the Go project tech lead since 2012, about stepping back & handing over the reins to Austin Clements, who will also join us! We also have Cherry Mui, who is stepping into Austin’s previous role as tech lead of the “Go core”.
Featuring
Sponsors
Fly.io – The home of Changelog.com — Deploy your apps close to your users — global Anycast load-balancing, zero-configuration private networking, hardware isolation, and instant WireGuard VPN connections. Push-button deployments that scale to thousands of instances. Check out the speedrun to get started in minutes.
JetBrains – Sign up for the free “Mastering Go with GoLand” course and receive a complimentary 1-year GoLand subscription at bytesizego.com/goland
Incogni – Go to incogni.com/gotime and use code GOTIME
using our link to get an exclusive 60% off an annual Incogni plan.
Notes & Links
Chapters
Chapter Number | Chapter Start Time | Chapter Title | Chapter Duration |
1 | 00:00 | It's Go Time! | 00:46 |
2 | 00:46 | Sponsor: Fly | 03:31 |
3 | 04:17 | Meeting the guests | 06:29 |
4 | 10:47 | Matcha | 03:20 |
5 | 14:07 | Decision to step down | 04:25 |
6 | 18:32 | What is a tech lead? | 01:46 |
7 | 20:19 | Desired impact | 06:32 |
8 | 26:51 | Sponsor: JetBrains | 02:28 |
9 | 29:19 | Cherry's new role | 08:51 |
10 | 38:10 | What solutions will you bring | 09:00 |
11 | 47:09 | Engaging with the community | 09:27 |
12 | 56:36 | Where to learn more | 03:22 |
13 | 59:58 | Sponsor: Incogni | 03:29 |
14 | 1:03:27 | Unpopular Opinions! | 00:34 |
15 | 1:04:01 | Russ' unpop | 00:51 |
16 | 1:04:52 | Cherry's unpop | 01:14 |
17 | 1:06:06 | Austin's unpop | 01:33 |
18 | 1:07:39 | Angelica's unpop | 00:36 |
19 | 1:08:15 | Outro | 01:06 |
Transcript
Play the audio to listen along while you enjoy the transcript. 🎧
Hello and welcome to Go Time. I’m really, really excited for our episode today. Today we are joined by three wonderful guests, who are gonna be talking a little bit about the changes in leadership that have happened recently on the Go team. So I’m joined by Russ Cox, who joined the Go team all the way in 2008, and has been the Go project tech lead since 2012. And he’s gonna be talking a little bit about kind of stepping down from that tech lead position, and handing over the reins to the wonderful Austin Clements, who’s also with us, so very excited about that… So they’re gonna be talking a little bit about what this transition means for them, how they’re thinking about taking over the Go team, and bringing it into the kind of new chapter of Go.
We are also joined by Cherry, who is stepping into Austin’s previous role, as tech lead of the Go core, so an extremely important part of the Go ecosystem. So really lucky to have her with us today as well. So without further ado, Russ, hello. Welcome to the show again. I’m very excited to have you on. How are you?
Thank you. It’s great to be back. I’m doing pretty well today, and I’m just excited to be here.
And it seems you’re taking to stepping down very well, given that you said that you had a nice little sleep-in today. So aren’t you lucky…?
I’m actually on vacation today, but I made an exception to come in and hang out with you for an hour, so I’m excited to be here.
I’m very, very, very pleased to have you, and very blessed that you spend your time off with Go Time. Next up, we have Austin. Hello, Austin.
Hello.
So you joined the Go team at Google about 10 years ago, so –
Just about, yeah.
…you’ve been around for a while.
Mm-hmm…
And how are you doing today?
I’m great. Thank you.
We’re excited to have you on and kind of dig in a little more. And then finally, but certainly not least, Cherry. How are you?
Good, good. How are you today?
I’m excited to get to know you both. I have not had the pleasure of getting to know Austin or Cherry very deeply, so we’re going to be talking a little bit about you as just human beings, as well as gophers today. So given that I don’t know you all that well, and we want to make sure that all our lovely gophers know you even better now you’re in these critical roles, I’d love to give you both a little bit of time to kind of talk about kind of you, how you got into Go, how you found yourself in the position today… Interesting hobbies and fun facts are welcome, if you would like. So I’ll kind of leave it to you to how you’d like to introduce yourself to the Go Time listeners. Maybe we’ll start with you, Cherry.
I joined the Go team and Google in 2016. And yeah, since then I’ve been mostly working on compiler runtime and toolchain-related things. I’ve been working on other things, but mostly compiler runtime related. And I’m now being the tech lead of the Go core team for about two weeks.
Before that, I was a graduate student at Brown University, studying chemistry, and… How I arrived to Go. When I was at school, a friend of mine, actually - yeah, at the time I liked programming basically as a hobby, and playing with code for fun. I like to play with compilers and operating systems, so I spent some time hacking around the Plan 9 compiler actually, and the Plan 9 operating system. And a friend of mine told me that there was a new programming language called Go, and they have quite a bit of relationship to Plan 9. “You must find that interesting.” Okay, so I tried it out. It was actually very interesting. I was – it’s actually a pretty cool language, and I was pretty much convinced that I like Go. And he was right, and I did like Go. And he also convinced me to actually work in Google and in the Go team… So pretty much that’s what brought me here.
[08:25] Oh, my gosh. So we all need to be very thankful then. This friend was a very good friend, is what I’m hearing.
Yes. Yes, he is.
How about you, Austin?
Hi. Well, to answer what brought me to Go, I actually have to go back almost 20 years ago. So I actually met Russ about 20 years ago, when we worked together in grad school at MIT. Then in 2009, I was frantically scrambling to look for an internship, just because I hadn’t been on top of things… And Russ was already working at Google, on Go, at that point… And was like “Hey, we’re working on something cool. Let’s chat.” This was before Go was even publicly announced. So I met up with Russ, and he sort of described the core language design, and the philosophy, and from that point I was kind of hooked.
Then after that internship, I continued with my PhD, and then eventually, about 10 years ago, I started at Google, and started right on the Go team… And I’ve been there ever since. And yeah, then I became the tech lead of the Go compiler and runtime in 2017, and I’m now handing on the reins.
So I started programming when I was in first grade. I love coding, I love programming languages, I love how different programming languages shape your thinking in different ways… I taught Scheme at MIT for several years, and I just loved seeing sort of the spark of new understanding in students, many of whom had already been programming in C-like languages for years… But this was just like a new way of thinking to them. And Go sort of brings a lot of that familiarity and practicality, but also pushes you into newer and better and more structured ways of thinking. So when Russ introduced me to it, I’m like “This is actually really fantastic.”
Outside of programming, I love matcha, and have been perfecting my matcha latte skills for over a decade now. I actually just returned from a visit to Japan, where I tried to have at least two different forms of matcha each day. I also love smart home stuff. I went a little wild with it at the start of the pandemic, and now I have a complete mess of networks on my hands, that I really need to detangle at some point. And I love mid-century architecture. We live in an MCM house, and just finished a big renovation that we’re very proud of.
That’s awesome. So I’ve got a bone to pick with you. I got matcha powder the other day, and I love matcha lattes, but I live in New York, and lattes are very expensive, and I don’t have the money - I’m going to be candid with you - to be buying myself matcha lattes every day. I don’t have that luxury in life quite yet. I’m aspiring, but I’m not quite there yet. So I got matcha powder from Whole Foods, and I brought it back and I put a little bit of hot water, and whisked it up, put it in my milk… It tasted repulsive. It was not the matcha latte that I was expecting. So I’m gonna have to pick your brains on the ideal matcha latte. And I’m sure other Go Time listeners are also matcha drinkers, who struggle with the same things I do, let alone Go compiler issues. Come on, matcha mixture. More important.
So there are a few really important things. One is that you have to start with a good matcha powder. There is a huge range. If you google around, you will even find websites with like reviews, and tasting notes. It’s like wine. There’s a huge range. Even the really high quality matcha, different brands and sources of it have different flavors. So you also have to find the right matcha for you.
[12:11] How you prepare it is important. I highly recommend passing it through a sieve, because you don’t want clumps in it. So I always sieve my matcha powder first. And then when you whisk it, it’s really important to use the right tool. So you want to use a chasen, which is that bamboo whisk with a whole bunch of tines, because it’s such a powder that it can really easily reform clumps in the water. And when you whisk it, you want to whisk back and forth in like a W-shaped motion, not in a circular motion. And you can generally tell if you’ve got a good matcha powder, because it will be like a Viridian green… And you should get a thin layer of froth across the entire bowl.
And then the final thing that’s important is somehow sweetening it. So straight matcha powder is usually pretty bitter. I’ve had ones that are less bitter, but usually it’s fairly bitter. But if you sweeten it a little bit, then that actually really brings out the subtler flavors, and tamps down the bitter flavor. So you can either add a bit of simple syrup directly to your matcha, or it’s pretty common in Japan, Japanese tea ceremonies to drink the matcha straight, but to have like a red bean paste or something very sweet next to it that you sort of eat at the same time. So those are my tips.
I appreciate it. Not only a technical leader, but a matcha leader, to lead the Go community into better matcha. I’m ready. So what I’m hearing is that I need to make Austin’s amazing matcha, and then none of you are going to be able to see this, but maybe you will if you look at the beautiful picture of Cherry… Cherry has the most gorgeous green hair. So what I’m hearing is that I have to dye my hair green, and drink the beautiful matcha that Austin has supported. It’s a bright future for the Go community. I’m very excited about this. Thank you so much. I really appreciate you kind of indulging me, getting to know you a little bit more…
Russ, I’m going to talk to you a little bit now. I want to understand a little bit about your decision to step down from tech leading Go, and then the subsequent decision to kind of bring Austin in, bring Sherry into Austin’s old role… I’d love to kind of hear a little bit about how that happened, and what got us to here - where it’s been a month or so, maybe a little longer, depending on when this episode is aired - in these new roles that you’ve all taken on.
I mean, Austin won the matcha contest, and that was it.
That was it. [laughter]
More seriously, anytime you’re in a role, you want to be thinking about who’s going to take over that role if you can’t do it anymore. And so we had private discussions inside Google for many years about, for everyone on the team, who would take over if that person had to leave, or just for whatever reason needed to do something else. And so I’ve always thought that Austin should be the next person to lead it. Obviously, we go back very far, but Austin, I think, has a lot of the same sort of sensibilities that we were trying to do for Go, kind of baked into the way they approach computing, and Go, and just sort of technology in general… And so it feels like a very good fit. It always felt like a very good fit to me. I mean, that’s why I recruited Austin to the project to begin with.
And so it was always clear to me that I’d want Austin to take over, eventually. And so then at some point, I realized not only were they the right person to take over, but they’re ready to take over, and it’d be good for the Go project to make that switch. I’ve been the Go tech lead officially for 12 years. For some number of years before that, Rob used to introduce me and him as co-tech leads… I didn’t notice when he started doing that. And then I’ve been working on Go itself for 16 years.
[16:02] So 12 years is a long time to have one leader who’s sort of in charge of setting the tone and the direction of something… And things can get stale any time you have a leader for a long time. And do I want to be doing this in another 12 years? Probably that’s not good for me or the project. And by then, Austin and Cherry will probably have gotten bored and moved on to something else that gives them a broader role.
So it makes sense for me to step back, and let them step forward and take on this bigger role, and sort of grow their strengths. And I can step back and be around, and be here to make sure that anything they need, I can provide, and just sort of be there to support.
I also think it’s good for the Go community. In the early days of Go, we had Rob leading, and there was a tiny bit of a kind of personality cult that built up around Rob as the leader. And then Rob handed things to me, and I think the same thing happened over the… You know, Rob was maybe the leader for like four years, three years, and then I think the same thing happened to me over the last 12 years, that people think there are things that only I can do, or my idea of what the right way is the only way to do something… And I think that’s really unfortunate. That’s not the way that you want a technical project to go.
And it’s not just in Go. You see this - I’m not going to name any names, but you totally see this in other projects, where people have been leading for decades. And in some projects, they’ve been leading for decades and you look at what they’re doing now and you think “You’re kind of missing the boat on some newer things.” And so I don’t want that to happen to Go, I don’t want that to happen to me. I think that it makes sense to have new perspectives and new leaders from time to time.
And then the other thing that’s important to remember is that being the tech lead is really kind of like a service role. It really is that you’re serving a large group of other people, and it’s not just an honorary title and then you sort of go about your day. So it’s a lot of work. And I think that it makes sense for different people to take different approaches to that role, and sort of have a new leader in that role to serve the community and the team differently, and see how that goes.
So you talked a little bit about kind of different leaders bringing different things, a kind of notable shift in the community, and perhaps ways of thinking about Go, both technically and from that community people perspective… Austin, I’d love to talk to you a little bit about - first and foremost, for those who don’t really understand, what is a tech lead for the Go team? What does that even mean as a role? I would love to hear a little bit about that, but also a little bit about how you personally see the role of tech-leading the Go language.
Yeah, absolutely. So I really love Russ’es description as a tech leader as a service role. I believe that tech leads are sort of the glue that hold the project together. I believe deeply that great work comes from many different perspectives, and it’s a tech lead’s job to facilitate all those perspectives, and bring them together, and put them into a coherent whole. If you were to just take every idea that came, you would wind up with complete nonsense. But if you only take your own ideas, then you’re also not doing a good service.
So as a project leader, as a tech leader, that gives me a very broad perspective, and I consider it an important responsibility to personally focus on problems that require very cross-cutting solutions, because I have that particular perspective. But I don’t necessarily have the deep perspective into every area that the rest of the team does, and so it’s important for others to focus on problems that require really deep solutions in their areas of perspective.
And both are very important, and I consider it sort of a big part of my role to bring those together, and to make sure that those perspectives are heard, and seen, and taken advantage of, and again, put together into a coherent whole.
[20:18] And as you’re thinking about the impact you would like to have, both technically, but also on kind of the Go community, what are some kind of top of mind things…? I know this is a very new role that you’re stepping into, but just initial thoughts around like changes you’d like to see, the way you’re kind of thinking about your - almost your imprint on the Go community, and approaching, stepping in and making it your own. I apologize, Russ; this is no longer Russ’es role. This is yours, and it’s really exciting, and I think it’s an opportunity for a positive – like, leaving a wonderful chapter, moving on to a new chapter of Go leadership, as kind of, Russ, you characterize as these chapters of leadership as kind of you stepped into the role, and now Austin etc. I’d love to hear a little bit about that, your goals, how you’re thinking about it; your hopes, I guess.
Yeah, absolutely. So I do want to first say that stability is a very important part of Go. So I have no plans to come in and completely shake up the house. [laughter] So yeah, in terms of what’s sort of top of mind for me and what I’m thinking about… We like to say that Go is about engineering at scale, and engineering for scale. One of the things that was really top of mind for me coming into this role is how we can scale the engineering of Go itself without losing its minimalism and approachability. There are both technical and people aspects to this. For example on the more technical side, we’re starting to see dividends from the new diagnostics strategy that [unintelligible 00:21:58.09] has been driving. There we basically said “There’s no way that we on the Go team at Google have the resources to build all of the runtime diagnostics tools that people want and need. So how can we create a platform on which people can build their own tools?” That’s a very technical approach to sort of scaling the engineering of Go itself.
And then there are a lot of people processes. I’m thinking a lot about how we better communicate with and leverage the amazing developer community that we have. How can we improve our transparency? How can we better accept contributions? How can we keep people around better? How can we create long-term stability in the Go project, while scaling up the engineering of Go itself? Another thing that’s really top of mind for me right now is on sort of the engineering at scale perspective.
One of the sort of founding principles of Go was that programming should be fun. And while I do strongly believe that programming in Go is still fun, I also think that one of the costs of Go stability has been that there have been – there’s been a lot of experimentation in a lot of other languages. And there are places where I think other languages have been finding places that they can do better, that they become more productive, they become more fun. And I think that Go has never been about cargo-culting from other languages, but I think we can learn a lot from sort of the movement that other languages have had in recent years.
Go sort of started the development of more systems programming languages, of sort of more exploration in what can a programming language be… And I think it’s fantastic that we have both sort of Go on the more stable end of the spectrum, and also other languages that are willing to just like go wild and explore things. And I think we can learn from some of the best ideas in there, and bring them back into Go, and sort of keep things fresh… Because the bar for what makes programming fun moves. It’s not in one place. The bar for what makes programming productive moves. And I think we need to keep up with that, and we need to be learning and exploring.
[24:19] And then on engineering for scale - I’ve been thinking about how to engineer systems that scale for a long time. My PhD thesis was on the formal relationship between software API design interfaces and multi-core scalability. And I’ve been thinking a lot about Go’s approach to performance recently. We spent many years basically trying to float all boats, and we’re kind of running out of opportunities on that front. Obviously, we’re going to keep trying to do that, but it gets harder and harder over time.
So I think we’re sort of moving into a new phase with performance and scalability of Go, where we’re going to have to give people more mechanisms to do explicit performance engineering. But that said, I also deeply believe that performance engineering in Go needs to be incremental and composable. So engineers can trade higher engineering cost for lower resource cost just where it matters, without having to pay the high engineering cost everywhere… And so that they don’t have to be constantly thinking about the effects of performance optimizations as a system evolves.
Some of our experiments with memory allocation are actually a great example of this. A few years ago we put out this experimental support for explicit arena-based allocation. We never moved forward with that, because it really wasn’t composable. You had to be pretty careful to keep track of what was allocated how, and the details of that tended to leak out of APIs and libraries that used this. So that was kind of incremental, but it really wasn’t composable. For example, we’re now experimenting with a new variation of arena allocation that is composable, that works on these principles, so that an engineer doing performance optimization can drop in a few annotations in the code, and then not really worry about it again for a while.
And it’s possible to do it wrong, otherwise we could just do it automatically… But if you do it wrong or your system changes over time, it degrades gracefully, which also means that you can revisit those annotations as your software changes, but you only have to do it from time to time, and hopefully, the tooling can tell you “You probably need to look at this again and figure out what’s going on here.”
So those are sort of – there are a lot of things at the top of my mind, but those are a few of the things that I’ve been thinking a lot about.
Break: [26:39]
Now turning to you, Cherry, because you also have an extremely exciting new role. So similarly, I want to start with the basics of like - when we say, okay, you’re stepping into this role, you’re tech leading the Go core… What does that mean? What is the Go core, for those who might not be familiar?
Okay, sure. Yeah. In the past we had separate the compiler runtime subteam, and a separate Go release subteam. I think there was some time the release subteam was also known by other names such as open source project team, or the Go OSP team, or other names, as responsibility shifts… But it’s mostly related to Go releases. And I think it was during the pandemic, and partly related to team member change and other reasons, we decided to form a more integrated team called Go core. It basically consists of the compiler runtime tool chain aspects and the Go release aspects. And because the language itself and the compiler tool chain and runtime and Go releases, they all are in the very center of Go, other parts like tools, IDE plugins or platforms, cloud integrations, and all other things are built on top of that… So we call the subteam for the compiler runtime and Go releases the Go core team. That’s where it came from, I think.
And how are you feeling in your new role? I’m going to ask you a very similar question to what I asked Austin. How are you thinking about this new space that you’re overseeing? What are things that may be top of mind that you’re excited about as you’re kind of – a month in, is that right? About a month in?
No, just two weeks. Or three, if you will, because Austin was out for the week before the actual transition.
Okay, so truly top of mind as you just have started. What are you thinking about? What are you excited about?
As you know, it’s a very new role to me, and I’m still learning, and there’s a lot of things for me to learn… So basically, just learning. Even the basic operation in that role is something I need to learn and I’ve been learning. But I’m also excited and feel pretty good about the new role. And as Russ mentioned, I really like that thing, too. It’s like, this is a tech leader service role, so I’d really like to find a way to serve the team and the community in the best way I can. And as you know, the core team, as well as the community, have lots of talented engineers and contributors… And never a lack of ideas. They’re just full of ideas. So I’m here just – I’m very excited to work with all of the engineers and contributors, I’m excited about their ideas, and talk about designs and try to integrate all the nice ideas, fancy ideas, into a coherent part. That’s what I’m here for, I think.
And right now, on top of my mind, we want to make sure the transition is as smooth as possible, and we want to ensure the Go 1.24 release will be great… And that’s the most important thing. I guess the long-term, as we all know, Go is not just a language or a compiler tool. It’s a full platform. So I’d really like to see more integration with other parts of the platform, like tools, security, and even AI, how all these parts [unintelligible 00:33:35.14] and how this evolves.
[33:40] I must say, for two weeks in, I love the vision, and I’m excited to see it as it develops past two weeks. If you have this vision and this outlook after two weeks, let’s check in in a couple months. I’m ready. And then last but certainly not least, I know, Russ, you said that you’re kind of around, you’re going to be supporting this transition… But you also have a new role that I want to hear a little bit about. So I’d love to hear about kind of what you’re stepping into. And I’m going to ask you the exact same question - what you’re excited about, what’s top of mind for you in kind of this next chapter of your time in the Go space?
Sure, sure. So we’ve been trying to figure out what we should do with AI; what sort of new capabilities the recent AI advances in the particular large language models unlock. And one of the things that I think really helps – so there’s a lot of, I think, unwarranted optimism about exactly these amazing things that LLMs will be able to do. And maybe they will, maybe they won’t. But at the moment, I think that the main strength is - I saw an article, maybe a couple of years ago now, that was sort of framing them as kind of a word calculator. So computers are number calculators, but LLMs are really kind of word calculators.
So in the situations where you have a lot of words and you want to deal with them, having a word calculator sounds like a good thing. And so one of those times is when you’re doing software maintenance and you’re dealing with other people, right? They’re speaking English, and so having some sort of word calculator that can do things with the English text sounds like that would be a win.
And so in particular, we’re looking at what we can do to help open source developers with running their own open source projects, and we’re using Go as a test bed for that. And the goal is really to try to help automate away the stuff that no one really wants to do, like the basic triage of issues, or figuring out where the duplicate issues are.
Right now we have a bot that in the Go repo, when you post a new issue, it looks up using LLMs and vector databases other issues that are very closely related, and it posts a list of at most the top six or seven, maybe ten; I forget exactly what the cutoff is, but there’s a score cutoff of how related they have to be. So sometimes it’ll post nothing. And sometimes it’ll say “Hey, these three issues look very related to this issue.” And originally, we were looking at that for automated duplicate detection and just closing a duplicate, but it’s very hard to tell the difference between “This is a duplicate of the issue” or “Yes, this looks like exactly the same thing, but you thought you fixed it, and now it’s happening again, o maybe it’s different.” You can’t tell the difference between those from the reports.
But just even pointing out “Hey, look, these other issues might be related” has been incredibly helpful, because you get an issue that comes in and you’re looking at it and you didn’t even know about this other issue that someone else took care of. And you say “Oh wait, that does look like exactly the same thing.” In fact, the bug fix for that one has a subtle mistake in it, and that would cause this new one. Seeing those connections is really, really helpful. And in particular, having this kind of database of the context for the project, and having the computers take care of saying “Hey, this looks like related context that you should know about”, that turns out to be incredibly helpful. Because once the project is bigger that you can’t keep it in your head - none of us can keep Go in our heads anymore, all the stuff in the Go project. Having that kind of automated retrieval is actually just incredibly helpful.
So we’re looking for how can we use LLMs and sort of recent advances in AI to help with that kind of stuff, the stuff that people aren’t good at, and that honestly, isn’t that much fun. Groveling through all the issues to try to find the related issues is not something I want to spend my time doing. Whereas, you know, we’re not that interested in having the AIs write all the code, because that’s the fun part, right? Like, why would we take away the fun part? Let’s take away the not fun part.
So that’s the basic idea of the project… It’s just “Let’s figure out how we can point AI at the stuff we don’t want to do”, and also, just learn a bit about what AI can do. To me, it’s sort of still a bit of an experiment to just see what are LLMs actually good for, because none of us really know.
[37:59] That’s awesome. It feels like you’re actually under the new Austin’s regime of funness. So I love to hear it. [laughs] So now I want to hear from kind of all of you a little bit about – so we’ve talked a lot about kind of the new roles, we’ve talked about your past, what you’re excited about… I would love to hear from each of you what part of Go, or what thing about Go have you just been itching to solve, and itching to improve, that now you’re in a position to maybe make it happen. So maybe we can start with you, Austin.
Well, I’ll give sort of two answers to this. One is as I’ve been sort of closing out my previous role as Go core tech lead, one of the problems that I have been trying to solve ever since I started on the Go team was garbage collection. It’s just – it takes a lot of resources to do garbage collection. And a long time ago, I did this micro architectural analysis that basically showed - you know, the performance of garbage collection is like down here, and there’s some sort of like theoretical sort of speed of light limit on garbage collection, that’s way the heck up here. And there’s no way that we could actually get garbage collection to be that fast, but surely, surely we can close this gap more than we do right now. And like I said, I did this analysis years and years and years ago, and every once in a while I still bring up the slides that I made from that analysis, and I’m like “We still have this giant gap.”
So I’ve been experimenting with this new garbage collection algorithm that – you know, as tech lead, you draw on all these ideas. And so this draws on ideas from lots of people on the team, and I’ve sort of found a way to glue them together in a way that we haven’t thought about before… And I’m calling this new garbage collection algorithm Green Tea, because I happened to make a lot of progress on it while I was in Japan, going to cafes and drinking matcha every day. See, it all connects.
So I’m still doing lots of experiments, I’m still not completely sure that this new algorithm is a win, but it sure has been a lot of fun to work on… It is very unlike our current garbage collection algorithm. I mean, from a user’s perspective, it still does the same thing, but it’s been a lot of fun.
And if nothing else, one of the technical things that we’ve been thinking about recently has been how to get SIMD - that’s single instruction/multiple data - into Go. And we’ve been thinking about SIMD for years now, and we actually have various internal sketches of proposals and so forth, but we’ve never really made that much progress on it. And I believe that one of the reasons for that is that nobody on the Go core team has really done that much SIMD programming. We just don’t have that much in-house expertise with actually doing SIMD programming… And one of the things about this new GC algorithm is that it’s built around a whole bunch of like really tight computational kernels, unlike the current garbage collection algorithm… And that means that for at least some of those kernels you can really dive into micro optimizing them, and in particular using SIMD. And so this has been a great excuse for me to actually learn and get experience with actually writing SIMD code. And a lot of the ideas I had about like “Oh, we can expose these beautiful APIs that will make SIMD all nice and stuff” have gone completely out the window, because it’s a complete mess once the rubber hits the road.
[41:55] So that’s sort of the thing that I’ve been itching to solve for almost 10 years now, that I’m kind of trying to go out on from being the tech lead of the Go core team. And going into being tech lead of the overall Go team, one of the things that I’m sort of itching to solve is along the lines of what I was saying earlier, about how programming should be fun. I believe that – so you know, several years ago Russ started up what we called at the time the Go 2 process, although as we know it’s still Go 1… But the key part of the Go 2 process that Russ put out there was asking people for experience reports on the big things that really caused friction in real production systems. And I think that process worked extremely well.
So we got these top big issues with the production readiness of Go systems. And we’ve been tackling those over the past several years, and we’re still not quite done with that, but we’ve made really good progress on that. And I think one of the things, however, that that process has missed is the small stuff. Obviously, these big sources of friction and things that cause production problems are very important, but also, engineers are writing lines of code. It’s all made out of lines of code. And there are all these small things on an individual line of code where sometimes you’re like “Why is Go making me do this? This is silly.” And like I said at the very top, I do not want to come into Go and start tearing down the house.
I believe strongly in Go’s stability. But I also believe that there are sort of small-scale things we can do that will make Go more fun, and ultimately more productive, by focusing on sort of the micro as much as we have focused over the past several years on the macro.
It sounds like you’ve got your action items, and I’m very excited. I agree with them fully. I feel like that – especially the second point, around really focusing a little bit on those little, more detailed things that might feel minor, but are just so impactful when it comes to day-to-day programming in like engineers’ lives, sitting and writing their lines of code… Cherry, I want to hear a little bit about what your kind of bee in your bonnet is; the thing that’s been on your mind that you’d wanted to dive into and fix or change, and that now you maybe have the leadership position or you have the bandwidth to be able to address it or put a little bit more thought into it.
Just like Austin, I also value a lot about stability, so I don’t think I’m going to change everything and rebuild everything from scratch. But partly related to what Austin mentioned earlier, Go is about programming at scale. So both scaling to people and scaling to machine. And as the Go user base grows really, really fast, and ever-growing usage and needs, but the Go team have very limited resources. The number of people on the core team essentially never grow; it probably shrinks from time to time. So it’s very important to build a platform that is more capable of scaling to more people and usage; building tools or APIs or platforms that other people can build things on top of it, and to drive it to more of a solution. That is very important.
I’d also like to solve the scaling to machine part. Go is about scaling to machine, and it has really nice built-in concurrency support. But as the hardware trend grows, more and more cores, and larger and larger machines, and many, many containers and services, it’s kind of at a point where we’ve seen more and more scaling problems, too; scaling to machines, basically. Handling a large number of core counts, or very large memories, or other things. I’d like to have the opportunity to solve this problem as well.
[46:28] I feel like all of the problems you’re flagging are really important areas to be digging into, and excited to see as you are more than two weeks into the role, as you’re able to have that space to really settle into the role… I feel like it’s going to be exciting to see the things that you’re doing, the conversations you’re able to have with the community, and how you’re engaging with these really important problems, that are so important to all of the Gophers around the world.
I just have one final kind of area/question I wanted to explore with you all, which is around just that - the community. So we are engineers, the code is really important, but I think one of the things certainly that drew me to the Go community was the people. Agnostic of the language, honestly, agnostic of the wonderful things it can do and the wonderful apps I can build with it, the people made the community what it was. Both their brilliance, their ideas, their outspokenness, their willingness to change, but also to have constructive, sometimes heated conversations online with each other about decisions that are made… So I’d love to hear a little bit, maybe from Austin, you first, about how you’re thinking about engaging with this deeply interested, excited, deeply invested community that we have in the Go community, and how you’re thinking about that interaction in this kind of new role.
It’s tricky, because it is such a big community. I think the Go project has done a lot of really great things over the past several years in sort of growing and maintaining a really strong community. I think we have good community standards, which are very important, and I think we value a lot of different forms of community, which I think is also important. Just like bringing lots of technical perspectives to a problem is important, I think bringing lots of perspectives to community is also very important.
One of the things that I’ve been thinking about is how to sort of reduce the barriers between the Go team at Google and the community at large. I’m frankly way less online than Russ is, and for the past 12 years a lot of communication between the Go team at Google and the rest of the world has been through Russ. I would love to widen that channel, and sort of democratize it more, which I think would be both good for the visibility of other amazing engineers on the Go team at Google, and amazing engineers in the community, and I think it would also help break down more of the barriers between our team and the community.
I don’t know what a lot of this looks like. One small idea that I’ve been exploring is - I know that Russ has a very popular personal blog, on which he posts sort of the more detailed internal thoughts on some of the big things that are happening in Go, things that sort of aren’t official yet, or haven’t been accepted yet, but we want to get more in-depth in our thinking on… But I think it’s great that we get that out there, but I think it’s also kind of unfortunate that it’s on Russ’s personal blog.
[49:56] And so I’ve been thinking about how we could democratize that more to the rest of the team, and also to potentially even community members, because I think there’s a lot of really interesting thinking that goes on inside the team. And at the moment, a lot of it never sort of bridges that wall around the team. And I think part of that is that we just don’t necessarily have a good forum for doing that. So I would love to create a forum where just anybody on the team can say “These are the things that we’re thinking about. It’s not official yet, we don’t know what direction this is going to go. We don’t know if this is ultimately going to get accepted or produce anything”, but get more of the process out there in addition to the final product.
I think that’s a really, really great idea. I think that I’m certainly extremely excited about that. I mean, correct me if I’m wrong, please, but what I’m hearing is - is it more of a… And this is maybe – I’m giving away my perception. I feel like when I joined the Go community, I think it was maybe an hour into my first Go meetup, and I was like “Oh, do you know Russ? Do you know Rob?” There were three or four names, that it was like “Oh, if you don’t know these people, then you’re not a Gopher. These are the faces of Go.” Not in so many words, I will be clear. The Gophers were lovely, lovely people, but they’re like “Oh, no, these are the faces of Go.” It almost felt - not in a bad way, but these were the rockstars of Go. And there was somewhat of a feeling of - and this is where I’m going to be candid… These couple of white men who were leading Go, and I was looking around the meetup and going “No, but this is such a diverse, incredible group of interesting people…” And Russ, you’re extremely interesting, don’t get me wrong, but you see what I’m getting at. I feel like there’s always been this sense of “These are the rockstars of Go”, or the Go team even. You have meet and greets with the Go team, which I is brilliant, and it opens up that internal discussion of what you all are thinking about on the Go team versus what the community is thinking about… But there has always been that feeling for me, and it’s a feeling that I think many people I’ve spoken to also feel, that there’s this almost – there are the kings of Go, and then there is their incredible court of incredibly talented engineers, and then there’s the rest of us.
I know, Russ, you would never, ever say that in that sense, and this is not something that has been consciously implemented, but I’ve certainly felt it, and it is that almost feeling of accessibility, that I think you have to really be pretty outgoing to break through. I feel like the only reason I know Russ is because I was like “HELLO! I’m Angelica. How are you?” And let’s be very self-aware of the fact that not all that many people – not everyone feels cool about doing that.
To be honest, I did it when I was like “Oh, this weird Go language”, so I had no perception of the amazingness that was Russ Cox. Had I done it a few years later, maybe I would have been a little bit more quaking in my boots. But can you see what I’m getting at? Is that something that you are also perceiving, and something that you’re trying to shift away from? …this kind of unintentional, I will say, hierarchy of gophers.
Yes, absolutely. I think it’s a little tricky. We absolutely 100% do see that. It’s not intentional. I think it’s an easy consequence of the way that Go is structured; easy but unfortunate consequence. And I think we do want to break that down. At the same time, I think it’s important to maintain leadership in order to maintain the coherence of Go. And so it’s a difficult balance.
[54:00] For sure. Don’t think I haven’t forgotten you, Cherry. I want to hear a little bit about you, and how you’re thinking about kind of pulling on the Go community. I know your new space is a space that many gophers are very invested in, they care a lot about, so I would love to understand a little bit how you’re thinking about kind of drawing on the community.
Yeah. As Austin mentioned, this is a tricky and hard thing to solve. But I also think it’s very important to hear from the community and the Go users. That’s what’s most important to Go, actually. That also brings in ideas and opportunities that are probably very hard for us, people in the core team, to think about. We are a group of compiler runtime engineers, and for reasons we think things probably differently from the majority of Go users. They want to write Go code that’s very different from what I would write for the compiler. So I really think we should hear more from them, what they actually want to do with Go, and what problems they are actually facing, and what we can do to solve that problem.
I’m not sure what the actual form it’s going to take, but maybe we’ll just do more user studies or more QA sessions. I don’t know what the form is, but I’d really like to hear more from the community and users.
And to be clear, for those who might be kind of listening to this episode and might not be quite as close to how the Go team, Google and the Go community interact - there are a lot of forums already in existence, to be clear. There’s a Go developer survey that is sent out… There is a lot of open discussion forums that are out there around the language…
I just think – it’s kind of like, Austin, you mentioned there is a challenge that as the community expands, there are those who are extremely invested and know where to go for these discussions… They know to go to Russ’s blog. They know to go to XYZ issue tracker. But for newer Gophers, who maybe are not sure quite how to engage in the right way, and just given the volume of people too, I personally have jumped on some issues sometimes and read the 200 threads of back and forth, and been like “Um, do I really want to insert myself into this discussion…? I think I’m good for now. Let me just watch it.”
I feel that way often, too.
But actually, that might be just something to close out our discussion with, is for kind of older Gophers, newer Gophers, people who want to re-engage with the Go community, watch closely as there are changes coming down the wire, and kind of keep an eye on the wonderful work I’m sure that Cherry and Austin, you’re going to be doing… How can they do that? Where are the great touch points? Maybe, Russ, you can mention a few, and then Cherry, Austin, if Russ forgets anything, you can jump in.
Sure. So it depends on what volume you want. Certainly, the Golang Nuts email group is very high volume. The Golang dev email group is fairly low volume, but when there are sort of technical development questions, that’s kind of where they end up.
You can go to our code review server, which is go-review.googlesource.com, and then you can watch every single change if you want to. That’s sort of where all the kind of equivalent of the pull requests are. You can go to the issue tracker, which is on GitHub, but it’s also just go.dev/issue.
And then the other place that – and so the issue tracker is also pretty high volume, obviously; the code review server is very high volume. One place that’s slightly lower volume is the proposal issue. So it’s go.dev/s/proposal-minutes. It’s also just issue 33502 on the Go issue tracker. And we post once a week just the list of what proposal issues have active discussion on them.
[58:03] And so if you want to follow along with the proposals, and make sure that your input is represented there, you can star that issue, and then you’ll get the GitHub mails once a week with just the list of current active proposals. And then you can go through them once a week, and make sure that the points that you want to say either have already been raised, or you can raise them. And that’s a lower volume, but kind of high impact way to be involved, too.
Those are the ones that come to mind. Obviously, also, going to conferences, talking to people, going to meetups, meeting other people - all of these are good ways, too. But those are the sort of direct electronic ones that I can think of.
And I can ping in the episode notes, if you’re listening here, I can put a link to a number of the things that were just referenced. Also, I will put a link to Go Bridge, which is the – I’ll ping the link to the YouTube and to the meetup page. That’s where you’ll see a list of global Go meetups. There’s also – you can just keep listening to Go Time if you want to be engaged… And/or you can go to one of the many global conferences that we have. Unashamedly, I’m going to plug GopherCon in the US, because it’s a brilliant conference, with brilliant organizers, that do a really good job, I think, of structuring a good agenda… As she smiles and pats herself on the back.
Thank you so, so much for spending this last kind of hour with me. It’s been truly a pleasure to get to know you, your thoughts around go, the things that are exciting you about these new roles… And I’m sure I speak for the Go community and the Go Time listeners when I say we’re very excited, we’re here to support you, and more to come.
Break: [59:51]
Now we’re going to move over to the more spicy part of the episode, that Russ knows I love, which is where we talk about unpopular opinions.
Jingle: [01:03:36.12]
So who has an unpopular spicy opinion?
I have one that maybe is unpopular outside of this room… Which is that the best place to be a software engineer in the United States is the Boston area. I was looking around, and Cherry, Austin and I are all in the Boston area, and so clearly, that is right.
Wait, really?
Yeah.
Why? I feel like you need to validate this statement.
It’s just empirically true.
Statistics.
So we’re moving away from this king model, but yet the reason why it’s the best is “Because we’re here.” [laughter]
That’s exactly right. We don’t have earthquakes, and Massachusetts is not going to fall into the ocean.
Okay. I will think about that point. I might not agree with it, but I will accept it. Cherry, Austin?
Okay, I can go. I usually don’t have that many – I have an unpopular preference. But opinion, I don’t really – I’m not that opinionated, but I do have one. You were mentioning pull requests… I find the pull request workflow, the GitHub pull request - it’s very hard to understand. It’s very hard for me. I think Gerrit model is way better. It’s just way easier to do.
Plus one.
Can you say a bit more? Is it the order of steps? What specifically is the thing that annoys you?
Yeah. Say I just want to send a one-liner patch to some project I’ve never worked on… I have to first fork their project into my own GitHub page, and create a branch, and commit to my branch, and then send a pull request… At that point, I’d rather just send an email, say “This is a one-line diff. I want to make this one-line diff.”
[laughs]
Why do that…? I find it’s really hard.
Okay, so for any of you who work at GitHub, get on it. Austin.
Yeah. Let’s see how quickly I can get myself cancelled here. Tech debt is good, actually, if it’s managed intentionally as an investment. I think our industry has developed a very visceral reaction to tech debt, but I like to view tech debt as a form of debt. Just like a financial investment, some tech debt has a very low interest rate. And that’s probably a good investment, because it meant you could shift engineering resources to higher-value work. In fact, fixing that tech debt may actually be more expensive than paying the low interest rate on it, because you will probably introduce bugs.
Of course, tech debt that has a high interest rate is a problem. This is tech debt where it regularly causes bugs, or it’s in code that you’re actively changing, and it’s impacting your engineering velocity, and honestly, happiness… But because I believe tech debt can be good, I also believe that it’s important to create a culture where people actually feel empowered to pay down tech debt when it becomes a problem… Which can be really tricky, because usually, when tech debt goes from being low interest to being high interest is exactly when you’re trying to build something new, and get something out the door. But I think it’s important that you have a culture where it is okay to fix things when they need to be fixed, and to leave things alone when they don’t.
So you started that statement and I was like “Oh, I don’t know about this…” But now that you’ve explained it, I don’t know how unpopular it’s gonna be.
Well, I really appreciate the time. My unpopular popular opinion is that we need a gopher drinking matcha, that is light green, and has maybe some lovely Cherry locks of green hair, and maybe it’s like one foot on top of a crown, to symbolize the new era. [laughter] It’s been an absolute pleasure. I hope to have you on again soon. Have a wonderful rest of your days, your week, your month, your year, your many years, continuing the beautiful development of Go.
Thank you very much. It was a lot of fun.
Thank you.
Thank you so much.
Our transcripts are open source on GitHub. Improvements are welcome. 💚