In this insight-filled episode, Bill Kennedy joins Johnny and Kris to discuss best practices around the design of software in Go. Bill talks through scenarios, lessons learned, and pitfalls to avoid in both architecture and coding of Go projects.
Bill Kennedy: Now, when we talk about Go - one of Goās language design philosophies was being able to do more with less code. Itās coming from these ā I believe. I canāt speak for Robert, and all⦠But I believe itās coming from these ideas that if thereās less code you need to write, thereās going to be less bugs. Now, why did they say this? Because Stroustrup says āIf youāre writing more code than you need, it results in ugly, large, and slow codeā, where ugly means youāre leaving places for bugs to hide, large means youāre ensuring incomplete test coverage, and slow means you now start to make shortcuts and dirty tricks away from your frameworks and your patterns, because youāre moving fast and the code gets out of control⦠And these things absolutely happen.
So I think weāre talking about all of this⦠It all works together, and I think Go is tied into that. And we complain about Goās error handling. We love to complain about it. But do you know that there was a study done where they looked at 48 critical failures that brought down systems. Hundreds of bugs in Cassandra, HBase, MapReduce, Redis⦠How many systems run on Redis today? And theyāve found in this study that out of those 48 critical failures, 92% of them could have been avoided if error handling was done better. Failures from bad error handling.
So again, I think that Go designers knew this. They knew this, because they were developers themselves. They were not necessarily academics. They had to build software. They knew what the average developer needed, they knew where they were falling down. And I think Go comes in and solves these things.
[36:15] Personally, I think when somebody complains about error handling in Go, theyāre complaining about ā they want it easier to do, not easier to understand. [laughter] We come back again, right? So sometimes when you make things easier to understand, things have to be a little more tedious.
But hereās another design philosophy, Johnny? Two of them. One, you shouldnāt be writing code for yourself. You should be writing code for the next person that has to come along. Because if you donāt, if youāre not thinking about the next person and/or the average developer on your team, when you leave, that codebase leaves with you. It gets replaced. And the 3, 4, 5 years you spent on that ends up resulting in meaning nothing. Iāve got code thatās 20-something years old, 10-something years old in production right now. The 20-year-old code should go, because thatās way too long⦠But I think itās there because I always wrote code with the understanding that somebody else has to be able to maintain this. It wasnāt about me, it was about the next person, and that allows that code to now not just have to be replaced, right? You need to have that design philosophy in your head; you need to be thinking about that, āWhoās the next person thatās gonna come along here?ā And then youāre always writing code for the average developer on your team.
If youāre the average developer on your team, that means I can wake you up at 3 in the morning (God forbid) if I have to, and you can handle the bug. Thatās the average developer. If I canāt wake you up at three in the morning, then youāre below average. So another question is āWhy are you below average? Is it because Iām failing you, or are you just not coming up to speed?ā And then for me, the next thing is the above-average developer. Thatās scary, because those are the developers that tend to get bored, and instead of being able to write for the average developer, or bring the team up - thatās where the clever code comes in. Thatās where we trip up.
And I tell people all the time, āWhen youāre hiring, evaluate who this person is for your team. Are they below, are they average, or are they above?ā And consciously understand what youāre gonna need to do as an individual and a team to get this person in the right place. If theyāre below average ā which is great; letās hire developers who are below average for our teams, so we can bring them up and we can create a stronger team. Those are the best developers in the world, because you can really teach and train them. And now youāve got somebody who will stay a long time and really work hard and thank you for the opportunity.
But if you put me on a team thatās doing business APIs, Iām above average. If you put me on a team doing crypto, Iām below average. And if I wanted to learn crypto and you gave me that chance, I would be ecstatic, and Iād work hard, and weād get there. But if youāre hiring somebody whoās above - and Iāve done it before - they can either be amazing mentors and coaches, which is why youāre hiring them, I hope, or they can create utter chaos and destruction, because everything theyāre doing is not comprehensible to anybody else on the team, and youāve gotta maintain it.
[39:40] So those are design philosophies around building teams, around the ideas of all of this stuff. And you wanna apply it back to micro-level decisions, like constructions, functions versus methods, to macro-level decisions around app layer, business layer, foundation layers of code. Policies for these. Import policies. Error handling policies. Who can shut down an application? Who canāt? Who can log? Who canāt? Who can wrap errors? Who canāt? Who can set certain import dependencies? Who canāt?
And you donāt have to have all of it day one. You have to develop it as āSuddenly, thereās a hole in the engineering decisions. Hm. We donātā know what to do here. Okay, that means we may not have a design philosophy here.ā I get excited when that happens. Iām like, āOh my God, weāre gonna have a design philosophy for this. Oh my God, we get to do something new! WOOOH!ā
Now, youāve always got some of your base, foundational, right? But those are exciting days. And itās also exciting sometimes when somebody finds a hole in a design philosophy or policy, where we thought this was the right thing to do, and suddenly weāve found an exception. And thereās exceptions to everything. There are some exceptions you just canāt take. I donāt really take exceptions between project layers. Iāll never let the foundational layer log. Thereās no exception to that. If you have to log, youāre in the business. Thatās it.
But then there are other exceptions⦠Hereās a good one, Johnny. Hereās one where you might take an exception. So baseline design philosophy - a type system is not to be shared. A type system exists to allow package, which is a unit of code in Go, a clearly compile-time unit of code. A type system is design to allow data to flow in and out of the package API, where a package has a purpose. So if the type systemās job is to allow data - if. Thereās my philosophy. If you agree with this. You donāt have to agree with anything Iām saying today, by the way⦠It is totally fair. But if you believe that a type systemās job is to allow data to flow in and out of a package, then that type system is highly localized to that package and that package only. So now you have to make a decision about every API. When it comes to data flowing into an API, you have two choices. You could say āI want the API to accept data based on what it is.ā This is what I would call concrete functions, accepting a concrete type. It can accept a user, and only a user. Thatās what it is. But thanks to interfaces, we can write polymorphic functions letās say āNo, this API will accept concrete data based on what it can do.ā And thatās a next level of refactoring, hopefully; I donāt wanna start there, but suddenly you realize āNot only can I work with a user, I can also work with a customer. Based on this common behavior, we make it polymorphic.ā Okay. We all agree with that. You have both choices, and those are the only two choices you have. And those types should exist as types within the scope of that package.
Now, hereās where the fun begins⦠I have a strong rule that functions should only return concrete values. The functionās job is not to pre-decouple or wrap concrete data already in an interface; that is not the APIās responsibility. It is the callerās responsibility to decide whether or not they need the decoupling or not. Not mine.
So minus the error interface, which is a whole another set of interesting design philosophies and things I have, I donāt wanna see a function that uses http.handler as the return type. I donāt care if you know or think theyāre gonna put it into a handler already. I donāt care, itās not my job. My job is to give them the concrete value that they can then do with what they want.
āWell, Bill, then weāre leaking a āā No, youāre not leaking a type. They already imported your package. Thereās no leaking there, what are you talking about? Stop trying to abstract for the caller. Let them do it. Now, there are two exceptions to this. One is the error interface; weāre handling errors in a decoupled state. Thereās lots of reasons why we wanna do that.
[44:04] And until 1.18 comes out, there are times where you might need the empty interface. It should be a little bit of a smell, but letās be real, Iāve had to write a function or two over the last six years where I was trying to be, for whatever good reason, generic. Maybe I was just doing some data flow⦠And we were using the empty interface, which - now in 1.18 weāll be able to replace with a concrete type. [laughter] I mean, what is generics at the end of the day anyway? Generics is concrete, polymorphic functions, where the polymorphism isnāt happening at runtime, the polymorphism is happening at compile time. Weāre choosing the concrete type, the data ā because the only data that flows is concrete data anyway. Weāre just choosing that at compile time. For me, itās concrete polymorphism, as opposed to runtime polymorphism.
But thereās a philosophy - we shouldnāt be using the interface as a return type, minus those two exceptions, when they happen. And people disagree with me there, but⦠There it is. So if I see a function thatās returning an interface, itās immediately code review style, so āWhat are we doing? Why are we doing this? Prove to me that we need to take an exceptionā, but itās gonna be hard, because if I return the concrete type, that doesnāt prevent the caller from doing whatever it is theyāre doing.