Go code organization best practices

Welcome, gophers and wonderful listeners out there. I’m your host, Kris Brandow. Today on Go Time, we are going to be taking up one of our wonderful audience members’ suggestions for a topic. So Thomas Eckert sent in an inquiry for us to talk about utility functions and code that we have in Go, and how we can organize that, where we should put it, and in general, what should we do with that code.

Today I am joined by my wonderful co-host, Johnny Boursiquot. How are you doing today, Johnny?

I’ve been better, but I’m getting better.

It’s always good to hear. We appreciate you being here. And we’re also joined by a now reoccurring guest, Ian Lopshire. How are you today, Ian?

I’m doing great.

Awesome.

Yeah, happy to be here again.

Fantastic. So I’m sure that we’re going to get straight into the meta, since both Johnny and I are in this, or on this podcast today. But I guess we can start with something kind of high-level. So I guess I can read the actual suggestion, and then I have something I want to prompt us to start with.

Basically, Thomas wrote in and asked, “There are a lot of utility functions that I end up writing myself in Go; these are the kinds of functions that get asked about on Stack Overflow, and met with the response of, “This is trivial to implement, do it yourself.” For example, contains slice, string of slice, value slice, bool, which is just a regular contains function for an item in a slice, to determine if a slice of strings contains a particular value. “Where should I be putting these functions? I usually end up with a drunk drawer package of functions, like my own standard library, or I define them one-off within a file or a function. How do I organize the code that isn’t in my business logic, but also isn’t in the standard library? Additionally, how might this change given generics being introduced to the language? Could we get a generics contains function? Should we?”

[04:19] So the question I want to start off - basically, have you guys run into this before? Have you had the struggle of “I have this code that’s somewhat generic, somewhat reused; where should I put it? I guess, Johnny, do you want to start off?

Yeah, I can start off. I think we’ve had various conversations that touch on this topic on the podcast before. Oftentimes, I think those of us that are sort of experienced with the language sort of typically settle on this rule of three, kind of thing.

For me personally, I don’t consider something reusable, unless I’ve actually reused it; meaning that I don’t try to sort of preempt the process of prematurely trying to make something reusable, even if I have a hunch that it is going to be reusable.

So to piggyback off of the suggester’s explanation of utility functions and things like that, like contains, or whatever it is - these things on their face seems like, “Yes, that should be a reusable thing.” But for me, until I’ve seen something at least three times, I don’t have enough data to understand all the edge cases, to understand the variety of ways a particular piece of functionality or even domain within my application, within my ecosystem, how that’s going to be reused. So I don’t consider anything reusable, unless I’ve actually reused it a few times.

Ian, what are your thoughts on that?

Yes, I mean, I definitely agree with that kind of rule three idea. I would much rather copy and paste code multiple times, than try to make an abstract version that is bad. It’s so much easier to add an abstraction later than it is to take one away. So I’d much rather just copy and paste.

Okay. Yeah, I definitely agree with that, too. This is something we’re gonna get into a little bit later, but I think sometimes people try and overdry their code, where it’s like, “Oh, no, I’ve written this once. I have to figure out how to never write these exact same lines ever again, and put them somewhere useful.” So yes, I definitely agree with you on that rule of three there.

So I guess like on that as well, let’s say that you do figure out that you have something that you’ve used three, four or five times, the code is very similar… What other heuristics do you use to figure out like, “Okay, should I just be copy-pasting this around?” Kind of as the Go Proverb says, “A little copy is better than a little dependency”, or at what point do you decide, “Okay, I need to like put this somewhere that is reusable”? And I guess they’re obviously reusable within the same codebase, but then maybe microservices reusable between codebases. How do you start to figure out where those lines are?

I have a fun heuristic for this… If I can think of a good package name for it, that makes sense, I think that means it’s reusable enough that I should probably break it into a package. But if it’s going to be like this package that’s four words long, I’m just not going to. I’m going to just put it where I need it.

Mine is more of a sort of, why do I think I need to make this reusable? So there’s a reuse - I think you hinted at this, Kris… There’s reuse basically within the same code base. There’s also reuse within multiple projects. So I could extract away a package and make that reusable. But interestingly enough, I don’t know whether maybe it’s the problem domains I work in these days… This could be considered a dirty little secret, but I try not to sort of create packages, even if I happen to reuse them within the same code base. Very rarely am I sort of abstracting a package and making it sort of reusable across projects. So I tend to… If I have enough data to make something generic - not generic in the sense of the feature, but generic in terms of you reusability - I’ll try to make that happen within the context of the particular application or service that I’m working in… Because I don’t want to create a standalone package that other teams are going to depend on, and now I’m on the hook for keeping that thing up to date. I’m on the hook for managing and maintaining that thing.

[08:28] So I’d rather keep reusable chunks of stuff in my own project, because I know the moment I make it reusable by other teams, the responsibility scope for that code has just grown for me. And honestly, perhaps this is an unpopular opinion, or perhaps I’m just getting to that point in my career where I just am not looking for more work to do… [laughs] I’m just keeping it constrained within my projects. If somebody happens to know of my product, whatever I’m working on, and they come, “Oh, can I reuse that?” “Yeah, sure. Just literally copy the code. But you know, I’m not making this into a separate package for you; just copy the code.”

I’ll mirror that. I’m a fan of the internal folder; like, I’m not even allowing you to use this.

I think in this case, using the internal folder to be like, “No, no. I know you might want to use this thing, but please don’t. Please don’t touch this thing.” But I’ve also seen the other extreme, where it’s just like, “Okay, we’re just going to assume that by default, and everything’s going to go under the internal package. And that’s just going to be like the top-level thing in our Go codebase, and maybe there’s a couple things on the outside.” Do the two of you kind of agree with that, or do you think it should be just like, “Okay, no, I’ve made an explicit decision. This is something that perhaps someone might want to use, so I’m going to mark it as like, “No, no, no, don’t depend on this, because I don’t want to maintain it”, or are you on the side of like, “No, it’s okay to just kind of put everything under Internal and pull things out as necessary”?

I don’t have a habit of doing that. What I do have a habit of is just not exporting anything I don’t have to, so they’re not going to be able to get to it anyways. But I’m known to put kind of very general packages and internal just because they look appetizing to use externally.

Perhaps my other dirty little secret - I don’t use Internal a ton. Even then, I’ve heard some folks basically approach projects with basically saying, “You start with Internal, and then if you need to, take things out.” I don’t take that approach. I tend to sort of reason about the stuff I’m working on in terms of domain-driven design, and what is it that I’m working on, and then I worry about sort of shuffling files around, and then I worry about packaging.

I will literally have everything into a single – sort of at the root level of the project before I even start to create folders, creating packages and things. Because I don’t know what I don’t know yet. So I have everything in the same top-level package, and if I know I’m creating executables, one thing I absolutely do do is I create a CMD folder, followed by another sub-folder to represent the name of the binary that I’m going to build. But at the start, that is literally the extent of my organization.

Back in the very early days, I started following this pattern of putting things in a pkg folder, and all that jazz… Honestly, I think that’s an anti-pattern at this point. Like, you don’t need a pkg folder. If you happen to work on a project you’ve inherited that has one - fine, so be it. But you don’t need these things.

So I really rely on sort of a complete understanding of the problem I’m solving. I’ll even go as far as to actually have in my cmd slash whatever name folder I need, I’ll even have my main.go in there, I’ll have other Go files in there. Literally, I’ll have everything that I need to make this binary – especially at the start of a project, I’ll have everything I need to make this binary build to a proof of concept, whatever it is. Literally, I’m not thinking about packaging or extracting or moving anything. I just have everything in that folder, then if another folder underneath my cmd now needs some of that same functionality, then I’m like, “Okay, well, I have multiple binaries that are using these constructs and things; maybe now it’s time to start extracting some things away from the first folder and move that up a level, underneath the main repository, and then basically start referencing these things from those sub-folders and from those executables.”

[12:23] It’s all incremental. Even if I’ve built this kind of project before, I know, I kind of have a sense of where the seams are, I know what I’m going to do. I have the discipline or I’ve developed the discipline to not jump the gun on that. I really need to start seeing the edges of the service, and sometimes I will be completely happy with a bunch of stuff being in the executables folder, because I don’t yet have a reason to move these things out.

Again, basically, you don’t know what you don’t know yet. And sometimes, you build these projects, and they might be proof of concepts, and they might be things that don’t end up going into production… Why am I going to expend all this brain energy trying to make all these reusable things and abstractions and things, and I’m writing way more code than I need to, trying to avert cyclical dependencies, and trying to do all the – that’s just too much work. Again, I think I’m getting to my grumpy old man stage of Go development, but I’m just not trying to do more than I need to. [laughs]

Right. So the way you follow things is like start with just one package, maybe it is even just like the main package, and then as you have reasons to start pulling things out, that’s when you start pulling things out… Which I would also say is a very good way to avoid having people import your code; because if it’s in main, they can’t.

[laughs] You can’t import main. Right.

Like, “Stay off my lawn, you can’t touch any of my stuff.” So I wonder, does that change when you go from building an application that’s going to run as like a microservice or something, when you shift from that to writing a library that is going to be consumed by other people? Do you still take the same single package approach, or do you start at that point trying to like predefine where things might be laid out?

Break: [14:12]

I pull a page from Ian’s book, and basically saying, “Hey, I don’t export anything I don’t want you to use.” So even then I might not have an internal folder inside, even if it’s basically a package that’s designed to be used as a library… Because the moment you make something a library - again, going back to the scope of responsibility; it’s not just the code you’re now putting out there; you’ve basically taking on the responsibility, if you’re a good maintainer, of maintaining this thing, and keeping up to date, and who knows, there might be security things you need to do for that package, depending on what the problem domain you’re in… You have taken on the responsibility of keeping this thing up to date for the users of your package.

So I will never expose anything prematurely. Everything will be unexported until I have a reason to export it. I used to have a bad habit of actually exporting interfaces out of my libraries… That says, “Oh, yeah, like you can use this public interface, this export interface, so you can know what to pass in, when you call in some functionality into my package, whatever it is. Here’s some nice interfaces to help you out with that.” I don’t do that anymore.

Right now, I rarely, if ever, export an interface from a library. Because you can actually, in your own code, you can create single-method interfaces, whatever you need. You can create local to your code interfaces for whatever you need. I don’t need to give you exported interfaces for you to know how to use my library. As long as I include a document, the exported functions, as long as that makes sense, the naming of things; that’s still very much important, whether it’s a library or an executable. I make sure that the naming and documentation are up to snuff. I never expose more than I need to. And even the unexploited stuff, I will document them as if I’m going to export them, because who knows, I may end up exporting them.

There’s a way using GoDoc where you can actually show unexported documentation for functions, and stuff. So when I need to actually, on the command line, see what unexported functionality documentation is, I can actually do that. So things are documented as if I’m going to make them available for people to use. But I don’t do that, unless I absolutely need to. So I keep my packages surface area for interaction, very small. Only doing just what it needs, never more than that.

Yeah. Ian, what do you think about that?

I mean, it’s hard to disagree with anything that was just said there. I think that pattern of just starting with one bit, like one package, one thing, even if you’re just adding a feature - like, I like to start in one place, and do that proof of concept; and it should be that iterative approach, where you’re getting it working, and then we can go back and organize if we need to. But avoiding that premature organization and abstraction, and all of that, I think, is probably the most important part of the job. One of the most important parts of the job.

Yes, it sounds like a key thing to building maintainable software as well.

Wouldn’t you know…

I mean, I don’t think I declared this as one of our maintenance series episodes; our mini, but now max series… I wonder if you can just follow like Apple’s naming instances, where’s it’s like Pro, and then we can have Max, and then we can have Ultra when the series gets longer and longer. So I guess this is the Pro of maintenance series now, soon to become Max, when we level it up some more.

But yeah, from what you’re saying Johnny, and what you’re saying, Ian, it sounds like these are also like good strategies to kind of build more maintainable software for us in the future as well. Because once you do export something, you can never really take that back, at the end of the day. It’s kind of there forever… Which, kind of going back to what we said in the beginning of the show, sounds – you know, copying something around, “Okay, there’s two copies of this thing now.” Now, you don’t have this kind of piece of code there that two things are vying for control of at the end of the day, which I think is super-important.

But let’s say that we have figured out that like, okay, here is some code that really is generic, at the end of the day; it really is just like, okay, I’m just like copying it and pasting it all over the place, and I’m changing… Like, the changes I make, I can just run a Copy and Replace, or a Find and Replace in my code editor, just changing some names. But aside from that, the code is truly copy-pastable.

[20:21] At what point, or I guess in what way do you then take that code and make it so that it’s not so much copying and pasting? Or do you just say, “Well, this code is small enough, and I don’t want to make it into a library I have to support” so I do just continue copying and pasting it forever. Or do you do something else to kind of make it less arduous over time?

I think a good example here is the aws.string, where we’re taking a string and returning a pointer to it. I have that sprinkled throughout the codebase. It might be a struct that I’m trying to do… And before generics, I think the place where those were just where you needed them. But I think that’s like one of the prime candidates for like a generic package, where we can just take a type and return a pointer to it.

I’m kind of surprised there isn’t a package that just does that, that someone maintains out there. Or maybe there is, I just don’t know. I mean, I think it’d be hard to come up with a name for it, but…

There’s one just called Pointer, ptr.

Oh. Well, that’s a good name. So following your rule, Ian, that makes sense.

Exactly.

You see, for me, I consider things like that to be extraneous. [laughs] Again, I’m going to be that old man pointing at the cloud. I’m like, “Okay, if I need a pointer to something, I will use the language proper to make a pointer to something.” Yeah, it might be a little more verbose, it might not be a nice one-liner from a package, whatever it is… But again – I’m not saying it’s a bad thing. It’s just style, or desire for bringing in a third-party – I mean, I’m very shy when it comes to bringing in third-party dependencies into my codebase. Very, very shy. Because again, that whole responsibility model right for that package - I’m more likely to look favorably upon a package that I’m evaluating to bring into my codebase if it is actively maintained, if there’s activity on the codebase. And obviously, certainly for very specific kinds of functionality, where I’m not going to reinvent wheels. So again, basically, not doing more work than I need to, kind of thing.

So if I’m really going to be aggressively using a package to do things, in a specialized kind of work - yes, I will find a suitable third-party library out there. And of those, I will evaluate them to see which one is documented well enough, which one is maintained actively, which one is referenced by other things, like the pkg.go.dev now shows you - or maybe has always shown you - how many packages import whatever package you’re looking at. So you can see how used, how well used that particular package is. And that way, you know folks have worked out the bugs and issues and whatnot. If it’s been imported, and it’s heavily used, then you know a lot of people have run into the potential issues you might have run into. All these things factor into the selection process for bringing in a third-party dependency.

To me, it’s a matter of – like, I’ve been around the block long enough to understand the cost of abstractions, to understand the cost of bringing in third parties to take away perhaps some labor you might have had to do on your own, to perhaps write things the long way, or the more verbose way. So everything is a trade-off. Everything. So it depends on what costs are you willing to bear today, and also in the future, as your software, especially if it goes into production and you end up having to maintain it for the long-term - are you willing to bear the costs of all these third-party dependencies, of all the abstractions that you’re introducing, even the ones you introduce into your own codebase?

I’ve seen very large projects that basically have some abstractions that no longer hold water today. Perhaps back then, when they were introduced, they made total sense. But today, you have little notes, little side notes on these things, “Well, I don’t use that anymore,” or “Deprecated. Use this thing instead.” Every time I see this, I’m like, “Yep, I know exactly how this one came to be. “There was some sort of abstraction going on, we thought we were going to do this, but business changed and now we’re doing this instead.” So yeah, I think it’s all the lessons learned really, at this point.

[24:23] And part of me wonders if this is like a shift from the pendulum swinging… Like, I know when I first started my career, the kind of ethos for everything - I think partially because it was so difficult to get dependencies - it was like, “Build stuff yourself.” And I think I joined the industry right as we were swinging away from what was deemed kind of negatively as a not-invented-here syndrome, toward the proudly found elsewhere, which I think is the positive spin… Obviously, trying to get toward one side of those things.

I kind of wonder if what you’re saying, Johnny, is the kind of evolution that’s happened from us swinging a little too far to the other side of “probably found elsewhere”, where we have these problems where doing this type of evaluation on external dependencies to do it properly takes a lot of time and effort and energy. And it seems like the rate at which we keep pulling in dependencies, we’re not really doing that at the end of the day. It’s like, there’s a famous Node.js bugs where someone deletes a package, and then the entire ecosystem just breaks for a long period of time. And even more recently, the different types of people sneaking malware into their own packages to get people to stop using them, or because they’re mad about something… Just all sorts of problems that exist because we depend on code that isn’t our own for a little bit too much stuff.

So I wonder if either of you – do you kind of agree with that, that we have swung a little bit too far? Do you think it’s something else that is kind of like, “Okay, well maybe we should be rethinking our dependencies”? Or is it just like my perspective, like… I’ve seen this change over my career, and it’s just like, “Okay, well, if you’d started ten years earlier, you would have seen the same change over the course of those first ten years” or whatever?

I think we’re kind of lucky with Go, because we have such a good standard library. A lot of what you would have to like kind of import from a third party before, you just don’t need to. So I think in Go, we might have veered towards the kind of just rewrite it and don’t import a dependency, but I think that’s okay, because we have such a good standard library. Obviously, there are things not in the library, but… Yeah, I don’t know where I was going with that one.

[laughs] I can add some flavor. I’m not sure if it’s so much as sort of the pendulum swinging back and forth… And it very well could be, depending on where you’re sitting. But I think generally speaking, whether it be with Go, Ruby, or Rust, or whatever it is, there’s a reason why you have things like best practice. You need bad practice, or you need not-so-good practice, in order to have best practice. So that means it takes a whole lot of people doing a whole lot of things, sometimes smart, sometimes not so smart, in order to see enough patterns of usage and start to recognize over time - and that’s the critical element here, time - to understand, “Well, if I engineer my piece of software this way, I am least likely to run into these kinds of problems that we’ve seen occur again, and again and again and again.” Hence, best practice.

So you need the time element. That’s sort of a key ingredient in understanding sort of that pendulum, if you will, to understand where are we now, given this context, and how is it likely to change when things swing back around. And really, as you mature as an engineer, I think that the best thing you can do for yourself is expose yourself to enough patterns that you can recognize which one is applicable where, and when. There’s never going to be the perfect solution; such a thing doesn’t exist. It’s only what are the trade-offs you’re making today, given the information you know now, with the hope of having maintainable, well-engineered software into the future. You’re just hedging, it’s all you’re doing. Only time can tell you. And even if the pendulum seems to be swinging, “Oh, this is the new hotness, organize your code this way” or, “This is the new hotness…”

[28:22] I mean, I remember when – again, going back to that whole pkg thing - I remember when that was the new hotness. I mean, everybody was putting pkg into every darn Go package. Heck, I did it, because I was just going along with the flow. I’m like, “Oh, yes, some of the most well-respected people in the community are using pkg.” I’m like, “Okay, I’m going to start doing that, too.” And I’d literally cargo-culted the thing into my own Go code. I don’t even know why I was doing it. I was like, “Oh, yeah, so and so does it. They must know something I don’t.” But only over time do you realize, “Okay, why am I doing this?” You start asking yourself, “Why am I doing this?” Because I read a blog post somewhere, where somebody had a specific set of concerns and constraints, and they said this was the right way, and now all of a sudden I’m using their solution to solve a problem I do not have? What kind of sense does that make?

So when I see blog posts around package structure and things like that - and there are a few well-written ones - I always try to understand “What problem are you solving?” Some of the best blog posts on the matter - I think Jon has written one as well, that we’ll put in the show notes. I’ve also seen some well-written ones from Ben Johnson of BoltDB fame. I’ve also seen some from Bill Kennedy… People who know their stuff. But not all of their approaches and solutions works in all of my use cases.

So you could say, “Well, Ben said do this. So all my projects now are going to look like this.” Well, no. No. What problem is Ben trying to solve in his blog post? What problem is John trying to solve in his blog? So you have to take context into account. And you have to use time as your friend, as an engineer, to learn to recognize patterns and say, “You know what? You know what would work best here? I’m not a big fan of MVC, but you know what? Organizing things as if it was sort of a Rails project, by having models, views, controllers, folders, instead of arranging things as a sort of domain-driven way - this model may be better in this particular case, because I’m building a CRUD app, than this model here, which is more of a business vertical, very specific service that I need to build, or a third-party package I need to build right for other teams to use.”

So again, the context is going to be what drives you towards one or the other, but you’re not going to know that you have options, unless you educate yourself, unless you add time to mix to learn to recognize patterns.

Yeah, no, I think that’s spot on. I wonder too, I wonder if this is like part of our problem with best practices. The way that a lot of people conceptualize them right now is that it’s kind of like – sometimes it’s just kind of like, “Oh, no, just do this thing.” Because as you said, like “Oh, these important people are saying that we should do them” or “These important people are saying that this is the way that we should make these things happen.”

I feel like Go is kind of unique in this way too, where - when I look at things like the idioms we have, or even things like Go Proverbs, I’ve never seen them as things where it’s just like, “No, shut up and just accept that’s true.” I feel like they kind of call out to you to and they’re like, “No, no, no… Sit down and actually think about this thing a little bit more thoroughly, and make sure you actually understand what it’s saying.” Because even at face value, it forces you into that kind of thought process. I wonder if that’s a hallmark of what a good best practice is; this thing that’s not simple on its face. I think “Don’t Repeat Yourself” is one of those things that is like sort of a best practice, but it doesn’t have that same type of impetus for you to kind of dig a little bit beneath that to make sure you understand it a bit. I feel like it’s so catchy that we just kind of repeat it all of the time, without understanding – maybe its genesis at the end of the day. Do you agree with that, Johnny?

[32:16] We have these sort of sayings, these best practices that we’ve accumulated over years of practice, the software engineering practice. “Don’t Repeat Yourself” is a good one; like, DRY, DRY, DRY, DRY all the things, all the time. Man, I violate that rule every single day I write Go code. I’m like, “Yeah. I mean, yeah, don’t repeat. Yeah, sure. Yeah. I mean, yeah.” In most circumstances, generally, that’s – yeah, you don’t want to repeat yourself. Well, you have to understand the intent behind that saying. The intent is, “Well don’t repeat the same functionality, because now when you need to modify it here, you also need modify it there, there, there and there.” But if I never have a there, there, there, there, and there, and only have here, why do I care about DRY? Or if I only be repeat it a couple of times, or three times. Again, that rule of three - once I hit it three times, I might think, “Okay, is it worth refactoring?” Okay, maybe the fourth time it comes up - yes, maybe. Yeah, maybe I’ll create some abstractions and sort of reuse it within that codebase.

But we can’t go into projects with sort of this Bible of sayings, be it Go Proverbs, or the Gang of Four Book, or domain-driven design… We can’t go into our projects with these seminal works, and expect “Well, I’m just going to throw these books at the problem.” Honestly, especially at the early days of a project, I’m never concerned about design. Again, maybe dirty little secret of mine, but I’m never concerned about design. I’m never concerned about reusability. I’m never concerned about abstractions. It’s too early. It’s too early.

The only thing I know is – you know, a manager comes to me and says, “Well, we need a service to take this data from point A to point B.” That’s all the requirements I have. Now I’m digging through docs, a lot of projects, asking other team members, “Hey, have you seen this thing? Do you know what this thing is?” I’m literally researching what the heck it is I’ve just been asked to do. I have zero idea what it is I’m about to build, so what makes me think I know how to design this solution?

I mean, don’t get me wrong, I’m not going to approach it as a novice; I’m not going to be shipping something that is not of high quality. The code will be high quality, but the code will not be concerned about design and reusability and architecture and all that stuff. That’s the stuff that I do when I know I’m going to put it out in the world. If it’s a package that’s going to be reused by teams, then I sit down – after I’ve solved the problem, then I sit down, I worry about design and architecture and rearranging and putting things in the right places. If I’m worried about sort of performance, then I’m going to take my proof concept and then okay, I’m going to run that through pprof and saying, “Okay, well, where do I have unnecessary Go routines spinning up and doing things? Where do I have waste? Where do I have some allocations that I could maybe reduce?” Only then do I have these concerns, but we can go into projects with these - granted, great ideas that have come over time… It will be an analysis paralysis for days before you even try to solve a single problem.

It’s maddening to think that we can just throw all these best practices at every problem every time, and then we wonder why we can’t move fast; because we just stuck designing.

Or do you wind up with factory, factory, factory, factory. [laughter] “I just took the whole Gang of Four book and started applying it all over my codebase, and now my code is correct. These are all the best practices and the patterns we should be following.” Ian, do you have anything you want to add to that?

I just want to kind of extract something from what Johnny was saying. I think what’s underlying everything he said is thinking what problem am I solving here? You get your proof of concept, and what’s my next step? Is it making this available to another team? I’m going to make my decisions based on that. If it’s performance, I’ll make my decisions based on that. So I think no matter what your best practice or proverb is, the underlying “why” is what you have to think about.

Part of what you were saying, Johnny, reminds me of this thing I’ve been thinking about for the past week. I don’t know why I started thinking about it, but it just kind of like popped into my head. Our industry is very good at taking those early prototypes and just kind of shoving them into production. Right and just doing “Okay, we’re going to we’re going to make an MVP, and then it’s going to prod, and we’re going to rewrite that code later.” And then later never actually comes until it’s like so terrible that we are like, “Okay, we’ve got to throw out this whole project and then do a greenfield,” even though greenfield is really a brownfield, but “We’ve got to just rebuild the entire thing from scratch.”

And the thing I was kind of thinking about was like, if you’re going to go build an actual product at the end of the day, whether it’s a car, or a widget, or whatever, there’s the prototype version of it you build, the proof of concept that’s like, “Okay, we can do this.” But then you have to go and kind of have like a production process, a manufacturing process, where a lot of those elements of that thing might change; the overall idea of it stays the same. You’re still producing a car, or a widget, or something. But the way that that thing gets produced is significantly altered in order to make it so that you can mass-produce it, so you can put it on a production line, and turn those out so people can buy them. And it feels like we don’t really have an equivalent of that in software engineering. It feels like we’re missing that step of, “Okay, we wrote this thing, we built this thing as the prototype. Now we know our idea works. Okay, put that over there, we can reference that. Now we’re going to go rebuild the new thing.”

I kind of wonder, do you guys feel that yeah, we are missing that kind of step, or do you think like, “Okay, we don’t really need that sort of step”, because we’re not making a million cars at the end of the day, we’re making like a set of features, so that’s a completely different thing, and that doesn’t really fit the same analogy… What do you guys think about that kind of analogy I just made there?

So the car industry - I’m not from a car industry, so I’m just looking at it as an outsider here; but from what I’ve noticed, the car industry - they’ll have these shows, where they have those sort of beautiful, futuristic-looking designs of automobiles… And you’ve look at the thing, you’re like, “Man, this thing is like – why don’t they actually build these things and put them on the road?” But these concepts, these concept vehicles - their purpose is not necessarily to sort of provide a proof of concept, even though there’s concept in the name, but it’s not to provide a proof of concept for a practical version of the thing. The part of the reason you have these very futuristic-looking, not really going to go to production sort of looking vehicles, is that they’re part of marketing. They’re beautiful to look at, they’re catchy, they demand attention… You look at it and it’s like, “Wow, I would love to sit in this thing and ride around with this all-glass vehicle”, or whatever the main appeal is. But they know these things aren’t going to go to production, one, because there are lots of constraints, including government requirements that certain vehicles have a certain mileage or safety things associated with it… There’s a bunch of constraints that are going to force this beautiful, futuristic-looking vehicle to being something lesser than the concept itself. But that doesn’t mean there’s no need for the concept. But its purpose is not to give you something that you can sort of take to production. Its purpose is to get people excited about something.

So if we take that exact same analogy and bring it into our world, as software engineers, the purpose of a proof of concept is to have something that proves an idea is possible. It is feasible. The problem we have - and this is something I’ve noticed and been a part of, unfortunately, many, many times over my career, is that the proof of concept, somebody will see it, and be it service or frontend, or whatever it is, the business will see it or somebody will see it and be like, “Oh, you know what? That’s good enough. Let’s put that in production.”

And all of a sudden, you’re an engineer, you put both hands over your head, you’re like, “Nooo…! This is not what I wanted. This is not what this is meant to do.” And you argue, and you fight, and you say, “Hey, really, we shouldn’t do this. This is not ready for production.” You make all the arguments and all the cases, and then at the end of the day, the thing that you didn’t think was going to be production-worthy finds its way in some version of production. So now we get shy, we become shy about the possibility of that happening again. If it’s happened to you more than a couple of times, your proof of concepts start looking less like proof of concepts, and they start looking more like, “Possibly, maybe, might go to production” versions of a proof of concept. You spend the time with on design, you spend the time on packaging, you spend the time on folderization, and making sure if a developer stumbles on your code, you’re not going to be embarrassed. You spend your time on tweaking and massaging this thing, when you could spend the time actually proving out a concept, that an idea is possible.

[44:27] So I think it’s an unfortunate place that we’re in with proof of concepts, but it’s one that I think we’ve been driven to this state where we as developers are shy, we are afraid that our proof of concepts that are throwaway, what’s meant to be throwaway code will actually end up being thrown into production instead.

Right. And we also have the engineers, they go through and they make that proof of concept and business asks, “How long till we can put it?” And they’re like, “Oh, it’s like 80% done.” Okay, that might actually be technically true, but that last 20% is the most difficult part of it. So yes, it’s not really that.

I think there’s a little bit of both playing in there as well. Some people are very like, “Okay, no, we’re only going to do this once, and we’ve got to do it the right way.” And you have a whole bunch of other people that are like, “I don’t know, we just need to need to build it and go.” Ian, I’d be curious to hear your thoughts on the analogy, on what Johnny said, and all of that as well.

Yeah. So only analogy side, I think the equivalent of those production cars, those sample cars you see at the big shows are like the dribble UI mockups. I think that’s where those exist. But as far as proof of concepts - I’m at the point where I kind of struggle knowing when a proof of concept is even necessary. When you’re just building a new feature or something, a lot of times you know it’s possible. So what do you think of that? Do you think proof of concepts are always necessary, or do you think it’s—

If you’ve never done a thing before… For example, a few years ago I was asked to look into this whole serverless thing. “Is this going to work for our workloads?” and things like that. And I was given a particular problem to solve, and then it says, “Well, can we do it without EC2 Instances?” I was like, “Well, I don’t know, it depends. How long is the workload?” Because at the time, I think there was a five-minute time limit, two Lambdas or 15 minutes, whatever that was, whatever the initial limit was. And I was like, “Well, if your workload is going to take longer than that time, then no, you shouldn’t use serverless; you shouldn’t use Lambdas and things.” And they’re like, “Well, what if we break it apart? What if we do smaller things, a bunch of small things”, like the best practice at the time? What if we make a bunch of different Lambdas, or invoke the same thing, put it in the pipeline and invoke the same Lambda multiple times, and you can scale it infinitely, according to the AWS marketing? Just do that instead.

I’m like, “Well, now we need to orchestrate”, and at the time AWS had Step Functions, which I think pretty closely followed Lambdas. Or at least the use of Step Functions for Lambdas basically became a thing. And then I’m like, “Well, orchestration - yeah, rather than to have one Lambda call another, and then one function waiting on another and timing out the whole thing, and now you can use step functions, and you can have ASL, Amazon State Language, whatever it is, you can write some of that YAML or whatever, and basically orchestrate all your things” - these were things that we had never done before. So in that case, yeah, I’m definitely doing a proof of concept, because I don’t know what I don’t know; I need to understand the characteristics of the platform, what it offers. I need to understand what all the knobs are right for the service that I’m supposed to be using. So there’s a lot there that I need to do.

So this is kind of an example that goes that’s sort of at perhaps the infrastructure level - to some code level, obviously, because the programming language that you use is going to have an impact on your serverless stuff, no doubt. So if I’m working with a Java project, because of the JVM, and I know that needs to be booted up, whatever it is, I’m going to take that into consideration. If you’re telling me to build something with Java and Lambda, as opposed to if you tell me to build something with Go and Lambda, then I’m not going to suffer the cost of a JVM. So again, all these trade-offs that you’re looking at.

[48:13] So again, the more you don’t know about the kind of problem you solving, the more I think you need a proof of concept. And that goes – this was an example at that sort of infra level. But if somebody says, “Hey, decode this data stream.” Like, I’ve never seen that before. It’s not like anything I’ve ever seen, so I might need to do a proof of concept to sort of understand what I’m doing. So I think it’s going to depend. To your point, Ian, I think it’s just something that we don’t naturally gravitate to at first, because - maybe it’s hubris, I don’t know; maybe we think we can just start working and we’ll figure it out, and whatever we come up with, “Ah, ship it to production.”

I guess I would say that I think – if I’m thinking about this from a writer’s perspective, I would say you should always do prototyping and proof of concept. And I think that way because it’s like - if I think about how I generally tend to write code versus how I would ideally write something… If I was going to write something, I’d sit down and I’d do some drafting, or some basic sketching of an idea… But I’m not going to try and edit as I write. I’m not going to try and copy it as I write, or even try and make the thing fit together properly, because that’s going to slow down the whole process overall, and it might even prevent me from getting to the goal I want, because I’ve taken up so much time doing it.

So I’m just going to rough out the idea, quick and dirty, get all of the hearts of it on page, and then start the editing process and refining it and trying to get it all shiny to send out to production or publication. And I feel like that is something we should do with code, and I feel like this is something that I would love to see more engineers doing, is like, “Okay, I’m just going to like rough out the idea, quick and dirty” like maybe not even have that many tests or anything… Just like, “Does this thing work for basically everything, even the smallest of features that we want to do?” And then have the solid idea of what we want to create, and then go back, and then rewrite it and redraft it, and put it into the structure and format that we want.

Because I think not doing those types of prototypes means that we’re kind of cutting down on the number of iterations or the space that we can think of an idea. For me, if I go to write something, and it’s like, “Okay, well, I know this will have to go to production”, and I spend a bunch of time having to write tests to make sure this small part of it works, it kind of kills my creativity at the end of the day. I’m kind of like, “Oh, I’ve gotta think about all of this stuff”, and it kind of knocks me into a different stream of thought than the stream of thought I was using when I was prototyping this feature, and I was kind of figuring out how to put this thing together.

So that to me seems like one of the failures we’ve had as an industry, is that we don’t encourage people to prototype always. And I think the reason we don’t do it is because – I think at one level, we kind of hold prototyping up on this pedestal, and it’s like this fun thing that you get to do. It’s what you do when you go play with new technologies. It’s not something you do when you’re just going to implement some boring business feature. I think it’s that mixed with a little bit of hubris, too. It’s like, “Well, this thing is so simple.” Of course, we can just write it correct the first time.

There’s so few lines of code that I’ve written right the first time that I actually came back a week later and been like, “No, that was the right thing.” Actually, I don’t think I’ve ever written code where it’s like “I wrote it once and that was it”, and then I came back and looked at it and didn’t think it was like… Every time it’s like, “No, no, this could be so much better.”

I think that’s like a practice that maybe we should start introducing into the way we do things, of just, you know, ease the business into it… And be like, “No, no, we have to actually write the code rough and dirty first, get all the small, high-level parts correct, and then we’ll go in productionize the code. And then we’ll go and add all the testing and then monitoring and all of the logging and all of this other stuff that has to go around it to make it spick and span.”

And obviously documentation, too. The amount of code I see in prod that doesn’t have a line of documentation, or they add the function name or the type name and then, dot-dot-dot to get around the winter, or whatever thing they could do. I think that part of that is because we don’t sit down and we’re not like, “Oh, no, we have to actually prototype this first, and then there’s this other series of steps we take to make it productionized.

[52:18] Yeah, I almost don’t think of it as like a proof of concept. I think of it kind of as levels of drafting. Like you said, I guess the first draft is like that proof of concept. I’m going to do it quick and dirty, and make sure it works. And then we go back and make sure we have observability, and all of this. Yeah, I almost think about it more as drafts.

Yeah. So I get to move on to the next segment, but Thomas, if you’re listening, I hope that we answered your question. And if we got on a meta rant and didn’t, I guess we’ll just have to do another episode and try to answer your question. But now it is time for unpopular opinions.

Alright. Ian, since you are a guest, do you have an unpopular opinion?

I feel like I never do, but… I don’t know. I cannot think of a good one.

It doesn’t have to be good. It just has to be unpopular.

Give me a minute, maybe come back.

Okay. I’m sure Johnny has a few. So do you want to share one with us, Johnny?

Sure. I mean, I think I’ve shared quite a few during this episode… I mean, yeah, if I could maybe summarize some of what I guess I hold most dear these days, especially around sort of code organization, is don’t organize your code in the beginning. You don’t know what you’re organizing yet. Just get the thing done, solve the problem, and then say, “Okay, well, what am I trying to achieve in now organizing this code? Am I organizing for readability and maintainability? Am I organizing because it’s going to be used by another team? Or by maybe the open source community? What am I organizing for?”

Obviously, I’m a big fan of domain-driven design. I always keep that in the back of my mind. I don’t tend to use sort of frameworky things, generating folders for me, and scaffolding. Again, the type of work I do - I have very little used scaffolding and things, and frameworks, and whatnot.

But if these are the kinds of problems you’re solving, if you’re building CRUD apps for a living and you want to use frameworks to generate things, and put files and generate things for you, put them in a folder, and for things to wire up properly you need to put them in a Models folder, and or whatever - I mean, if you if that’s your jam, that’s your jam. I’m not kicking it, by any means.

But yeah, beyond those use cases - man, solve the problem first, and then worry about organizing it, then worry about abstractions, then worry about what’s being used and reused and things, and don’t be shy to copy from across projects even. Just don’t be afraid; you can just copy a file here and drop it in your project. Just get things done. Just get it done, and then worry about organizing it.

So if you were to write a blog post on how you should organize your Go code, it would just say, “Don’t.”

Don’t. [laughs]

Like, by the time you need to organize your codebase, you won’t need a blog post to tell you how to do it.

Exactly.

We just publish that on the Changelog blog right now… [laughter] No, I mean, I definitely agree with that. I think people do rush way too fast to try and sit down and organize their codebases. I personally always start with one file, one package, just put everything in it, and then once it starts making sense to split things up, I start splitting them up. I think that’s the nice thing about Go, is that you can have a file that is maybe even 1,000 lines long, and it’s still very readable. You can still find things in it. I think the language is—

Whoa, whoa, whoa! This is where you and I fork. [laughs] A thousand lines long file? No, you and I are going to have words… [laughter]

[56:08] Disagreements! We’ll have a whole episode about this. Whether it stays at 1,000 lines is debatable, but I think files are – I’m more okay with people splitting things up into files. I think the thing I like to avoid is people that make 40 files with 20 lines of code in them. There’s a middle ground here. If I do write a file that gets to 1,000 lines, then I’m kind of like, “Okay, this is a little long in the tooth. Let’s, let’s break this up into pieces. Let’s see how we can format this better.” I’d definitely do that. But yes, I definitely think one package to start with, and then go from there. So, Ian, did you come up with an unpopular opinion?

I think I did. This might be a repeat, though. I don’t know.

Okay.

My unpopular opinion right now is that monoliths are probably the way to go for most companies right now. I think all these companies going microservices are doing one of those things where “Microservices are best practice, so we’re just going to mindlessly do it”, and not thinking “Why am I doing this?”

Wait, we haven’t learned from all the case studies of people who went micro and went back to, I guess, macro, or monolith, or something? A lot of times people think that, “Yeah, I’m going to adopt microservices, because it’s cool and trendy and whatnot.” And they’re realizing that when you go microservices, you’re not making just a coding decision, you’re also make it an operational decision.

So your developers, while they may have input into the decision to microservice or not to microservice, I think the operational folks, the SREs, the people who need to keep this thing running and orchestrate all the things and deploy all these things across infrastructure, make your thing into a distributed system that actually cohesively works. They need a say into the “to microservice or not to microservice” decision, because everybody is affected by that. So it’s not just the developer writing code and choosing to put a network boundary in between their projects. Everybody needs to go in with both eyes open.

See, we’ve just got to rebrand monoliths into macroservices, and that will be the new hotness, and that’s how we can get people over macro. It’s like, “Obviously, macroservices are better, because they’re bigger, right? Macro. Why would you do something micro when you can do a macro? It’s innovation. It’s bigger. We want to do bigger.” [laughter]

Alright… So yeah, we’ll put the polls up on Twitter and see if your unpopular opinions are actually unpopular. I am curious to see if they are. And as the rule goes, if they’re not, you have to come back and try again, to be unpopular.

Justify.

And perhaps one day you will be as unpopular as some of my opinions have been, to once again, warrant us some negative reviews on Apple Podcasts. If you liked the podcast, go and give us a nice review on Apple Podcasts, please. But yeah, with that, I want to thank you, Ian, for joining us for this lovely panel discussion.

Yeah. Thanks for having me.

Yeah, of course. And thank you, Johnny, for co-hosting, as always. Hopefully, we didn’t get too meta for our listeners out there.

Just the right amount.

Just the right amount.