How to make mistakes in Go

Hello, and welcome to Go Time. I’m Mat Ryer, and today we’re talking about how to make mistakes in Go. “Why would you want to make mistakes?!” Well, I’m glad you asked. “Tell us then!” I will, if you don’t interrupt – “Well, get on with it!” Right. So we all make mistakes, and they can be a great way to learn. So that’s why we’re doing this episode. We’re gonna hear about some common mistakes that are made. Or if you’re a brilliant programmer and you wanna make more mistakes, so that you fit in better, this is the episode for you.

We’re joined by the author of Manning Books’ “100 Go mistakes - how to avoid them”, Teiva. Hello, Teiva. Welcome!

Hello, guys. Thanks a lot for inviting me. I’m a big fan of the podcast.

Oh, then you can stay… We’re also joined by regular co-host, Johnny Boursiquot, isn’t it? Hello, Johnny.

Yes it is, isn’t it? Hello. [laughter]

Hello…

Wow, this whole show is turning into a mistake… Can we start over? [laughter]

No, we can’t. It’s live. It’s recorded live. That other voice you heard - and this may be my mistake, but we’ve invited Mark Bates back. Hello, Mark. Welcome.

Hello. I don’t think you can talk about making mistakes in Go without me being president… President or present? Either one…

[unintelligible 00:03:47.17]

Maybe both. I’m really good at it.

Yeah, exactly. And… And that’s it.

Really, really good at it. As a matter of fact, 99 of Teiva’s examples are from my repos… [laughter] Which is kind of nice. I feel very proud about that.

So much material…

[04:04] Great. Well, let’s dig in. Teiva, maybe you could just kind of tell us about yourself first. You’re an engineer at Beat, right?

Yeah, indeed. Software engineer in a company called Beat. You can visit it on thebeat.co. We are in the ride-hailing domain, and by the way - just a small part at the beginning - we are recruiting people, and we are doing a lot of Go, a lot of cool stuff, so if you want to take a look at it, please do.

Brilliant. Feel free to, if you wanna plug something, use the platform to plug something. Feel free. You don’t have to ask in advance, no problem.

Bitbar. [laughter]

[unintelligible 00:04:42.14] Johnny, come on… I wanna also do a quick shout-out to Lagos Gophers, which is a Go community in Nigeria. We’re doing a regular segment, Shout-At’s, where we shout at… Because that’s what Johnny said last week.

Do we just yell at them, is that what we do?

Yeah.

That seems awfully rude.

“Hey, Lagos! Nice one!” So if you’re in the area, in Nigeria, then you know, head over to it. Okay, great. Right. So - mistakes… Let’s see. Teiva, I asked you to pick some of your favorite mistakes that we could then talk about… So maybe you could pick – we can go through the first one. Tell us what was the first mistake you picked.

The first one I picked was related to nil receivers, because I believe it’s a very common one. Many of us have already experienced it, and if you don’t, at least I hope that it will prevent you from doing it. It’s a mistake that I have already seen on a project I was working on, and basically it was something related to returning a nil receiver.

So let’s take here as an example - I have a concrete example, where we can say “I am going to implement a validate function, for example, that validates an HTTP request”, or something like this. And we don’t want to return only the first error we met. We would for it to both convey that the validation error – like, convey multiple errors.

So there are different schools of thought here, and one of them being for example to either return a slice of errors, or just return a custom error type. And here we will take the example of returning a custom error type.

We can create for example a custom struct called MultiError, let’s say, that can contain some different fields and everything, and more importantly, at some point it contains a slice of errors that we want to kind of mutate throughout the errors.

So we take this multi-error struct, we make it implementing the error interface, and it’s a pointer receiver, because the slice needs to be mutated. So in our validate function, what one guy could do in that case is to say “First I’m going to create a variable called ‘result’. It’s going to be a pointer to multi-error, so I write var result *MultiError and by default, result is going to be assigned to nil, as the zero value of the pointer is nil.

So let’s say that we implement the wall function, we do the sanity checks, and if there’s an error, we mutate results to append a new error. And eventually, at the end of the function we say “I am going to return the result variable.” So in the end, result can be either nil if we face no errors, or it can be a pointer referencing a multiError [unintelligible 00:07:48.26]

[07:52] So let’s now say that we implement the consumer side, we called our validation function and we check whether the error is nil or not. And here - surprise, the error is never nil, actually. Even when we faced no errors, the error itself that we return is never nil. So what happened in that case? We have said that eventually we were returning the result variable. So if there is no error, actually what we are going to return is a nil receiver, not a nil value directly. And as the return type was an interface, because we returned an error interface, we didn’t return a nil value here; we returned an interface implemented by a nil receiver… Which is actually different from nil, and that’s why the check by the consumer is never nil. It’s because in Go a nil receiver is allowed, which might sound a bit odd at first, but it’s actually allowed. And the reason is because a method in Go - it’s just some kind of syntactic sugar, just like if the receiver was actually the first argument of a function. And it’s actually allowed to pass nil arguments for pointers, right?

So a nil receiver is completely allowed. And if we want to fix it in that very case, instead of possibly returning a nil receiver, we should return a nil value directly. So eventually, in the end, we could do “if result == nil { return nil }” instead of returning result. And it’s actually the same on the consumer side, on the other side - if a function accepts an interface, and we pass to it a nil receiver, not a nil value, the variable assigned to this interface won’t be nil.

So just as a small conclusion here, when a function accepts a returning interface, and that we pass our return a nil receiver, the variable assigned to this interface will never be nil. And in general, actually, having a nil receiver is probably something we never want in Go, and it means a probably bug, so it should be avoided.

Yes, so that’s such an interesting one… How common is this? Because – first of all, if that happened to you, you’d be very surprised, I think, by that. Johnny, have you ever run into this issue?

Perhaps it’s in my general approach - I tend not to have custom error types that basically themselves contain multiple errors, kind of like the example Teiva has illustrated. I tend to prefer a slice of errors over a custom type containing multiple errors. So I haven’t run across this particular issue as laid out, but I can absolutely see myself easily stepping on that landmine as he explained it, because it would be a natural thing. I wouldn’t be thinking “Oh, a nil value that’s supposed to point to another type that just happens to be nil.” So I’m not sort of slicing it that thinly, I guess.

Yeah… I’ve written code that allowed for methods to be called on the nil version of something, and it was just the kind of – usually it’s a no-op, or some default behavior… But is that too magic, do you think, these days for Go, Mark?

Yeah, I think so. I think that’s definitely way too magic. I agree with Johnny. I’m well aware of this problem; we talk about it when we’re teaching at Gopher Guides. But it’s not something like – Johnny, I think the way I write my code, I’ve never hit this particular area, and I almost never use custom error types. When I do, I use a slice of errors, like Johnny kind of talked about… I don’t know, I kind of write my code so that I never hit that.

Francesc did a really good – I think it was a GopherCon talk, about when nil is a nil, on this very same subject… So it does come up. And it’s a really good talk, obviously. It does happen all the time to people, and it’s a super-easy, very easy way to find a bug in your code that will take you three, four hours easy to track down.

I’m curious, what is the antidote? What is the advice that you present in your book, Teiva? Is it to avoid these kinds of situations, or is there a pattern, an approach where you can still use that custom type –

[12:17] …return an explicit nil?

[laughs] That’s one way, yeah.

Well, in that specific case, of course, the solution would be to return nil instead of a pointer receiver… But in general – okay, we discussed about returning an error and saying “Okay, perhaps I’m not going to create a custom error type here”, but we may still have the case where we return an interface. I mean, in general – okay, in Go we tend to say we shouldn’t return an interface, right? We tend to take rather the approach to say “I’m going to accept abstractions, so accept interfaces, and rather return concrete types.” Kind of be liberal with what we accept, and we don’t want to force our function to force abstractions, right? But I believe that it’s not a hard rule, because sometimes even you can see from the standard library directly some functions returning directly an io.reader or io.writer directly, instead of the type itself, of a struct.

So I believe that we may still face the case where we have at some point to return an interface, and in that case we still have to bear in mind that nil isn’t exactly the same thing as a nil receiver. So it’s still something I believe that we can face.

Yeah. And why do you think this happened? Is this just something that emerged? Do you think this was a known thing when they were designing the language?

Well, I believe it happened, again, because it’s in method [unintelligible 00:13:45.17] just some syntactic sugar, as I said… But I don’t have more insights on this one.

Yeah, I don’t it was a concrete decision to have it do that, it was just more a matter of it fell out of the type system that way.

Consequence, yeah.

From my understanding of the issue it was just that. It was something they obviously would have loved to have avoided, because it is a bug, obviously, that hits people, and a weird bug, but it was just more – it felt out of the type system and it wasn’t necessarily designed to…

[14:25] Yeah. I never thought I’d say this, Mark, but you should be a little bit louder. Move a little closer to the mic, please…

I should be a little louder… I don’t know why you don’t think I’m loud.

Well, that’s just what I’m hearing…

Well, maybe turn yourself up, did you ever think of that?

[laughs]

I don’t think that solves the problem, I think that makes it worse, but that’s fine…

Here we go, here we go…

I’ve got a common mistake in Go, working with Mat Ryer. [laughter]

And that’s from the president–

I was gonna save that for the unpopular opinions, but I think it’s a popular one, so… [laughter] So far only David Hernandez seems to be able to stick around and hang in there…

Multiple times, too.

Don’t use his name in vein.

Multiple times! I don’t. I love David. He deserves all that he gets from putting up with you.

I love it. I’ve never had a banter happening and insults concurrently, from both Johnny and Mark at the same time. [laughter]

We try, we try…

Speaking of concurrency, here’s another one… Teiva, you talk about concurrency like – we feel like Go is a bit famous for concurrency, it’s got some great primitives that makes that so much easier to do than previous languages… Should you always strive to use that, to make things concurrent, do you think?

That’s a nice transition… Thanks for that. [laughs]

I’ve been professional.

I don’t think I understand the question.

Is concurrency always faster?

Yeah.

That’s the question?

That’s the question.

No. Next question. [laughter]

This is not a quiz show, Bates…

Okay, fine… Continue on with your line of questioning. I mean, it was a yes or no question. Is it faster? No, it’s not always faster.

Is it a common mistake that people make, that people think concurrency is always faster, Teiva?

Did you just call–

[16:12] Teiva, you have to speak before Bates gets in, otherwise [unintelligible 00:16:13.04]

I keep seeing the setup, and I’m like “Teiva, jump in! Jump in! Jump in!”

Why am I even on the show…?! [laughter]

Okay, cool. Yes, so I believe definitely it’s the case… I believe that a common misconception from junior or even mid-level developers is to believe that a concurrent solution should always be faster than a sequential one… Because if we implement a concurrent solution that leverages multiple CPU cores, it should obviously be faster than a sequential implementation, right?

Yeah.

That’s a false affirmation… [laughter]

Wrong…

[laughs] Now, I believe actually that it’s even more of a belief in Go thanks to goroutines, as you have said, Mat… Because goroutines compared to threads are great, they are more lightweight, they are faster to spin up, they are faster to contexts-switch, and so on… So there shouldn’t be any real reason for a concurrent application to be slower than a sequential application.

I took here a concrete example, actually… I took as an example the merge sort algorithm. And just as a quick reminder about what the merge sort algorithm is - if we take for example the recursive implementation, we basically get a list of elements as an input, and we will break down repeatedly each sub-list into sub-lists, into two halves. We are going to do it repeatedly. And once we reach sub-lists of a single element, we go up again and we merge the two sub-lists in a sorting manner.

A quick example, if we have 2 and 1, for example, we are going to split it into two halves, two on one side, one on the other side, and as each sub-list contains a single element, meaning it’s already sorted, then we are going to merge it in a sorting manner, so we will have one and two.

So in a nutshell, the merge sort algorithm - we just get a slice as an input, we check the length of the slice, if it’s bigger than one, we compute the middle, we apply merge sort on the first half, merge sort on the second half, and then we merge.

So the structure, for example, for this algorithm seems like a perfect fit for concurrency, because we could say every time I can handle each half into a specific goroutine. So the first half in one goroutine, the second in another goroutine, and say I will introduce some sort of synchronization at some point to wait for both goroutines.

So if we implement this parallel version of the algorithm, I run it on my local computer with a certain number of elements, and actually this parallel version is about ten times slower than the sequential version. And despite the fact that the parallel version leverages multiple cores, right? So it’s more than ten times slower. And what is the reason for that, if we think a bit about it? As we said, the algorithm is about to repeatedly split lists into two sub-lists; so at some point we will have 1,024 elements, then 512, then 256 and so on, until we reach 8, 4, 2 and 1 elements.

Now let’s try to imagine, in your opinion, what’s the fastest between spinning up two goroutines that will both merge two elements and wait for them, or in the current goroutine merge two elements and then merge two other elements? And of course, it’s gonna be the latter here, right? Because it’s gonna be faster to do it in the current goroutine.

[20:10] And if we think about it actually, in the merge sort algorithm, the deeper we go, the less efficient it will be to spin up a goroutine. And sure, goroutines are fast, but spinning up a new goroutine - it has a cost, because we have to wait for its creation, we have to wait for the internal Go scheduler to execute it, we have also the fact that concurrency introduces some form of synchronization because of mutex, or channels, or whatever… So everything has a cost, right?

Here one possible solution for this algorithm - the goal is not, of course, to design the most optimal solution for the merge sort algorithm, but discuss about a potential solution. It could be to say I will define a threshold, and I will apply the parallel algorithm that we’ve just described, but if the number of elements is below a certain threshold, it’s simply not worth spinning up new goroutines. So instead, I am going to execute sequentially. And this threshold may depend on the machine, and everything. On my side it was about 2048 elements. And if I run this new hybrid version, let’s say, of the parallel algorithm, it’s about 40% faster this time compared to the sequential implementation.

And one very last thing to say - I have done the same test in Java actually, where we don’t have the principle of goroutines (we just have threads here) and the threshold actually was higher. It was around four times bigger, if I recall correctly, compared to goroutines. So it’s kind of interesting, because somehow it shows that goroutines are actually somehow more efficient than threads for concurrent workloads, because they are for example faster to spin up… But as we illustrated with the merge sort algorithm, it’s not magic nonetheless; concurrency isn’t always faster.

Break: [22:19]

So a couple of things there… One is when you’re sharing data, when you’re working on the same data, you have to be safe concurrently, so you have to then log usually, and things are gonna be waiting. So that’s always something to consider. And then I guess, Johnny, you benchmark things, don’t you? That’s how you find out these little [unintelligible 00:24:36.23]

Yeah, exactly. That was gonna be my – again, in the same spirit of my previous question, like “How do you find out what the right threshold is?” Other than maybe benchmarking, doing some profiling work and figuring out “Okay, where is the diminishing return? Where is that point?”

Yeah.

It’s interesting… We’re talking about concurrency and we talk about how sometimes concurrency is not faster, and this whole kind of idea… One of the biggest things I’ve noticed with newbies and anybody coming to Go really, from any language, is this “Let’s make everything concurrent, because we can.” And you’re demonstrating very empirically “Hey, if you do that in the wrong places, you’re actually gonna hurt your app.” But I see it a lot, not even from the numbers perspective, just from the architecture and the complexity, that I see a lot of people introduce it into their apps when they just have no need for it… And one of those is channels. They build in a lot of channel usage early on, because they think they should… And then you’re like “Oh, actually you only use channels in these three occasions…”

So concurrency in Go – what drove us all to Go originally, I think… I know it certainly drove Johnny and I, being Rubyists before this… It drove us here, it drove everybody else here. But then to sit there and say “Hang on… Just because we’ve got concurrency doesn’t mean you should always use it.” That’s a different thing altogether.

Yeah. I think that’s a good point. I was gonna say also that the complexity thing - that applies actually across the board to any of these little cool features, particular if you’re doing it for optimizations, you’re doing it for performance reasons. In most cases, you’re gonna make your code more complicated and therefore less maintainable… So you just have to weight that cost. Maybe it’s worth it in your case… And maybe it’s not.

I often find when I’m building apps I start serially… You know, because I start in the morning…

Yeah, [unintelligible 00:26:41.12]

No? Too many dad jokes? Okay, I’ll skip that one. [laughter] Anyway… That one didn’t work. That one’s out of my repertoire.

It’s not a dad joke, it’s a serial/cereal killer joke. [laughter]

It was that, too. I start very serially. I’m just gonna start my program and I’m gonna call my methods and my functions as need be, and I’m gonna find as I’m working and as I’m using this app that I’m struggling all of a sudden here or there. I go in, I benchmark, or I profile, and I say “Oh, you know what - this bit of code here would really benefit from being concurrent.” But not the whole thing, just this section, where there’s a lot of process, a lot of stuff happening, and we can get a big win there. But I only do that once I know that I’m getting that from the testing and the benchmarking.

Yeah, I think that’s great advice. Chris James from the Gopher Slack channel GoTimeFM - and by the way…

Long-time listener, first-time caller…

[27:47] And by the way, please join in the chat. That’s where we hang out. Chris has a question for the ladz (with a z). Chris says “One of the strengths often touted about Go is that because it’s very simple, it’s more difficult to make mistakes compared to other general-purpose languages. But as we are learning, there’s at least a hundred mistakes you can make, if not more. Does the panel have any ideas on changes to the language that could/should be made to reduce developer mistakes?” And kind of shadowing springs to mind for me on this one. I feel like if –

Shadowing, definitely. The magic okay is another one… For those of you who kind of know what I’m talking about - you know, you kind of get the map, map returns this random okay, boolean second variable if you’re asking for it. Or if you’re doing a type assertion. If you ask for the second random option value that’s there - these things I don’t even think are documented; you just kind of know that they’re there… And none of us mere mortals can add them; the Go team can add them in weird places… Like, why isn’t type assertion – and that’s one of my big ones; why does that not force you–

Check if it’s okay or not.

…to check the okay? Because the alternative is a panic. And I see that all the time. I see type assertions constantly in the code without that okay check, and it’s just panics waiting to happen. And if I want a panic, I can panic, even with the okay. But I also think panics shouldn’t exist, but that’s another language change I think that could solve a lot of problems.

And do you think that generics is going to be a nice use case for that, instead of having to cast every time that you will have to handle a generic set of data, or something like this?

In some cases…

Generics…

But the thing is - if you’re dealing with data that comes in from JSON or something, because sometimes you have generic data, so you don’t have structs to demarshal it into; you have maps of string interface. And then if there’s an object inside an object, that becomes a map string interface, too. So in that case you have to rely on that. So in a way – in some cases probably not, but… I don’t know, what were you gonna say, Mark? Do you like generics?

[30:05] What I was gonna say is I grew up – my first job I was writing Java, we didn’t have generics. Then I went to Ruby and we didn’t have generics.

How old are you? No, carry on… [laughter]

I’m old. I’m old. My first job out of college I actually was doing ASP, so I kind of lied; I skipped a whole–

Classic ASP?

Classic ASP.

Wow. You go way back.

Bates’ ASP was classic. Full of banter in the comments…

It was. It was as classic ASP as it could get.

…funny variable names… You had a blast. Absolutely classic.

Good times. Good times.

Now I can’t even remember what I was talking about.

Generics.

Oh, generics. Thank you. So I’ve never worked in a language that has had generics to date, kind of built-in as the term generics is known throughout the community. I have as of yet run into one area in particular where I’m like “Ah, generics would have been perfect there.” I’ve been coding for over 20-something years professionally. So I think I like the idea of what generics can possibly do for the language; I’m also afraid of everything they can do to the language. [laughter]

Okay. Well, we’re gonna find out together – and don’t worry if you’re scared by generics. Go Time is your friend…

Well, because I can see performance problems… Because we were just talking about people coming to Go and abusing concurrency. And let’s be honest, that’s what we were talking about. We were being very nice about it calling it a mistake, but it’s abusing the technology, right? And we all did it when we first came. New people come and they do it. And that’s the mistake, is we keep abusing it, right? But I see that mistake about to happen again with generics, and I can see everybody rushing to implement everything in generics, and I can see programs crawling to a halt because everything’s super-slow.

[32:07] And tools, parsing things and working in all these different ways in Go are gonna be not supported or working incorrectly because of generics… I just see a lot of potential mistakes (if we’re gonna keep using that term) coming down the pipe here.

Yeah.

I take a more optimistic view… [laughter] I mean, I do see the same dangers Mark sees. I guess I’m choosing to look beyond what happens after the initial spike, the abuse.

Yeah, but that’s gonna be the first couple years, and those couple years are gonna stick with us for 5 or 10 years.

Yeah, I mean – there is some of that.

Still optimistic, Johnny? Still feeling optimistic? [laughter]

No, I really –

Climate change, Johnny? I mean…

[laughs] Look, the way things are going we won’t be around maybe to see the change after the spike.

That’s a good point. I shouldn’t worry about generics, because climate change is gonna take care of the problem.

It’s gonna take care of everything.

I’m still worried about modules. I don’t have time to worry about generics.

[laughs] Wasn’t that episode on that last time?

Yeah.

I think we hit that, right?

Yeah.

They don’t let me on those episodes anymore.

Johnny, are there are other kind of common mistakes that you’ve seen?

Well, I can give you one from personal experience which actually dovetails quite nicely with the very thing we were talking about, with the abuse of concurrency. So when I first came to Go, and my previous language before that was Ruby, and - basically, I was like “Oh, this is so easy to do concurrent stuff.” So I started sprinkling it on everything. And I was like, “Oh, channels. How do I use those?” For those who don’t know, you don’t have to use channels to use Go concurrency, right? They’re a communication medium, a synchronization medium. So you can have simpler mechanisms, like a WaitGroup, or things like that. So you don’t have to use–

In all fairness - I don’t mean to interrupt Johnny, but I’m going to…

[laughs] Cool. Go right ahead.

Early in the days of concurrency we only had channels. We didn’t have a lot of the other primitives for controlling things like WaitGroups, errgroups…

Those came after, yeah. Same package…

…contexts, stuff like that. So all of the early documentation and all those early blog posts and videos, everybody’s like “You’ve gotta use channels to start and stop goroutines, and keep them in sync, and protect mutexes” and whatever.

[unintelligible 00:34:29.25]

That’s his grumpy old man voice… [laughs]

My normal voice is my grumpy old man voice… That was my grumpier old man voice.

As he climbs in age, everybody… [laughter]

That’s what I’m gonna sound like in five years.

Right, right. No, so the personal experience is one where I was actually in charge of building this batch job thing, where I’m communicating with some remote endpoint, and making multiple API requests… And for some reason, there was no batch API endpoint to hit to give you a bunch of stuff. You kind of had to make singular requests, if you will. So I’m like “Yay! Goroutines are gonna be great for that.” So basically, I do testing; not always test-driven development, but I always test my code… There’s a difference, and we’ll get into that later…

[laughs]

But I write my mocks, I have my stubs locally and everything else, I’m sort of faking the API response, I’m throwing goroutines at it… Everything’s going perfectly. Everything just works. And then I put the thing into production, and then all of a sudden those hundreds of goroutines, perhaps even thousands of goroutines that I was launching during testing to hit this mock endpoint [unintelligible 00:35:38.21] I start seeing a bunch of 429’s in HTTP status code in my logs. I’m like “What’s going on? What’s going on?” For those who don’t know HTTP status codes, 429 means basically you’re making too many requests. The server on the other side is saying “No mas. I’m not gonna bother responding to you because you’re simply abusing this API, basically.”

Oh, your code was too good.

[35:59] [laughs] Well, perhaps not that sensibly written, again…

So why didn’t you put a sleep in before each one? A random sleep… [laughter]

That fixes it, doesn’t it? Doesn’t that solve the problem.

It fixes it, yeah. [laughs] It was one of those things where I’m like “Yes, perhaps too much concurrency will bite you.” So you have to factor in – which is something I also teach when I’m talking about Go concurrency… You have to factor in the constraints of your system. Go is not gonna be a bottleneck. The language is not gonna be the thing that makes your stuff go slow. Your databases are gonna be bottlenecks. The network is gonna be a bottleneck. The third-party endpoint you have to talk to over the wire - that’s gonna be a bottleneck. Not the language.

So factor in the constraints that you have to work with. What problem are you solving, what are the constraints that exist within that problem domain? The language itself - that’s not your problem.

We teach the same thing at Gopher Guides. And that’s one of the nice things about Go, is for 98% or even higher percent of us, the language has our back in terms of performance and just making sure we’re not shooting ourselves too badly in the foot… And it is - it’s the database, it’s the network, it’s the file system. Those are the things that are gonna make your app just crawl, crawl, crawl, crawl. It’s not gonna be that bit of business logic you have invalidating a struct to make sure that there’s an email address, [unintelligible 0:37:29.21] All that sort of stuff -it’s the network calls, it’s your database… Which is just a network call onto itself. That’s all a database is. Or you’re calling the file system, which is even worse than the network half the time.

So yeah, those are all real issues that I think people just kind of get in their heads and they think they need to fix it in the language, and not kind of look at what they’re programming and what they’re coding and say “Okay, let me focus on that and find those performance issues later, when they become that problem.” I think this is the second time we’ve come back to that kind of concept…

Yeah, it is an important one. It comes up a lot. It’s worth bearing in mind as you’re underway, as you’re writing code. So Teiva, there’s also another one about time after… This is from the book. Time after - you could possible have a memory leak?

Yeah. Well, not really memory leak per se, but yeah, having a peak of resource consumption in some specific cases, let’s say. So just as a reminder time.after accepts as an input time.duration, and it returns a channel of time.time. And it waits for the duration to elapse, and then sends the current time on the return channel. And also the time it’s also used in concurrent applications, let’s say, because of the [unintelligible 00:38:54.08] Otherwise, if we just want to wait some time in the current goroutine, we can just use time.sleep.

So let’s imagine we want to implement a kind of smart consumer, or I don’t know how to call it… That will keep receiving messages from a channel. But let’s say if we don’t receive any messages for, let’s say, one hour, we also want to log a message, to log an alert, for example, or something like this. And one way we could implement it is using time.after in this case, and we could say “I will have a for loop with a select inside and two cases.” In the first case I’m going to receive messages from the channel, and then call a function, do whatever with the message we have received, and in the second case we are going to receive on the channel that is returned by time.after to log the alert “I haven’t received a message since one hour.”

[39:59] So in that case, let’s say, we just deploy our application, we keep receiving messages… That’s great, but we notice that it consumes more and more memory and it seems to steadily increase, actually. So what’s the reason actually for that? We have to know that in every iteration Go will actually evaluate time.after, and it will create a new channel. And we may actually expect this channel to be closed in every iteration, but it’s not the case. Actually, the resources that are created by time.after, including the channel itself - they will be released once the timeout expires.

So in the worst-case scenario, where we keep receiving a high volume of messages, we will keep looping, we will repeatedly call time.after in each iteraction, and it will keep creating resources until the duration - in our case one hour - elapses, basically.

Yeah. Now, I have definitely made this mistake, doing exactly that–

I’ll be right back. I’ve gotta go fix something… [laughter]

Yeah. So the answer must be to create the channel outside? What’s the answer, how do you avoid that?

There are different solutions. One of them could be to say “I can use context.withtimeout”, but it’s not perfect nonetheless, because it would mean that in every iteration I have to call context.withtimeout, and it’s going to create a channel; it’s not lightweight. Here, the most appropriate solution could be to use within the time package the timer searcher. The timer searcher basically - we can create it with timer.NewTimer, where we’d pass a time duration, and internally it has a channel field that we can access [unintelligible 00:41:49.20] and it has also a reset method. And the reset method - that’s going to reset basically the duration.

So what we could do in our case - we could say “Before [unintelligible 00:42:03.11] I’m going to create this timer.” In the second case I’m going to listen on the timer.seeField, which is the channel… And then in every loop iteration I’m going to call time.reset to reset the duration.

And here it will just reuse the same channel, basically. So it won’t have to create a new channel. So yeah, that’s probably the most appropriate solution here.

So it is possible to use time.after in a safe way then, or should we just avoid it altogether?

I believe if we are in the context of a function that keeps being called - it’s in a loop, for example; or we can have also the case in an HTTP handler, right? Because in an HTTP handler if we call time.after and we keep calling this HTTP handler, we may have the same case. So my recommendation here is probably to not use time.after when we are in the context of a function being called repeatedly.

Yeah, that’s a good one.

Nice.

Yeah, that is a good one.

Very good one, yeah.

Break: [43:14]

Bates, you were talking earlier about another common gotcha…

I was…

…to do with the API footprint that you export.

Oh. And by talking about that earlier you mean in the private message I sent you this morning…

Yes…

…completely offline and out of context of this call.

Yeah, that’s true. You weren’t talking about it earlier on this podcast.

Yeah, yeah. That’s right.

It’s just a mistake. It’s part of the theme of the show. I’m gonna be making lots of mistakes…

I love it! I love it.

No, please [unintelligible 00:45:03.24]

So yeah, Mat obviously asked us all to submit common mistakes that we see a lot…

Well, it’s rude telling them how we do it… Just do the thing, don’t tell them how we’ve done it. [laughter]

Mate, it’s not that big of a deal… The sausage is being made here, right?

It’s a mistake, yeah.

That’s the problem though - I want them to think this show is way more sophisticated than just me texting you while you’re having your cereal.

Well, that would be his mistake now, wouldn’t it?

Well… Exactly. Anyway, one of the things I see new developers to Go do a lot is basically export everything. Everything starts with a capital letter, and that’s it. Like, their entire package, all of their types, every field in their type, every method on their types… Everything is public. And of course, when you do that, you can’t take that back now. That’s it. It’s out there, as soon as people start hooking into it. And you know, modules have made this problem a little more complicated in some areas…

But it’s definitely one of the big issues I see, where people should be starting with everything unexported, and only pushing up those pieces that they need… I think it’s a lot to do with just people misunderstanding the way Go does exporting and unexporting. Because it’s different than every other language I’ve used, this whole idea of capitalization to infer whether it’s public or private. I love it now. Once I got it, I’m like “Oh, that’s brilliant”, because I can look at anything in my system and immediately know whether it’s visible or not. I’m not scrolling up to look for a private or a protected method above it. But at the same point, I think new people come to a language and they see things that are upper-case and whatever, and they think “I’ll just upper-case everything, because it’s a proper word.” It’s the user’s struct, so therefore u must be a capital u.

So would you say that people should just by default use lower-case letters everywhere, until you then need to export it for some reason?

Yeah. I mean, you’d have to sell me on why you would wanna do it the other way around. I mean, there are definitely types when I start – I open up a package and I go “This type has to be exported.”

Because you’re thinking about the use of that package…

The actual API.

Because I know what the use of this package is… So I’ll say “Okay, my user type I need to export, because I need other people to use it.” But the fields may or may not be fully exported. Or the methods might all start as unexported methods until I need one to be promoted; I needed to use it outside the package. And again, it’s like “Well, do I need that method, or do I need something similar to that method [unintelligible 00:48:01.02] or somewhere else?”

You know what you just made me think of? It wasn’t a deliberate choice…

I’m glad this is coming out of your mind and not mine… Go on.

[48:12] [laughs] Yeah… Now that I think about it, it wasn’t a deliberate choice. It was more of a – maybe I’m adopting some of Mark’s grumpy old dude sort of a persona, but I was like “My interfaces inside of my packages - I don’t export those, at all.” Because I don’t want the consumer of that package to have a dependency on my own abstractions, on my own interfaces. So it’s almost like I say “Well, if you wanna create abstractions or whatever it is, you create your own interface. You create a local interface to you, for whatever I’m gonna hand over to you, but I don’t tend to expose my interfaces for my packages.” It wasn’t deliberate, it just kind of happened.

Oh, amazing.

I do expose a decent amount of interfaces, but I also write a lot of local, unexported interfaces inside of another function or method sometimes…

In-line, yeah.

Yeah, just in-line, because I wanna check “Does it have this method on it?” and if so, I wanna call it. And I don’t need some big interface somewhere else, I can just do it right there in-line. And that’s one of the nice things about Go, is that you can do that sort of thing. But that’s an advanced feature, Johnny. That’s a feature that – I don’t know about you, but when I teach… You know, I teach a lot of intro to Go at Gopher Guides, and interfaces is one of those things that people really struggle with coming to Go.

It surprises me, because they’re on the surface relatively easy. There’s not a lot to them; it’s a collection of methods, and you either implement those methods or you don’t. But people really struggle with that and they struggle with the implicit versus explicit declaration of an interface… I’m not implementing.. this isn’t my Foo interface… And then they certainly don’t realize that you can create in-line, unexported types and interfaces inside of a method to make your life even that much more easy right there.

Right, right.

Yeah, one downside to that technique is you can’t hide things inside the function. It’s not always obvious – you have to use docs or something to say “If it implements this interface, then it’ll have this different behavior.” So it is quite an advanced case, I think..

It’s very much so. I use it mostly in an exception case, where I’ve got an error or something, and I wanna try to see if I can inspect a bit more information out of this thing, if I can… Otherwise, if I’m letting people know that I’m supporting these methods via interfaces, then I’ll expose those interfaces, more for the sake of hanging documentation on them. You know, just saying like “Hey, this method is gonna take these five interfaces”, so it’s a documentation thing versus a required to be passed in.

Well, it’s that time again… Teiva, hold this base. Johnny, you’re on the drums. Bates, pick up the guitar… It’s time for Unpopular Opinions!

Jingle: [51:13] to [51:30]

Who’s got a meaty unpopular opinion?

I do. I’ll fire the first salvo.

Okay… [laughs]

That came out of nowhere. You look surprised.

Yeah, I was surprised.

Yeah, I don’t often have unpopular opinions.

Yeah, I know. Because you’re nice. That’s what we were saying earlier, Mark, privately, in the text.

That’s what we were saying earlier, yes. We were saying what a nice guy Johnny was.

[laughs]

Don’t you think Mark Bates looks like he’s just got back from being shipwrecked?

Oh, mate. Fire!

[52:04] [Captain Sparrow voice] “I’ve been stuck on that island for so long… Thinking about generics and stewing away…” I’m like [unintelligible 00:52:10.25] from Lost. Somehow I gained weight being stranded on a desert island. I’m not quite sure how that happens.

No, you look great though, mate, really. [unintelligible 00:52:17.19]

Oh, thank you. That’s your unpopular opinion, I assume.

[unintelligible 00:52:20.09]

Then I look good. Because there’s definitely an unpopular opinion in my house.

You definitely make a good Guess Who character, from that game Guess Who. Like, way more interesting.

Fair enough.

Can I opine now?

Yes, Johnny, I’m sorry.

Yeah, go for it. Please, help us.

Okay. [laughs] I have to break this up. Alright… Yeah. And for those who don’t know, this is actually what happens in real-life too, hanging out with these guys.

Apart from I began to piggyback off Bates by now…

That’s true. And there’s usually a lot more food involved.

And alcohol, yeah. Sadly. Anyways, so my –

Still waiting on you, Johnny…

I know, I’m trying to get it out and y’all keep making me laugh. [laughter] Alright, so here it is. I think everybody coming to Go should make as many mistakes as possible. Because that’s how you learn. That’s how you learn.

I don’t think that’s an unpopular opinion.

Well, I’ve never heard that…

I’m gonna disagree with you.

What?!

Because I think the adage of manage mistakes – like, people know that you’re supposed to learn from your mistakes. Your mistakes are good. So don’t go and purposefully make mistakes. I don’t know how you’d even do that, because they’re not mistakes; then you’re just breaking your code, because you’re like “Johnny said I should make my code terrible… I should introduce as many bugs as possible if I’m gonna become a better Go developer.” That’s not I think what Johnny was trying to say, I hope…?

Right, right.

Because that really would be an unpopular opinion. “Johnny says do a terrible job at work and you’ll become a better Go developer.”

“Break all the things.” No, no, no. So here’s some nuance… I don’t know about y’all, but when I’m learning something, like maybe on the job, or if I know that at some point somebody’s gonna look at whatever it is that I’m trying to do - I go into this sort of learning paralysis where I just keep learning and learning and learning and reading and reading and watching and watching, just consuming everything I possibly can, so that the first piece of code I write has to be perfect. Because I’m so sensitive to the criticism. Over time I’ve learned to sort of feel less of that pain or feeling of inadequacy or whatnot, but I will spend a ton of time just becoming an expert at something without actual practical knowledge that actually does make you an expert.

So don’t be in a learning paralysis mode. Put stuff out there. Make mistakes, talk about it… I mean, you will find a jerk that says “Hey, this stuff is crap.” That’s fine. Brush it off. Most people are not like that. So don’t be paralyzed. Just make mistakes in public, learn in public. That’s okay.

And that also extends, like, we should be forgiving as well when we see people making mistakes…

Right, right.

When you give that feedback, do that in a way–

Yeah, don’t be that jerk.

Yeah.

We all do it. Johnny, I think you and I have very different coding styles… Because you said “I wanna learn something new, and what I’ll do is I’ll go and I’ll find something that says “This is how you get up and started to Hello World”, and that’s usually where I drop off, and I just start coding, and start hacking, and throwing things in, like “Well, that didn’t work. What the hell?!” Personally, that’s the way I code… When I’m learning something new anyway. If it’s a brand new tool, a package, a library, some concept I’ve never done before, I just write code. I can’t just read and read and read. I’ve gotta just bang it out until I get to a point where I go “Oh, that worked. That worked. Yay!”

Now, how do I make that clean and good, and do what I actually wanted to do, not just “I got it to make a connection and return Hi to me.” You know, like, “Great. Now let’s take this and try to build on it.”

[56:05] Yeah, you can’t really – you learn so much by building the real thing… And this also speaks to software engineering generally. We wanna design it all upfront sometimes; we wanna know exactly what it’s gonna be, so we can do good estimates and things… You learn so much by doing it, so get on and do it. You’ll just make so much more ground, and making mistakes in that sense is great, because that really is how you find out what the real thing to do is. Sometimes it’s impossible to know upfront, isn’t it?

[56:43] Yeah. I would say 9 out of 10 times I run tests, they fail. And I’m okay with that, most of the time. There are times when I’m like “Why the hell are you failing?!” But most of the times it’s like “Yeah, okay… Still not there yet. That didn’t work. What can I try now to get this test to pass?” You know, just keep making those mistakes till you find it, and that’s okay.

Yeah, absolutely. Well, speaking of mistakes, Teiva’s book, “100 Mistakes” is –

Are you saying that his book is a mistake?!

Goodness! We treat our guests better than that.

Mat, you’re doing a terrible job… I think it’s a wonderful book, despite what you say…

Teiva, don’t believe what he says.

Yeah. Teiva, man, Mat’s just – he’s jealous.

Yeah. Great book, great work… Your stuff is not a mistake.

That was rude.

He’s jealous. His Blueprints book hasn’t sold well in years… [laughter]

[unintelligible 00:57:24.23] It’s absolutely full of mistakes.

Yeah, that’s why it hasn’t sold well in years, mate.

Well, you can find out for yourself by buying my book, and also getting a – Bates, let’s try this.

You can go to my torrent site and just download his book for free.

[laughs]

That’s how we treat friends. We just put their stuff–

I’ll put the link up on Twitter after this show… [laughter]

Good. Thank you for putting it there, so nobody will see it.

Don’t mention it.

Bates, pick a number between 34 and 36.

Oh, I hate the way this works. 42.

No, a number between 34 and 36.

I was told there wouldn’t be –

Maths? It’s not really maths, is it? 34 and 36. There’s only one option, mate.

35! Okay, 35.

Okay, you said it. 35. We’re gonna give a 35% discount to anybody that wants to buy Teiva’s book using the secret code “podgotime21”. We’re also gonna be giving a free copy away as well, Teiva. This is very nice of you to do this. So if you can tell me what title did Mark Bates give himself by mistake earlier in this episode. It’s “The *something* of mistakes”, he gave himself this very grandiose title, as we’ve come to expect from Mr. Bates… If you can tweet what that is, we’ll pick one over the next week. Tweet at @GoTimeFM the answer to that, and we’ll send you a secret code and you can get a full eBook of 100 Go Mistakes and How to Avoid Them. That’s good, ain’t it?

I like that. It is. I like that.

It’s cool.

I’d read that book.

You’re gonna try and win, aren’t you?

I may. I’m not quite sure what I called myself, actually.

Yeah, just listen back.

But I just didn’t think I’d say that on the air, so I’m surprised…

Well, we’ll see. It has to make it into the podcast episode. I’ve made it part of the format. And that’s how you do it.

It’s gotta be. It’s gotta be.

Teiva, thank you so much for joining us. Absolute pleasure to have you here and learn about mistakes… And good luck with your book; we look forward to reading it. Johnny Boursiquot - always a pleasure. And Mark Bates was also here. Goodbye!

[laughs]

Thank you!

Thank you so much.

Outro: [59:49]

Um… Okay. Brilliant stuff. Thank you again. I mean, this is it. We’ve done the podcast.

Thank you.

Editors will turn that into something that sounds really professional.

Okay.

I hope so.

Right. They’ll just cut out all the parts Mat was speaking. That’s usually how they do it. [laughter]

You nearly said Mark as well, I love that.

No, I said Mark and Mat, yeah.

Oh, yeah, yeah.

So it’s just gonna be like 45 minutes of Teiva speaking, and that’s it.

Yeah. That’s probably what people want anyway.

Honestly, that’s probably a much better podcast than what we just did.

Yeah.

Next time, Teiva, you just do it. We’ll just be quiet.

More like the audiobook version of your book. You could just read it, with your accent. That’s the thing - people love accents.

That’s cool.

We’re delivering.

You know, Mat and I could do the audio reading of your book… [laughter]

Say no, Teiva. Say no. Say no. Say no.

…with different voices. And we’d do it for a reasonable rate. Well, one of us would take the narrative and one of us would take the code.

Yeah, yeah.

So one would be the voice of the code–

Have you ever heard Mat do the filepath walk? [laughs]

Yeah, that was my idea, mister…

[laughs] Okay…

Yeah, but would you read the code in like a Stephen Hawking voice? Because it’s kind of fixed width font…

Well, who says I’m reading the code? Why can’t you? I wanna do the narrative.

Okay. Yeah, you’ve got the sort of bouncy kind of voice…

You’ve gotta keep the people interested.

Yeah. And I’ll be just like, reading out the code in a boring way.

Yeah, exactly.

While having some tea.

See, Teiva, we’ve got it all sorted out. You just send us a copy of the book; we’ll get it to you in the mail by Tuesday. Oh, today is Tuesday. End of day today. We’ll do it by the end of day today. We’re that good. One take, without having read the book. We’ll just – write down.

Mistakes were made.

That would be a mistake–

Many, many mistakes were made!