With the number of libraries available to Go developers these days, you’d think building a CLI app was now a trivial matter. But like many things in software development, it depends. In this episode, we explore the challenges that arose during one team’s journey towards a production-ready CLI.
Wesley Beary: All of the above, kind of. So there’s a couple of different things. One of the other things that I initially picked up at my time at Heroku and now I’ve really dived into is doing more like spec-first API development. So when we’re working on API endpoints, which - for us, a lot of what we’re doing is basically like we want to add a new capability to the CLI, which relates to CRUD operations on a resource, or something… Well, how are those going to be driven? Well, there’s going to be a REST API on the other side, right? So we have our Open API spec that defines what all of the API ought to be doing.
So usually, when I’m going to develop a new endpoint… Like, I was working on a new thing earlier this week. So I start, I go into the spec, I add in - in this case it was a new operation to create organizations. Previously, if you wanted to do that, you had to just go into the web interface. Now I’m trying to add it in the CLI. So I went in, I defined it in the spec, and then right now we’re using a tool called - hopefully, I get all of these right. There’s a bunch of tools. So I believe Prism is the right one for that… So Prism is a Node-based command line tool; it relates to Open API stuff. You spin that up and you can actually say “Here is a spec that has example values in it. When I make a POST request to the organizations thing to create an organization, just use the example data from this file and return something that doesn’t necessarily quite match up with what I said, but it is a valid representation of an organization.” Because for my initial stuff that’s fine.
[00:15:49.20] So usually, while I’m developing, I’ll start with that. So I’ll be developing the spec and developing the CLI in parallel, and then I can actually build out the CLI endpoint that works just against all of these mocked data. And then that I’ve found in terms of iterating and stuff is super-valuable, because - I don’t know about other people… Even though I’ve been doing API stuff for a while, I almost never get it right the first time. And it’s pretty costly if “getting it right” the first time means writing out all of the end points, and all of the backing stuff to that, and all of the tests to that, and all of the database interactions to that, and the tables… All of that stuff - there’s a lot of stuff to that. And so if you make a mistake, it can be a real pain to fix it.
So being able to just iterate quickly against the spec directly - it’s way easier, because I can be like “Alright, great. I’ll do this. I’m banging out. I’m working on the CLI endpoint. Oh, wait… There’s two or three more fields that need to be serialized onto this record that I just forgot about. Okay, let me go add those to the spec. Great. They’re there. Okay.”
Now the CLI thing does everything it needs to. Now I can go do the implementation, and I have a clear contract that I’m basically implementing against.
So we start with that. And then in the same way when we’re doing tests, we have a lot of tests that are marked to run just in that mocked mode. And so in the test context, we spin up a Prism mock server, and we run tests against that. Not everything works that way, because - the example I gave a second ago of like create an organization, but don’t really pay attention to the parameters I pass into you, or whatever, just give me back something that’s technically valid… Like, for some tests, that’s great. That’s all you really need. But for other tests, it matters that the database really gets touched, and the valid records are there. And especially in our context of X509 stuff - I wish it weren’t this way because it can be a real pain, but sometimes to create a valid record, you’re actually creating a constellation of records. It’s actually like it’s not just this one record that is valid by itself. You need to create these six different records that all have relationships to one another, or something. And so mocking that becomes a nightmare very quickly.
At the same way we have – Prism can also be run in a proxy mode, where instead of being a mock server, it instead passes requests through, but as the requests pass through in both directions, it checks to make sure that everything that’s passing through matches against the schema that you provided. So that helps to guard us then that now we’re running tests against real, live stuff, creating real records or whatever, but if there’s discrepancies from that contract that we’ve written, we’ll find out about it. So that helps, again, to keep us honest.
And then on the server side, we have some similar tooling. There’s a – so our server in this case is all in Ruby, that runs the API, and there’s a library that’s called Committee, that also has the schema loaded into it. It’s what’s called a rack middleware. So as the requests come in, it checks the requests against the schema, and if they don’t match, it will reject it and say “Hey, you’re including a field that’s not in the schema. I don’t know what to do with that. Please, don’t include this field.” Or “This is field is the wrong type” or all these kinds of things that it can find out just from the schema.
And on the same token, as the request comes back out, it checks it again against it and says “Hey, wait a second. Actually, you included three fields that aren’t in the schema. What’s up with that?”
All of that like provides nice guardrails and helps us iterate faster. And then we don’t always do this, because we’re such a small team, but it can also be really nice in terms of being able to parallelize some of that. Like, once you have a schema that you’ve agreed upon, I can continue working on the CLI and I can potentially hand off the implementation of the API side to one of my colleagues, and we don’t even have to talk to each other or whatever. We’re not stepping on each other’s feet. We’re both implementing against the same contract. As long as the contract stays the same, we can do that without even really having to talk. That forms the talking that needs to happen.
So all of those aspects have been really nice, and definitely have helped us iterate faster. I actually – this is a whole other aside, but in some ways I wish that I had something kind of like that schema set up for CLI stuff, where I could kind of define, I don’t know, somehow what the CLI ought to eventually do or look… Because then we could work backwards from that. We could have the contract… I haven’t seen anything like that, so if anybody knows of something like that, please let me know.
It seems harder, because the ultimate output of the CLI is much more freeform. In the case of an API you’re talking about JSON blobs in and out, so it seems a lot easier to define something that says like what the shape of those blobs should be, what the types should be, stuff like that. CLI is like a bunch of characters on a screen, so what do you even do?
[00:20:12.02] But yeah, not having that can make it harder, right? There’s a lot of just like guess and check. I don’t know. I mean, the closest we got is – this is iterated over time, but we do a lot of sketching basically, where a sketch is… In the early days, my sketches were basically like I would - again, former Rubyist; still Rubyist, whatever. I would write a Ruby program that was just like a bunch of print lines and stuff basically, that more or less did something in the shape of what we wanted the CLI to do, so you could just like see it happening dynamically… Because in a lot of cases, for me at least, that would give me a much better sense of “Does this feel right? Does this feel close to right? Does something seem off here? Does it seem too noisy? Does it seem like it’s not giving enough feedback? How does this feel?” I don’t know, for me at least it’s hard to do that without a little bit of poking and prodding, some just try, guess, check etc. So yeah, that’s been super-helpful too in terms of iterating quickly.
Break: [00:21:09.01]