Darklang Diaries

This week Jerod is joined by Paul Biggar the creator of Dark, a new way to build serverless backends. Paul shares all the details about this all-in-one language, editor, and infrastructure, why he decided to make Dark in the first place, his view on programming language design, the advantages Dark has as an integrated solution, and also why it’s source available, but NOT open source.

Discussion

2021-02-27T15:56:02Z ago

Interesting opinion about the SSPL debacle. I like what I’m hearing - not everything have to be open source and if your business model isn’t actually about working in open source (see Elastic) - which means allowing other people to compete directly with you using your source code - then choose another model, like “source available”.

That being said, I personally am put off by “source available” licenses such as the Dark Core License (great name, BTW) - if I understand their use case it is this:
Use our service and if you want to see a new feature or a bug fixed, you can -

download our source;
figure out how to build and run it (it looks like they’ve taken considerable efforts to make that easy - kudos on that)
write and test code

Then petition Dark to accept it and wait for delivery - all the while you can’t use the feature or fix you invested time and money in creating, and they might not even accept it (or it could take months).

I personally will not be onboard with that use case - for me its the same as using a completely closed source system, i.e. if its breaks for you you can either grit your teeth or leave. I was in a similar situation a few times and in every single case I eventually left - at great costs to me, so I rather not get into this in the first place..

2021-02-27T17:50:25Z ago

One of the main selling points of Dark is that you don’t have to run it yourself. Similar to Heroku, or AWS - this is a complicated system and if you had to run it yourself you’d never get anything else done. And the downside of that is that, yeah, you aren’t running it yourself and so can’t randomly apply code to the bit that someone else maintains, just like you can’t on AWS or Heroku. If it’s important to you to manage those parts of the system, you’re using the wrong product.

But of course, we have to be sensitive to fact that people will sometimes get stuck, and so we work to make sure that we respond quickly, and that there are escape hatches - in Dark’s case you can always create a separate service and talk to it over HTTP. So you don’t get stuck (and if you think, “oh, but I have to run another service to do that”, well, that’s what you’d have if you were running Dark yourself).

More importantly perhaps, is that we spend a lot with the community. I don’t think it’s a good model to do “fire and forget” PRs - talk to us before you write code and we’ll work to make sure it gets merged, so you’re not wasting your time.

2021-02-28T08:04:52Z ago

These all sound like good practices to manage concerns such as those I mentioned. Thanks for the info. Time will tell how well you execute on those.

2021-02-27T16:05:56Z ago

Regarding the “testing before deployment” issue being “solved” by the Dark Editor - I think its the epitome of the third virtue: if the syntax is correct, it is good for production - because if you understand your tools and frameworks, and the code compiles then it can’t be wrong.

Not to say I don’t hold that virtue close and dear to my heart: yesterday I deployed a modification to production without testing(*), but to say that it is the preferred way for all developers to work, regardless of what they are building - ahmmm.. not so much…

*) it was a simple refactoring plus a boolean check, and worst case the operation I made optional and off by default cannot be re-enabled by the flag I tried to put in. I know all of my users that it might affect and can coordinate with them directly if there’s an issue. The benefits on working on a commercial product that is not widely used.

2021-02-27T17:53:54Z ago

if the syntax is correct, it is good for production

This isn’t what we believe, in fact, we have totally the opposite opinion. Syntax is not enough, and Dark is designed around much more advanced tools to make instant deploys safe.

Firstly, any change you do a slow roll-out. Just because the code is available in production immediately, doesn’t mean you have to use it in production immediately. Just like in code today, you do a feature flag, and you test it in production with a small number of users (maybe just yourself!) before rolling it out widely.

Feature flags are core to Dark (though I will say that aren’t nearly as advanced as they need to be, hence still in “beta”).

2021-02-28T09:33:32Z ago

Thanks for the info - I was wondering how someone might setup a “staging” environment. Having deep integration for feature flags in production sounds like a great idea. I wonder how you make that work in customer code - the linked article didn’t go into details - but from my experience managing feature flags in standard code workflows is quite a hassle and when you have more than a few it can quickly get out of hand.
I would have loved to hear about that in the podcast 😀 please make a note to mention that on your next interview 😉.

2021-02-28T17:09:59Z ago

Super interesting, innovative idea. Love that you just code, you don’t have to stand a bazillion services and worry about maintaining them. I also like no syntax errors!

My dad also went to trinity college dublin…64 years ago!

2021-03-05T12:36:54Z ago

I’m disappointed, not for the first time, by the broad brush with which “open source” is painted on The Changelog. Specifically the Free Software perspective is missing. The Elastic case (permissively licensed code being absorbed by some corporation) is one of the core things it addresses.
Of course Dark should choose its own way, and the GPL would not suit the stated goals of the project. I just think it doesn’t hurt to present a more nuanced understanding. This helps to make people aware that the currently dominant “free beer” mindset is not the only alternative.

2021-03-05T14:14:33Z ago

I think this is a fair criticism and an excellent example of why we attach a discussion to each episode and encourage people to participate. Thanks for letting us know!

I agree that discussing the possibility of using the GPL in Elastic’s case would have strengthened that episode, but like you said it wouldn’t suit Paul’s goals with Dark.

(Inside Baseball: we recorded this episode before the Elastic episode, so when Paul brought that up in this conversation I wasn’t as steeped in the details of SSPL/etc as I became later in the month.)

Two things to end on:

We have an episode upcoming with Ben Johnson where GPL is discussed in some detail
What are some concrete steps we can take to elevate the Free Software perspective on the show?