Programmable infrastructure

Alright Kurt, so you’re with Fly, previously with MongoHQ, which was renamed Compose, and before even that was Ars Technica, way back in the day. Take me back to Ars Technica, what you were up to there, because that’s a publication that I really admire and respect, and then tell us how you got to be co-founder of Fly.io, from journalist at Ars Technica.

I actually was always a wannabe writer. In college (1999-2000) I started writing what I could for Ars Technica, and it was just a hobby; it was a really ugly site, we were just kind of doing it for fun. The hot article at the time was like “How to overclock your dual Celeron for maximum performance in– one of them was Alien vs. Predator.” Anyway, old-school game stuff, it was cool.

Nice.

So I wrote a little bit for the site. I remember reviewing the first version of Mozilla with tabs, and I’m very proud of myself for in the review being like “I don’t know why anyone would bother to use tabs. I think this is just a fad, it’s not gonna happen in browsers…”

Oh, no… [laughter]

So I was spot on with that one, as far as I can tell.

[04:06] That reminds me of the guy from Slashdot - his initial take on the iPod was how lame it was.

Yeah, exactly.

He never lived that one down. Now, you’ve managed to escape the history of saying tabs were useless… Until now; you’ve just brought right back up.

I did. I think I hedged a little bit, because at the time Windows had just released the – they collapsed the taskbar buttons, so I was like “The OS has tabs” was kind of my general back-off… Anyway, it was funny. I’m very proud of that one. But anyway, I’m not as good a writer in terms of like actually regularly producing content as I wanted to be, so I started just working on the technical portions of Ars, like writing code for them; I did image hosting for subscribers at one point, and it was just a fun way to work on really interesting projects.

Then ultimately I ended up writing a lot of content management stuff. I tried to start a company to build the content management system, but Ars was the only customer, and that didn’t work. But I just continued working for them, and it actually became a real gig when Condé Nast acquired the company in 2008. Then I worked there as an actual employee for three years, and it was a very interesting experience. Actually, a lot of what I’m working on now is from problems and frustrations I had even 10-11 years ago at Ars. It’s kind of a fun full-circle story there.

Your last couple articles here, the Recent Stories by Kurt Mackey, as according to your author page, still on Ars Technica: “IE9 preview 6 was available now as a secret Beta UI.”

I think I was at WWDC when I was writing that one. There’s probably a picture of a scanwich - a scan sandwich. It’s my author picture, too. Is it still there?

Yes! Yes, it is.

[laughs] Scanwich.com was a fantastic blog.

That’s spectacular! Link in the show notes for people who wanna check out the scanwich. And then “Phone in Motion: messaging on Windows Phone 7.” I guess this was eight years ago, but it feels like even further, because of how fast things move in our industry.

Right, yeah. Windows Phone 7, wow. [laughs] I don’t remember what phone I was using at the time. I probably had a Palm Treo; that was my favorite. I don’t know if I’ve ever reviewed that one.

So you were an early adopter on the smartphone.

Sorry, I was way off. The Palm Pre, the one with the sliding keyboard and with the magnetic charger thing.

Did that have Web OS on it?

That’s the one, yeah.

Oh yeah. I never had that. I was straight from flip phone to iPhone, and I didn’t get the original iPhone, I had the second one (iPhone 3G) and I’ve never even tried anything else… But I know a lot of people were huge on Web OS.

It was very cool. I think you can actually see a piece of it everywhere now, if you look close enough… Either because people copied them, or just they made really smart decisions that everyone else kind of got to at some point.

So you left Ars and you started - according to your Ars bio - a MongoDB hosting, which we now know as Compose. It was MongoHQ for years. Is that the right name, Mongo?

MongoHQ, yeah. Actually, MongoHQ existed; I was a very early customer, and then joined up with Ben and Jason at Y Combinator. So I quit to go do MongoHQ with them. That was the next five years of my life, I think. It was a lot of fun.

We basically started as a MongoDB hosting company at a very interesting time, because all these people were trying out this hot new database that helped them ship stuff faster. One of the things I learned about devs is it’s actually really frustrating to be able to write something locally and to not be able to show it to people, so like “mini production.” I made air quotes when I said that. [laughter]

I think we ended up just capturing this wave of people who wanted to publish applications that were using MongoDB on the back-end, and no developer wants to run a database, so it was a very natural thing to try out. We ran, at one point, like 150k free databases for people. We ultimately killed the free databases, because we learned that free database users were never gonna pay us money for anything.

There was no upgrade path from those people? They just stayed there?

[08:13] Yeah, not only was there not a path… What we learned from databases specifically was the people who will pay you money will pay you at the beginning, if it’s cheap enough. Because $15/month just isn’t that big a deal. And the people who refuse to pay you money are probably never gonna change their minds.

Yeah, that’s kind of the downfall of the freemium model - the free people who you think are growing those opportunities for upgrades are a) rarely gonna upgrade, if ever, like you found out, and b) they tend to be the most needy customers in terms of support. That’s not my experience, I’ve never ran a SaaS, but that’s what I’ve heard.

No, they are, definitely. They are the loudest on Twitter… One other problem we ended up having is we’d basically send everyone to try out MongoHQ, the free database, and the reality of it was it wasn’t even our best experience; we were better off giving people a free month on the actual paid product, because it represented all of the things we’d be doing for them, instead of them having to imagine and get a subpar product.

I learned a lot about freemium there. I’m mildly cynical about it now. I have strong opinions on how I would like to do freemium because of it.

Interesting. We might get to that, but let’s talk about this next move… So you left MongoHQ/Compose and now you’re co-founder of Fly.io. Tell us about that decision in your life, and then the opportunity that you had with Fly.

We sold Compose to IBM in 2015, and I made it eight months at IBM. I learned I’m not a big corporate guy… And quit, and had just this giant list of stuff that I kind of wanted to try. Essentially, I quit that job and didn’t really have a thing to focus on, and I ended up trying out a lot of other stuff. I started taking impro classes, which I still do. It was great.

Oh, really?

Yeah, yeah. But I needed something to work on. It wasn’t even necessarily a money – I mean, obviously, we need income. But what I really needed emotionally was like a project, so a friend of mine named Jerome (who had worked at Compose) and I were just going through ideas… At that point I had sworn off infrastructure, I was like “I’m not gonna do it again. I want to do something entirely different.”

Like improv.

Yeah, exactly.

Did you try anything else? Was there any other random things that you did that maybe didn’t stick?

I tried racing cars, because I had always wanted to, and I had time… And that was fun, but it didn’t scratch any creative itch. It’s a very mechanical – like, to go faster on the track you have to do the same thing right every time, and not a lot of creativity there.

I did like rally cars, because off-road is a lot more unpredictable and you slide around and stuff, but still, it wasn’t fulfilling in the way that I think I was looking for. The improv helped, because that was creative.

I also tried being a full-time parent. I have four kids, so I spent three months as a full-time father, and that was plenty for me. I was fully content after three months.

[laughs] You don’t really get out of the house after that.

I really enjoyed, I’m glad I did it, but I’m not a full-time parent forever.

What about financially? Full-time parent, obviously, is part of a household, but in terms of things that bring in finances - the rally car thing, the improv thing… These were hobbies or creative endeavors, but was Fly your next effort at financial success, or were there things in there, false starts before that?

We didn’t really have any other stars; we had a lot of ideas, for things like helping parents control springtime, and other things that it would probably be really hard to make money with… But I think the farthest we got on any of that stuff was mainly writing out the idea, trying to determine if it’d be useful.

[11:54] One of the reasons we did Fly is I decided it’d be a good idea to go talk to our previous investors just to keep the relationships warm, just in case we ever wanted to raise money, and a lot of them actually – I was just talking through ideas, and we actually had several people offer to invest purely on the basis of Fly, basically, the one infrastructure idea we had… I was like, “That’s interesting…” That’s a big shortcut to save a lot of money fundraising and just be able to do something.

I feel like that was fortunate, and we probably won’t ever get that experience again, but we basically got an easy round of money out of it, which is never something that happens as far as I can tell.

Were there any question marks, like it’s almost too easy? Sometimes if it’s too easy, if something’s too good to be true, it probably is. Were there ever doubts that maybe – like, how is this going so well? Because so many people struggle to get there at all.

Yeah, so I’d gotten pretty – I think I have a relatively healthy take on fundraising… I didn’t really have doubts about it being too easy, because fundraising isn’t actually a sign of anything; it’s like a thing you can use to do good stuff. But it was like “Well, we’ve raised money. Now we’ve gotta actually get to work” That was my mindset.

But it has been a really fascinating learning experience to deliberately build a product from scratch underneath a round of money and get it launched. Everything else I’ve ever worked on has been honestly accidentally successful, but what’s really been a hobby I spent a lot of time on, that then became a business because people wanted it to be… And it was a lot less intentional, I guess I should say. So it’s been really fascinating to learn how to build a company from scratch without already having a product, basically. It’s been a lot of fun. So that all the doubts from that.

Gotcha. So what did the pitch look like back then, around Fly? What were you telling people you were solving, what was the problem space?

A lot of this actually came out of Condé Nast. Basically, the way I was explaining this to people was there was a layer of the internet that developers don’t have API’s for, which is kind of the load balancing/proxy layer; CDNs work at that layer, but you don’t really have the ability to control your CDN. Load balancers work at that layer, but other than setting up a load balancer, maybe a REST API on Amazon, you can’t change how it behaves. So the way I was pitching this was like “All internet infrastructure will ultimately be a programmable environment, because you need that level of control to apply business metrics to your infrastructure.”

We were right about how we were pitching this. I didn’t know the exact metric at the time, but one of the things we’ve learned is companies used to track time to first byte religiously, so when they were setting up a new service, they’d track time to first byte for any request that was made to their application, and try and optimize that. That was a very technical metric, but it didn’t really mean anything to businesspeople or salespeople, or execs at a lot of these companies.

Right.

Then what we found is that there’s like Lighthouse scores, which is the Chrome app performance tool.

Yeah, it’s meaningful now.

Yeah, it’s very meaningful. They buy that like a good Lighthouse score equals money, and one of the really interesting things we’ve come across is like “Cool! Well, you can actually program at this layer to improve the Lighthouse score.” It’s weird, because it matches the very broad thing we were pitching, but it’s not exactly how I expected to end up making a business happen.

It’s a bit crazy… I just think about that and I think the reason why they are convinced that that is meaningful now - even though it’s always been meaningful in terms of end users, it foresaw performance and all those things…

Right.

…a) we have metrics, and we have the articles and probably case studies written by Amazon, talking about how much money they lose with every second or millisecond. Then we also have the other big player, which is Google, with the stick of Google search results. So as soon as they say “Now speed matters in your SRE or your pagerank”, now it changes everything.

[16:07] It does. It really does. And we were using those ten years ago at Condé Nast; we were beating the speed drum using those same quotes from Amazon, those same quotes from Google. So yeah, Google has basically put that stake in the ground, of “Speed matters. We’re gonna apply it to speed rankings”, but they’ve also simultaneously given people this score that is indicative of what they expect the user experience to look like.

Previously, we’ve used lots of tools to track stuff over time, and it’s that leap between what this tool is telling us and what we actually think users’ experience has always been too big. Now it’s very easy to – if you go to any Gatsby-generated site, it’s scoring a Lighthouse 100 and you click on a link on that site and everyone’s like “Oh yeah, that was really fast. That actually matches what I’m seeing.”

Exactly. So now it’s more apparent to the business side of businesses that this is something worth investing in, but back when you were giving the pitch, it probably was still somewhat of a hard sell. But you weren’t selling necessarily the speed; I guess you were selling the malleability of CDNs, or the programmability of infrastructure.

Yeah, speed is actually a really good thing to solve at this layer… But yeah, what we’ve ended up building now is kind of a global application platform, and ideas like “People should write JavaScript once, and they should deploy it, and they shouldn’t worry about where in the world it’s running.” It’s really well-suited for proxy-level problems, it’s really well-suited for people who wanna build CDNs, it’s really helpful for speeding things up. We’re hoping it’s actually just an interesting take on application architecture and deployment, and what we think the future of building apps is gonna look like.

So you had a pitch to investors that was convincing, and then you had to go out and actually build the thing… Give us the timeline here. So it’s 2018 June, you have a product out there… How long did it take, where was the actual fundraising, versus building a company and getting to a place where you are now?

We “finished” fundraising right at the beginning of 2017. At that point it was finished, like “We need to stop this. This is not something we should spend time on anymore. It’s time to get to work.” It only took about a month really, because like I said, it wasn’t intentional. We obviously didn’t need the money, because I wasn’t asking for it. I’ve learned the best way to pitch investors is ask them for advice, and mean it; just, all you want from them is advice, and sometimes the money comes. It was a very interesting thing to see.

[19:47] So we raised money really quickly, and then we sat down to start building. We had some ideas for the very smallest thing we could give to customers. I get really anxious without building things in stealth. There’s so many startups that are in stealth mode that never ship anything, or they make good promises but never get customers… And I really, really like the feedback from customers or users or people seeing your stuff. It’s nerve-wrecking to put things out there, but it’s less comfortable for me to not have something out there, because I just don’t – it’s hard to wake up every morning and be like “I’m just gonna work on this thing, but nobody can see.” It was very important to me that – which kind of goes back to the MongoHQ thing a little bit… It was important for me that we could put an app or database up really fast.

It was at the end of April 2017 we actually launched our first service, and all it really was was a way for people to combine applications on the same hostname. We actually have onehostname.com with like a Lord of the Rings-inspired cartoon, but the idea was while you have your app and you have your blog and you have your marketing site, and it’s actually really kind of a pain to put all those things under Fly.io, instead of blog.fly.io…

That’s a fact. So instead of sub-domaining, which is what you end up doing to have completely disparate applications hosted on the same domain, you’re providing a way that you can use subdirectories, basically, or paths, instead of that, all combined together, but have completely separate infrastructure for each one. There’s no coupling at all between the implementations.

Correct. And a lot of the paths could just be third-party hosting services. Your Ghost(Pro) blog could be on /articles, which is exactly how ours is. Your GitHub Pages site could be on the root, and your Rails app could be on the root if you’re logged in. That was the extent of the service at the time.

That’s useful.

It was useful. Startups have a myth, and when we go back until our founding myth, what it’ll be was like we needed to launch this service so we could build the platform underneath it… Which was never quite that clear in our minds that’s what we were doing, but that’s how it kind of worked out. So we got a lot of people using that… We ultimately expanded it to let people add a whole bunch of custom hostnames to their app, and that was useful for people running like a blogging service, or anything else; it was very difficult to issue 1,000 SSL certificates for your customers, and customers have started demanding it. Even GitHub Pages at the time didn’t do SSL, when everybody wanted it.

Right.

So we ended up running about several thousand domains on that platform, on that service, and serving several hundred million requests per month on top of it, and never charged for it. We actually launched with pricing, but hadn’t built billing yet… And as we looked at it, we thought “Well, this is a cool thing to give people, but it’s not really the product we want them to pay for.” It wasn’t a good enough product, we didn’t think. That was at the end of April.

Then in April of 2018 we actually launched the application runtime. So it took a year, basically, of running this for people and then building what turns out to be a custom JavaScript runtime from scratch, and then getting that deployed, actually launching that for customers.

That’s pretty fast, I think. Things probably felt slower for you as you’re the one toiling away…

They did…

But it was pretty quick moving. We were joking before we started hitting Record that things move very fast in this industry, and just to give a little bit of the inside baseball on this particular show, we first started talking in late March, and now it’s almost late June… And a lot has changed, even with your website… Even while I was talking with Christina, getting this how set up, I was at your website, and she’s like “Actually, hit Refresh, because there’s a whole new version of it. The branding is different…” So things move very fast, and I actually even noticed since then on your homepage there’s a big button now that says “JavaScript at the Edge” and right next to it, in this orange pill which I actually didn’t catch at first (which is strange) it says “Open Source”, which to me is new…

[24:08] So let’s talk about what Fly is today, and how you guys describe it, because it’s a bit nebulous; I think a lot of these infrastructure platformy startups are nebulous, you’re not sure… But the tagline right now is “JavaScript at the Edge”, and then there’s your open source tag, and then it says “Fly is a global JavaScript runtime that makes apps faster. You can optimize images, pre-render, cache partials, and more.” So I get a little bit of that, I understand what I can do with it, but how do you describe it to folks who are like “What’s Fly?” What do you tell them?

There’s two stories, and one of the things I learned doing this is like we have this story for what I would call the investor class, but it doesn’t always mean investors; it really just means people who think about the world and care about where technology is going. One of those stories is we’re building a global application runtime, because we don’t think that people should have to worry about where things are deployed; we think people would be better off if they just write code and run it, and it does what they want. And it’s resilient, and it’s fast for people wherever in the world they are, because nobody builds a startup targeting just people in Chicago anymore. You’re almost global by default when you launch software now.

So that’s the big vision, that’s the pitch to investors, and that’s this really aspirational thing that if we look like that in ten years, then we’ve built kind of what we expected to. One of the things I learned even at the last couple of companies was like customers don’t care about the big vision; they really wanna know how to apply this to their problems right now…

[laughs] Right.

And it’s fair… Partially because it’s easy to sell a big vision, but it’s hard to know when to use new technology even. This is extra-interesting, because at the last successful company we were hosting databases that people already knew about. There was no point when someone would go “I don’t understand what you do.” They’d go “I don’t know what Mongo is” and we’d say “Then you probably shouldn’t even be looking at us.” But if they knew what Mongo was, they understood what the product was and it scratched an itch they had right then.

Right.

Building a platform like this is interesting, because you end up having to build the platform – it’s a little bit like building an operating system, I think. It can do a lot of different things, and we think it’s a good way to do a lot of different things, but what we end up having to focus on for customers, just by necessity of how much time we can spend and the story we can tell - there’s one or two things that are very valuable to them.
So right now, the most valuable thing we can help people with is improving our performance. It just means speed, it means Lighthouse scores… So that’s the story we’ve sort of iterated to on the website. People like the word Edge. Devs like hot, new technology.

[laughs] That’s kind of funny, because I actually have a negative reaction to the word “Edge.”

Because of the browser?

No, it’s not because of the browser; it’s definitely in there… But just because everybody is using it now. We were just recently at Microsoft Build, and Microsoft is talking about the Intelligent Edge… A lot of the AWS’s, the Azures – everything’s Edge all of a sudden. So I get negative buzzword reactions. If I start hearing a word too much, even if it’s a fine word, I don’t have any problem with the word, it just starts to be like, “Yeah… Another Edge? Come on, guys. Come up with something fresh.” [laughs]

I have the exact same feeling, and what’s funny is – at one point our site said The Serverless Edge, which probably would have just made your head explode…

Yes, for sure.

What’s really fascinating to me is like for you that’s true, for me that’s true, because we’re immersed in this stuff. I think a lot of people, especially devs at some of the larger companies we’re talking to - they’re actually really drawn to serverless, because they feel like it’s this interesting thing. They’re really drawn to Edge, because they feel like it’s this interesting thing, and it’s one of the few places where the language works for customers and for–

And for investors. Yeah, that makes sense.

[28:08] Yeah. So we’ll keep changing that, just depending on who’s doing what, and at some point we’ll know what people respond the best to, but… That’s the other fun thing about devs - 10 of them will like something, and 90 won’t, or another 7 will like something else. Until we can get the site to say exactly what people are thinking all the time, we’ll just keep changing.

Right, exactly. Until the machines just know, customized based on what I’ve responded to previously… Or the GDPR might stop us from getting that done, but anyways, heading upstream there. You said something very interesting, that resonated with me, coming from somebody who’s – you know, you’re providing/building a platform, a runtime, a thing where I use it and I don’t have to worry about any of the other stuff, besides what my code does, hit a button and the world gets it. As a customer, buying hosting or buying infrastructure - either as a service, or going out and buying hardware - is a very big decision for us, and you never know what’s gonna happen.

A lot of startups disappear, and the worst thing that could happen is that my infrastructure disappears. Or maybe I’m not this way; maybe I definitely hate IBM and I love MongoHQ, but then IBM buys MongoHQ and now my database is with the company that I’d been despising privately… Especially with devs, we get very emotional about certain things, and it seems like it’d be a really hard sell, because to get me to either move or to try something new in a serious way, that’s not like a throw-away application, that’s a really hard sell… So what are your thoughts on that?

It is. That’s actually part of the reason we open sourced the runtime. There’s a lot there. I think devs expect what they’re building on top of to be open source at this point. Node is open source, the browsers are open source… It’s rare for individual devs to pick something that’s not open source to use, and that actually makes a lot of sense, because like you said, I’m investing in a tool; it would make no sense to learn how to – even carpenters would not learn this non-standard tool if they could help it, and open source is the closest thing I think we have to standard tools.

So we made that open source partially because we thought people expected that, partially because we’re devs and that’s how we expect things to be… And a lot to kind of give people some comfort that if we vanish or you just don’t like us or you wanna go a different way than we’re going, you can take those and run it yourself. It’s not easy to spin up servers all over the world and get this going, but it’s doable; it’s there if all else goes to heck… It can be done, and you’re relatively safe in that respect.

One of the interesting things that I’ve come across is I expected that to be a lot bigger part of the story to bigger customers, and it’s really not. When we talk to bigger companies, they’re primarily concerned about SLAs and data, and who owns data, and support agreements, and all of these things. There’s a little bit of like they have to sort of check the box; open source checks a box for them, and I think maybe defaults into a different mindset… But for the most part, they really just want something that solves a problem for them that they don’t have to think about. So this is when it comes back to we improve Lighthouse scores; what they want is to improve their Lighthouse scores, and if we can do that, and we can prove it (which we can), they – they’re not really worried about where this is gonna go, because they just want better Lighthouse scores. It’s a problem they need to solve. [laughter]

We got a little lucky, because I think just selling infrastructure to companies is really hard, but selling business improvement takes a lot of those questions and it doesn’t matter anymore.

Interesting.

Yeah. So a lot of the work we need to do is to get developer mindshare underneath, but the actual selling of the product is going incredibly well, shockingly.

[32:09] Awesome. So tell us about the open source side. You say it’s open core, so tell me exactly from your perspective what that means, and the nitty-gritty of what is open source, what’s not, what have you.

Open core for us means you can run – basically, you can run the app in one place as if it were running on our service. The parts of the service that are open source are the parts that are easy to distribute, so the way you get started with Fly is you npm install @fly/fly and off you go. So everything that it takes to do that is open source.

What’s not open source is managing things like thousands of certificates. That’s not really for a business reason, as much as it’s like just not easy to extract into a package like that for us at this point. I think at some point we’ll do it. We have a distributed caching mechanism that is, again, difficult to extract in open source.

What we’ve found is that – I think the bar for open sourcing stuff is actually a little bit higher than it is to just build it and run it. You can build a dirty service by patching the good stuff together and keep it running and no one ever cares… But there’s a certain – and this might be pride talking… There’s a certain level of quality and portability and installability that we want to have in the open source bits. So we find that what is open source and what isn’t is more a function of what we can package up nicely for people than any kind of business decision.

Basically, distributed cache isn’t open source, but they’re local and it works just fine. We have distributed data storage coming at some point; that’s not open source, but it works fine locally. And then you can’t do SSL locally, and that, I expect, would come some day. Local and open source are kind of the same in our particular world.

Gotcha. So I’m looking at the open source repo here - which is also in the notes for you all listening out there - and I’m excited to find your own avatar here with 36,000 lines of code, or 19,000 deletes, 118 commits… You’re in the thick of it here.

Yes. I would much rather build stuff than sell stuff, usually. Jerome and I could easily just build things and never have a functional company if we weren’t careful about the decisions we were making… [laughter] My happy weeks are the weeks that I’m talking to devs on Slack and not talking to big companies, and doing pull requests and shipping features. That’s when I feel the best. It’s quite fun.

And I’ve even gotten to do talks about embedding JavaScript at various JavaScript meetups, and that’s great fun. I’d much rather do deeply technical stuff than write a whitepaper for a VP of something at some big company to read. So yeah, I don’t know if I’ve done anything this week, but next week I will.

[laughs] Well, I see the only person that you’re behind on the contributors list is Jerome, so it’s a bit of a battle that you’re losing, but how do you balance that? How do you know when to put the code editor down and say “I’ve gotta go take this investor meeting” or “I have to go sell something”?

You know, that’s a good question. I don’t actually have a good system for this. It’s a lot of gut feeling. Like every startup, depending on the day, I’m either super excited about what’s happening, or I’m worried that we’re not gonna get over the next hump. And there’s always a next hump. Our next hump is to build a sustainable company out of what looks like a really good start. So if I’m obsessing over the hump, I will go come up with ways to try and get more customers. And if I’m really happy with – like, if we just landed a big customer, I might go back and reward myself with writing code.

One of my happy balances I’ve found is I can spend a lot of time writing just Fly apps and then showing them to people, and then it’s a little bit of double purpose, because I’m both user number one, and it’s helping people learn what they can do with the platform. So that’s my airplane time. When I have to fly somewhere, I end up writing a bunch of example apps.

I think my last commits were actually three example apps on an airplane ride where I didn’t have any Wi-Fi. That was my fun bit.

[36:11] Awesome. Yeah, I did find one example that I thought looked pretty cool, which was FlyGit, which is RawGit on top of Fly. Did you have your hand on that one, or was that somebody else?

I did, I did. Actually, we ended up consuming a lot of files from a repository and injecting them in other places. So if you go to our Docs site, which is fly.io/docs/apps, it’s just our readme, and what we do is we pull that markdown down and render it and put it into our normal-looking site. So we do a lot with mixing and matching content from Git. FlyGit is the basis for a lot of that, and we’ve taken bits and pieces of that and used them elsewhere.

Very cool. While I’m staring at the repo, I have one last question down here. 70% TypeScript…

Yes.

Do you have opinions on TypeScript you’d like to share?

Oh yeah, it’s 70% and trending up.

Is it?

Yeah, it’s more and more TypeScript. I really like TypeScript; I’ve always really liked types, and I used to be a pretty heavy Windows dev. I did a lot of F#, which if you’ve ever done F#, you have to like types, or you don’t do F#. And there’s a lot of that stuff that’s trickled into TypeScript. We both really like TypeScript, because it’s easy; it’s much easier to reason about. One of the things we had to implement for this runtime was a – so we were basically implementing browser APIs, because they work really well for our use case, and browser APIs tend to be really well-designed, the modern ones… So like Fetch and Request and Response are just a really nice API to give to people. Request and Response both have bodies that can be streams or array buffers…

There’s a whole bunch of different things that those bodies could be, and one of the body mixins I implemented was first in JavaScript, and it just got overwhelming because I couldn’t reason about what was actually happening throughout that big, messy JavaScript, and TypeScript made it about 1,000 easier to understand what we were dealing with and how.

The only thing that we’ve found – and there’s nothing TypeScript can do about this I don’t think… But since we do write a library, and a lot of our customers use JavaScript, you can still send untyped garbage into TypeScript and you don’t have any guarantees that what the compiler just said is happening is actually what’s coming through… And it’s not so bad; it’s just a thing that’s sort of always in the back of our minds when we’re making APIs. If we expect an object in a certain format, we still have to validate that at runtime, or it’s not gonna go well.

I see.

I love TypeScript.

Since you’re consuming somebody else’s code…

They’re data, they’re data structures… Yeah, exactly. TypeScript’s great. We’re huge fans. You’ve probably seen Ryan Dahl’s Deno…

Yes, in fact we had – so another show on our network is called JS Party, which is all about JavaScript… We had a TypeScript Party a couple weeks back on that show, and then we also record live on Thursday and we had a section (it’s kind of a three-segment show) that was all about his talk at JSConf EU, kind of just reacting to the ten regrets he has about Node, and then his new thing…

What’s interesting watching him do that is we’re sort of converging on similar ideas; TypeScript was one of them. Things like the browser APIs making it so your code runs in a jail and it can’t do anything except one specific Fetch function… It’s all very fascinating. I like that Deno is natively TypeScript, and we’ve thought about how to make Fly natively TypeScript for developers instead of just ourselves, too.

So we discussed the big picture of what Fly does today, what you’re trying to do eventually, how you talk about it with investors, how you talk about it with potential customers or current customers, developers… Where do you put it in the landscape? A lot of listeners, and myself included, understand there’s other products out there doing similar things, some of them are nebulous what they do, some are very obvious… You have your Netlifies, you ZEIT Nows, your kind of last generation platforms like Heroku, but then you also mentioned serverless, you’ve got your Lambdas… There’s just lots of different options out there; where does Fly fit in, and if you do ever have the question of like “How does it compete? What differentiates it?”, how do address those other people out there building similar things?

Netlify - we’re actually pretty decent friends with them. Netlify is a company that if we’d reverse to started when I think you would have built Netlify on top of Fly. Fly I think is a good platform for building a Netlify type service. There’s definitely overlap, just because if you’re targeting developers, you end up doing 5%-10% the same things that everyone else is doing… But Netlify is a really good possible customer.

ZEIT is really fascinating. They made some genius code. I feel like ZEIT is a better Heroku… And I should say a more modern Heroku, because it’s actually still really difficult to make something better than – Heroku is still really good. The developer experience just works; it’s all kind of still amazing. I think if Salesforce hadn’t bought them, it could be a really interesting alternative to, and asuming it had stayed alive and gotten bigger, Amazon, Google Cloud and Azure part of the world.
The people we get most often asked about and compared to is Cloudflare, because they have service workers, and Cloudflare service workers look a lot like Fly apps. I think it’s a natural API to implement for the types of stuff we’re doing.

The big difference we have with Cloudflare, and investors asked this one a few times, is AWS is like a fully programmable data center; you don’t ever call a salesperson to turn on a server or something else for you at AWS. Previously, there were less programmable data centers, places like Savvis, places like Verizon, AT&T, all these guys, Equinix sold space and power to people. Cloudflare I feel like is a slightly older generation of a similar problem we have. Whereas we have an API for creating a WebP image out of any other image to serve it up to the right kind of browser, they have a button you check to turn on the WebP. They’ve put these service workers in the middle, so you can script some stuff and there’s a fairly decent set of things you can solve with Cloudflare that you could potentially solve with Fly too, or vice-versa… But we’re like developer-first all the time. There’s not part of anything we do that we want to hide from an API effectively, instead of this being a smaller bit of a much larger stack.

[44:18] So you can see that because we’re open source. We also have local testing, so you can write and run and test apps locally, which means it works on like a continuous deployment process… Which is an interesting thing, because one of the people we initially thought we were competing with was Fastly, because Fastly is a great company that builds really cool stuff and is kind of the pinnacle of what a CDN should do… And one of the complaints we kept hearing from customers of Fastly - and there weren’t many complaints; everyone’s generally pretty happy with them - was “I’m terrified that I’m gonna change something and break my whole site. I’m a developer, I’m used to this whole development process, I’m used to writing tests, I’m used to having some level of comfort when I make changes”, that they didn’t give them.

Compared to those guys, I think we’re much more like the application-level developer-first, developer-all-the-way thing. I think more broadly we’re probably competing with anyone who wants you to write new code on top of them, like Firebase or any of those people. But it’s such a broad thing… The people we’re talking to customer-wise aren’t gonna compare us to Firebase usually. It’s just like – we’re kind of all trying to go the same direction.

Gotcha. So you mentioned you’re open source; another thing that you’re doing which is super cool and I want to hear more about is that you are giving equity somehow to open source authors, people who are providing (I assume) specifically the tools that you’re using to build Fly. Tell me about that. First of all, who thought of that? How’s it going? Or just tell me all about it. I actually haven’t heard that before, and I think it’s a super-interesting way to support the people who you’re building your platform on top of.

The genesis of this is I have very complicated feelings about open source in large companies. I think a lot of times large companies produce open source for somewhat good reasons, but it’s somewhat anti-competitive, too. Facebook creates React, open sources it, everybody builds React, and everyone basically learns Facebook’s platform, so Facebook can now hire more devs. It also incidentally has killed any ability for a company to build a front-end framework from scratch and make any money off of it. It’s a very interesting dynamic, I think.

There’s like an echo in here, because just a couple of shows back we had Zed Shaw on the show.

Oh, yeah.

That show, if you wanna listen to it, is called “Corporate interests in open source and dev culture.”

I’ve probably read everything he’s written about that, so I wouldn’t be surprised if it sounds very similar…

You wouldn’t be surprised, but he’s very covinced - and I think rightly so - with the specific thing he says, which is that corporations are using open source to commoditize their complements, which is really what you’ve just said there. It would be very difficult for somebody to build a front-end UI framework and make a business on top of it, because React is free and open source.

Exactly. That’s like monopoly 101 if you take economics. It’s what you do. Anyway…

Anyways, so I’m just hearing very similar things… Continue.

One other interesting thing we noticed - and we saw this at Compose, because we host at Redis… There’s a fair amount of hypocrisy here, because we were basically running open source databases and making money doing it. But when we were raising money - we were trying to have a B round and then we sold the company instead - we kept coming across Redis Labs, who were pitching themselves as like the home of Redis… Which I always thought was kind of interesting, because they took this open source project, they forked it, and then they were claiming commercial – it wasn’t “the home of Redis”, they called themselves “the commercial stewards of Redis”, and I was like “That’s really odd.” A guy wrote Redis, it’s really a great software; it’s interesting that all of the money to be made from Redis is going to VMWare, Compose, to Redis Labs, to now Google; Google has done their Redis software, and Microsoft.

[48:10] So he’s basically built this software that is ultimately worth in the billions of dollars, I would bet; if you were able to put a dollar value on Redis, it wouldn’t be surprising to find out it’s worth more than a billion dollars. So I’ve always been a little bit wary of exploiting people who build things like Redis just to build something cool for people and give it away. It’s like the most pure open source there is.

Didn’t Redis Labs hire him thought? I mean, they gave him –

They did.

That’s something.

It’s a job, yeah. It is, but if you were to do the math, they’re getting a deal, basically. I know what startups give the technical superheroes they try and hire for that, and I don’t think it’s even close to its worth.

It’s never commensurate with what value they’re bringing, you don’t believe it is…

Correct, yeah.

We don’t know the details of Salvatore’s deal with Redis Labs, but we know he did go there on his own volition, so it’s not like –

Well, after VMWare were hiring him for the same reason.

Yeah, he worked for VMWare for a while, and then he went and worked for Redis Labs, and he might still be at Redis Labs, I don’t know.

He is, yeah. And I have no reason to think he’s unhappy with it. I just think structurally it looks like an unfair portion of the world to me, that these people built these things that are so amazing, and people like me would swoop in and create a company out of it, because that’s what we’re kind of wired to do, is create companies instead of just create really great open source.

So do you feel bad about how MongoHQ went down then, or…?

No, actually… I’m not sure I’m supposed to say this, but I don’t care. We weren’t making very much money off of Redis. It was a cool thing to host, and we did it because we liked it, but it never was a very commercially successful thing for Compose. But we also launched it right before we sold to IBM too, which was like “This is cool! We just launched it!” So I don’t, for that reason… I don’t particularly feel bad about making money off MongoDB, because they made money off MongoDB as well.

We actually tried to make RethinkDB – RethinkDB I think we were helpful for, because they kind of needed the story of “Someone will run this for you”, and they weren’t gonna build it themselves. But like I said, it does sound mildly hypocritical, because I think most of the good stuff that happened to me has happened on the backs of open source.

Anyway, that’s the long-winded way of saying for this company we basically just had this idea, and at one point it was like “We should give equity grants to open source authors of tools that we really get value out of.” And it’s interesting, because startups give equity grants to people all the time; if you’re a startup and you go hire an executive recruiter to hire you a VP of marketing, that recruiter is gonna get some fractional personal ownership of your company, because that’s just the way that basically businesspeople negotiate this stuff. A lot of startups have advisors who literally are just answering emails and doing nothing else of value for the company, that have equity grants.

So we basically just looked at it and it was like, you know, the open source authors are every bit as advisory as these advisors, so we started to use just the normal advisor paperwork to create an advisor agreement. We took out a lot of non-compete stuff, because it wasn’t that kind of relationship, but it was a standard advisor agreement, with just the same kind of shares that I have, just a smaller percentage. We’ve offered it to four people at this point.

Are you able to share what the projects are, or is that like private?

That’s relatively private only because I don’t know – I haven’t really asked them if we can tell people who they are or not… But you might be able to guess one or two. [laughter]

Go look at your package.json and we can probably figure it out.

[laughs] Yeah. Or listen to this.

So is this a deal where you just contact them and you’re like “Hey, I’m using your stuff. I’d love to just give you some equity.”

[52:09] Yeah, it weirds them out at first. We actually have to send two emails to one person, because I think it sounded like spam, because it’s just so far out of any email they’re expecting to get. And they’re all super-excited about the idea… We had one turn us down, because they thought it might be kind of a conflict of interest for them based on what else they’re doing, but they were generally like “This is great! I’m really glad you did this.” It feels pretty good, I think it’s a nice thing to do, and if Fly is super-successful, it’s never gonna matter in terms of how much we’ve given up. And if Fly fails, it still doesn’t matter, so it’s a relatively easy thing once you get over that mental jump.

So here’s another radical idea… Maybe you thought of this one. It sounds actually less radical when I say it out loud, but… You raised a whole bunch of money and it was really super-easy; giving them equity - I think it’s a cool idea, and I think it’s way better than zero, which is what most people out there are doing, which is why we have this problem that we’re in, this sustainability problem… But Zed would say “Just give me the cash, man…” Why don’t you just give them some money? You have funding; this is development that you would otherwise have to build yourself, right? You could give them a percent of what it cost you in labor to build, or time wasted building out these dependencies. Did you consider just throwing them a bone?

We did. We actually tried. I think Zed might be – I don’t know if he’s unique, or just not everybody is like that… But we have found the open source developers we talked to either don’t need money, they’re just not wired that way; one of them I know is actually really comfy, so money is just not gonna change anything for that person… But the other flipside of that is a lot of them value their independence in a way that I think just getting paid for – like, it’d be interesting to just structure it as a no-strings-attached grant if you’re gonna try and do that, but generally it seems like there’s strings attached, even if there’s not, so we’re actually very clear with the open source grants; it’s like “This is for what you’ve already done, we don’t expect you to do anything as a result of this.”

But money is an interesting thing… Sometimes people aren’t motivated by it. I would actually take the money if I were a successful open source developer; I’d be very Zed about this. Sort of zen, but money… [laughs] But for the ones we’ve talked to – I guess it’s not surprising, but the equity is a more interesting thing to them. And it sort of makes sense. If the money is not gonna change your life right then, the equity - at least there’s the hope it might, at some point. I don’t know.

I think it’s definitely a case by case basis. I’ve been doing contract work for years, software for hire, and when you’re in that circumstance, you have a lot of people that wanna offer you equity instead of cash… So it’s like “Be my technical co-founder”, blah-blah-blah, and my answer to them is like “I’m running a business here, and I’ve got a family to feed. I would love to help you build your idea, but I don’t wanna take on all the risks that you’re taking. I will take the cash now for the work done, and then you can have all the upside later. If it’s amazingly awesome, I made a bad decision, but I need the cash to put food on the table.”

And there’s other people who may be like – if I was just independently wealthy, or like you said, comfy enough, and I’d believe in the project, then I might say “You know what, I’ll work for equity.” So it’s super case-by-case.

It is, it is. And you know, talking about this, I realize I don’t think we’ve offered anyone a money grant with the same “This is absolutely no strings attached. We don’t expect you to do anything”, like we have for the equity. I’m actually curious what someone would say, but my other knee-jerk is like it’s actually mildly difficult to do that without tax consequences. [56:16] Equity is actually a really simple thing to give somebody at this stage. The company, according to IRS income things, is basically not worth anything. So you pay 12 cents for shares, or whatever, and it’s just a relatively simple – anyway, it’s a very interesting discussion, the money vs. equity thing…

For sure.

…and it’s mildly surprising how different people are.

Yeah, absolutely. And especially on the receiving side for them, which is why organizations like Open Collective exist. Some people aren’t set up to receive money as donations. They’d be like “Neah, it’s way too much of a headache for me to even accept whatever it is you’re offering”, because of their governmental situation. So there are organizations like Open Collective that will act as a non-profit on the behalf of other individuals, so then you can filter it through Open Collective, making things easier. But it’s definitely not a utopian situation. We’re not there yet in terms of like making this an easy transaction for people.

Right, exactly.

So I guess it hasn’t played out yet, we haven’t seen that Fly is a huge success and can eventually sell those off for some decent cash, but… In terms of how far you’ve taken it so far, is this a model that you think you’d love to see other startups do? It sounds like they’ve been pretty well received by the people that you’ve contacted, except for the one who has a conflict of interest. I guess my question is “Is this a model to follow, or not?”

I think it is. One of the things I like about it for us is I think it’s a good thing to be thoughtful about. So just the act of doing it I feel like makes you think pretty hard about where you’re getting value. I feel more comfortable with my life when I can reckon with like how privileged I am to be doing what I’m doing, and how lucky – like, we could just go raise a round of money; not a lot of people can do that.

I think it’s probably just a really good mental exercise, but I also think if you’re at all worried about the concentration of tech power and you’re worried about the commoditization of open source, if every startup that went through Y Combinator and raised a round of money was giving open source authors a little chunk of their money, I think it’d be creating a lot of wealth over time for the people that are building the backbone of what we’re all doing.

[58:51] Yeah, I agree 100%. And I started to think about it, and then I’m like “You know what, I feel sorry for those startups”, especially the Y Combinators, because that’s like 20k and some training… I know they’re gonna raise after that, but even that… The number one reason why startups die is because they run out of money. Of course, maybe they run out of money for different reasons; maybe it’s not viable, maybe they can’t manage it, or whatever the reason happens to be (competitive pressures), but there’s lots of companies who aren’t running out of money, they’re printing money. If they were involved even at a very small percentage… I think we asked Zed Shaw, like, what percentage would make sense, that you’d be happy and you’d sleep well at night…? He was talking specifically about the huge tech companies would give to their dependencies - it would take much as a percentage to really raise the level of a lot of people’s lives who are out there writing software in the open source space.

No, it wouldn’t. And like I said, it feels free; the actual cost for us to do this is mostly an imaginary cost. If we’re huge and successful, it will cost us some money, but at that point we don’t care… [laughs] And like every startup, there’s a very good chance that it won’t cost anyone anything, because we won’t actually be able to succeed, so also don’t care.

Alright. Anything else, Kurt, before I let you go? It’s been a very fun conversation. Anything else you wanna chat about?

No, I think we covered it all. It was a fun talk. I love talking about JavaScript and open source, so this was a good diversion from work today… If you’re not working on JavaScript and open source. [laughter]

Happy to divert you, and we thank you so much for your time. I love sharing that bit about what you’re trying to do, giving these authors some equity, and hoping that works out to their benefit, as you benefit from them. That’s a very cool idea, and I’m interested to hear how that goes over time.

Listeners out there, if you know anybody else who is doing that as a startup or as a small business - or heck, as a large business - providing funding or support for the software creators that they rely upon to get that money, let us know; we love to hear about these situations, we like to promote companies that we think are doing good in the space.

Thank you for Fly and all you’re doing there, and let us know, listeners, who else out there is supporting the open source community. Alright, Kurt, thank so much! This has been lots of fun.

Yeah, thanks a lot!