Justin Searls: …for a services company, it was really hard for me, that whole decade, as a sales guy, as the guy pitching. Because we founded the company, Todd and I, on this theory of the case that ROI, return on investment and business value, from which everything else flowed… Like, we had found Agile and extreme programming, and we’d iterated and innovated in teams in the past, and really helped companies figure out “How am I going to make or save money with software project, and how am I going to avoid failing with this investment that I’m going to make in software?” We were really good at it. And we hired a bunch of people who were really good at it. And then my job as a salesperson was to get on the phone and make the case of why us, why this approach, why we be different, why it’s worth it. And you know what I did instead? I’d get on the phone with people and I’d start up like “Well, here we’ve got TDD”, or “Here we’ve got this cross-functional team”, or “Here we work with the product owner and stakeholder and we educate”, and all. And I couldn’t get the fifth word out before somebody would respond “Alright, how many warm bodies do you have, and what’s your rate?” Because the only problem that they were solving for was headcount. I’ve got to get a thousand people into this Uber building so that it looks like they’re a real legit deal, because we’ve got a serious C – what are they, in series G at this point? No, they’re IPOed.

Hazel Weakly: And that’s not great for the company, or for you either. But like even just going back into - yeah, you’re part of the help desk, you’re part of… And you know what’s really great about being part of the help desk? You have this really good, really refined intuition of what can be automated, what can’t be automated, and you should be working on that. And one of the best ways to work on that, one of the highest leverage ways to do it is internal tooling for non-technical people. Nothing humbles you more than going “I built this thing. It made sense”, and everybody’s “Why the f**k is the button in the wrong spot?” And you’re like “Okay, what?” And they’re like “Okay, I don’t get it at all, actually. I get none of this. What do you mean this is basically just like [unintelligible 00:47:18.12]

And so you have to make things understandable for people. But also, when you work on internal tooling for non-technical people, it ends up being such a massive leverage point, such a massive influencing point, and such a massive return on investment that it’s shocking that people don’t do this in companies. Like, I’ve yet to see a company where you have someone saying independently, “Oh, I’m going to build an internal tool for the help desk, people, because people spend 60% of their time on this one type of problem, and I could just build this in an hour or two, and then it would help so many people.” And you can’t do that, because that’s a waste of time. And I’m like “No.”

[00:48:07.11] At a previous company we had DNS records, and we had S3 buckets. A lot of clients would want to do S3 bucket direct transfer, but that requires clicking a couple of buttons, setting up a couple of things, and touching AWS somewhere. And so the help desk people and the solutions people would talk to the DevOps types of teams, and those teams were overloaded with busy work of basically helping the help desk. And I’m like “These teams are underwater. They don’t have time for this.” It would take literally two days to just build something that you take a bunch of forms, you fill it out, and then on the backside it does a bunch of hideous selling out to Bash script that just creates all this stuff. It’s fine. It doesn’t need to be amazing. And I never actually got permission to build that. And I was like “This is stupid.”

Yann Lechelle: And then by design, the open source team that is full-time on Scikit-learn and other open source libraries, means it’s a cost center to the company. So that cost center is by design, and we know that’s a cost we have to cover. So we will cover it through different types of activities… For instance - and this was something that was done in the past, where brands were sponsors. So either they hire someone that becomes a core developer, and they’re naturally sponsoring someone to build up this technology, or they were giving money as a donation to the research center. But now that the team is with us, we are translating this into a contractual sponsorship framework. So brands who want to contribute to Segatern and help us compensate for salaries will get something in return - exposure. And if they actually put more money into it, then we’ll have conversations around the roadmap. Find a way to make it converge in a win/win kind of way. Because Guillaume, for instance, can say “This brand wants us to do something, but it makes no sense for the community.” Then we won’t take their money for a sponsorship type of business. However, if companies want to pay us to do a certain type of paid for software, we’ll look at it. But that’s a different branch of the company.

So we’ve really clearly separated, and by design we know there’s a cost to it… And that cost is actually - if we are doing well, it’s compensated by the fact that we have done good by the brand. In other words, hopefully the community will actually resonate with what we’re doing, and so they’ll pay us back by actually appreciating what we’re doing, which will carry the message further.

So we think that there is a self-fulfilling prophecy if we actually keep adding value to the whole scheme, as opposed to removing value. And I will not name certain projects that have chosen a different way. But on the other hand, going back to the governance of the company, when a company flips and becomes VC-funded, or only VC-funded, VCs require a sort of return on investment that is too radical. And so that sort of forces a change of posture vis-a-vis the community and the licensing scheme. In our case, we’ve actually created a structure that is balanced in terms of shareholding groups, and so we will ultimately have - or that’s the goal of the structure and the architecture - as much money from public support than from private support. So it’s, again, sort of balanced.

Break: [00:35:19.19]

John Nunemaker: Right? I mean, you can’t tell me if Gauges wasn’t still around and we were working on it… I mean, it’s around, but it’s some PE firm owns it, so who cares? But if we didn’t work on it for the last 15 years, you can’t tell me that it would not be killing it. I just, I would not believe it. I feel like it would be killing it. And Harmony as well. I feel like Harmony was way ahead of its time on its template system, and the control that it gave you, and the ease of use for people that are not devs to put content in, and stuff like that. There’s a side of me that always wishes genuinely that I could A/B test my life and see which path… Because I like control, and I like return on investment, and I like having an impact, and it’s really hard to do in a big company. Obviously, things went great with GitHub and stuff like that, but I’m just better in small stuff. Small pond is my – that’s my thing.

So I’m really curious. I don’t know. I don’t know. Maybe it would make sense to go bigger, or to do other things, or all that kind of stuff… But I know those other things, and so on this instance right now, I don’t want to sell. I want to – as long as I can, I want to just be like buy and hold. And also, the last three weeks I’ve been reading Warren Buffett… That’s probably part of it. But I’d formed the thesis before that. So now I’m just like “Okay, well, what does he do?” Because he’s a very notorious buy and hold kind of person.

So yeah, so that’s kind of it. I could be totally wrong. Maybe I am just like zero to one, or zero to 1.5 or 2, and sell. That’s entirely possible. But I kind of wonder what would happen if I didn’t. And I’m like, I don’t have to sell right now. So what if we just created –

Abi Noda: [laughs] And what we ended up doing - we froze features for a quarter. All of GitHub engineering. No features. A whole quarter spent fixing these problems. It was dramatic. And things got a lot better as a result. And another example is actually Atlassian. Their CTO is very public about how they’re focusing on developer productivity, developer experience. At Atlassian not only do they have a pretty substantial portion of their engineering organization that is devoted to this type of stuff, but they give all product engineering teams at Atlassian 10% of their time to be spent fixing things - as they call it, “fixing things that suck.” That get in the way of productivity.

But to answer your question, Adam - what do we do about it, and what’s preventing us from doing things about it? I think it actually boils down to the fact that… You see a survey like this from Stack Overflow, right? People are unhappy.

It’s because of technical debt, and the developer experience. But to actually do something about it as a business, you have to be able to calculate that the cost of doing something about it is outweighed by the return on investment you’re going to get after you do something about it. And I think that’s a really hard problem right now. And no one knows how to actually quantify this set of things, the developer experience. It is something that you can take to the CFO or CEO and say “Hey, we’re slow because of X, Y, and Z. And if we fix X, Y, and Z, we’ll be this much faster, and it’ll be worth it.” That’s the hard problem. So no one can make the case for doing something about a lot of this stuff, because you can’t talk about it in terms of dollars.

Autumn Nash: I’m going to cite that in my GitHub talk. But no, but I really think that we’re out here acting like diversity and inclusion is some sort of charity work, and I’m like “So we’re just going to ignore return on investment in building good software, and having to iterate because you made a mistake that nobody caught, because you didn’t have diverse perspectives in the room?” Let’s make good software. This is not charity. This is good business.

Autumn Nash: They basically said YOLO to all warm and fuzzy things. But the thing is, is let’s go back. Let’s talk about the capitalistic way that these things – because that’s the part that pisses me off, right? As a black woman, people will be like “Oh, but DEI just makes you feel good.” No, sir. Return on investment. The fact that you make better software. These are data-backed things. Carbon-neutral - you can say that in a warm and fuzzy “We’re going to save the environment”, but the less compute power you use, the more money your organization saves. There are capitalistic reasons that these are good things, that have nothing to do with just feeling good about them. They’re better for business. And now you’re getting a sh*****r product, and you are spending more money. And you’re going to make crappy software… What? Being a bad capitalist - this is not good. It’s not even good being a bad person. Like, I don’t get it.

Autumn Nash: So it’s evident that you’ve not just listened, but then you’ve implemented and you’ve really made an investment into making that easy for people. So I just think that is – I hope other people that want open source contributions and help from community really listen to that, because that is awesome. And I definitely think that it sucks that you have to always bring it back to return on investment, but I definitely think that you will get a return on investment on that, for sure.

Zach Leatherman: I mean, I don’t know… To me, the whole point comes back to “Do we want to make content for algorithms and computers to slurp up, or do we want to make genuine connections with people, and grow community that way?” And to me, there’s just so much better return on investment trying to make connections directly with people, rather than worrying about how your page ranks on Google, or… I know it’s important to at least think about it a little bit, but… Yeah, I would rather make connections with people online, so you can grow community that way… And the fragmentation of social media is making that very hard now, but…

Andrew Atkinson: Yeah, I feel like as I’ve worked at different companies, and have been more as an application developer, and then more on infrastructure teams too, I’ve kind of come back to some fundamentals. SQL - it’s beneficial, whether you’re doing OLTP application work, or analytical-style OLAP type of work for application developers… Whether it’s on-prem or hosted, I think the best return on investment is learning query planning, learning index design, and being able to –

Justin Garrison: Yeah, it’s a return on investment. It’s like “Hey, we’re investing in this person. We have a contract, and their time now should be making us money.” Andy, you were mentioning Disney… I don’t know if we actually have mentioned it on the podcast yet, but I used to work on the productivity engineering team at Disney Plus. And so this topic is near and dear to my heart. And so you’ve talked about platform engineering… How do you differentiate that from productivity engineering?

Autumn Nash: I mean, come on now. Not only are you not getting the return on investment you get, you now have to scrap projects and start over. We’ve had racist Twitter bots, we’ve had all these different things that have shown us “Hey, guys, we need to do more testing. We need to have more people in the room to have more opinions.” And now we want to put AI and data in everything when the main component to fighting data bias, which is diversity and diverse opinions, we’re getting rid of diversity. Is that not scary? Because I love AI, and I love data, but what are we doing to make sure it’s good for everybody?

Jack Shanahan: Really, all germane points, Daniel, and there’s so much to talk about just on that alone. I think I’ve now spent enough time, now that I’m out of uniform, I’m out of the Department of Defense, working with venture capital companies, inside partners in particular, and I get a chance to go and spend time with CEOs of everything from small startups to pretty big companies that are still in that venture capital business somehow, whatever seed round of funding that they’re getting. And the journey is remarkably the same. Actually, surprising to me… I would sit there and cry in my beer, so to speak, and say “Oh, nobody understands my problems”, and then one day I’m driving into the Pentagon, just despondent about how I’ll never get this thing called the JAIC built, because I have no people, I have no money… How are we going to get there?

[00:10:09.16] I’m listening to Guy Raz, How I Built This, and I was listening to the CEO, the founder of Belkin Routers. It was a fascinating story. It was exactly my story. He says “I built this thing in my parent’s garage, and there were days when I was just ready to throw in the towel and give up, and there were other days, 24 hours later, where I would suddenly - some technological breakthrough or some big contract that we got, and all of a sudden it looked right again.” O lived that. I said “Well, if he could turn it into that company, then hell, I can do the same thing in the Department of Defense.”

And I want to come back to something you said, Daniel, because it’s so incredibly important to this discussion. The technology, of course, was fundamentally at the center of what we were trying to do. But everything else was even more important to “How do you get to scale?” So I would say maybe the differences are - well, they’re used for military operations… But I would say that if you look at any big commercial company that was not built as a digital company, it was built as a hardware company in the industrial age, we have the same experiences. I’ve talked to enough people that say “Well, yeah, it’s the same thing we’re going through.” It’s getting that pilot project scaled and built in a way that you can then put it in one - for a company, let’s say for a medical or a financial business, it’s for that company. But for a lot of other places, it’s “How do I get this thing scaled to the rest of the world?” But in terms of industries, very similar to what we were going through.

And I talk about this all the time, I have eight things, and I don’t have to go through all of them, but they have nothing to do with technology… It’s mandate, vision, alignment, obstacle clearers, bureaucratic enablers, resources, authorities, and then sort of talent management. All of those are not technology-focused, but they have to exist, or you can’t get there. We all want to focus on “I have this cool new widget/gadget, and this thing is going to change the world.” Well, it might… But if you don’t have all those other pieces in place, which if you understand the bureaucracy known as the federal government, you have to have somebody that understands how to navigate in that world. Whatever I didn’t know about technology - and that’s a fairly long list of things at the time I started all this journey - I sure knew a lot about how to work in the federal government. I commanded six different levels in the United States Air Force, I had been in all these places where I had to do this day in, day out… So that part I was very comfortable with. I wasn’t nearly as comfortable with technology, so I had to – it was a hockeystick learning curve. No if, ands or buts about it. I still feel like I’m a novice today, by the way… And that’s seven years in the past when we started. It’s actually eight; we got started in 2016.

So I think there’s more similarities than there are differences, at least for the non-digital companies, because they have the same problem that a CEO is experiencing today, saying “Is this AI thing real? Should I really invest millions of dollars of this company’s money? What is the return on investment?” That return on investment question is an unsettled discussion, and I had lots of those with people very skeptical about what we were trying to do in the Joint AI Center.

Autumn Nash: I think that’s a really interesting article, because it’s not a way that I don’t think people think of when they’re building projects, or when they’re building maybe engineering teams, or just how they want to try to shape their goals for the year, and all of that… So I think it would be really interesting to see the return on investment of thinking that way. I wonder if people would be innovating faster, or innovate more because of the reduced cognitive load… And it would be really interesting to see if they could not only implement that, but then take metrics and see if they were more productive or not… Because it just seems like if you’re lowering the cognitive load, you could be more productive and do more with your time.

Nithya Ruffnithyaruff: We do GitHub Sponsors… We support foundations like the OpenSSF, Apache Foundation, Linux Foundation, Project CNCF, that sort of thing. And we have lots of developers who are behind the scenes actually contributing to projects. It’s never enough, because all of us consume a lot, so we have to keep working on that. And most businesses, not just Amazon, is challenged with business justification; why should I dedicate five engineers to doing this work? Because there’s so many competing needs. Customer needs, and product development needs, and so on and so forth.

[41:52] So we work hard as an OSPO and open source marketing team, who’s downstairs at our booth, to work with businesses, to educate them on why they should be involved, why they should contribute back, what’s the business case for setting aside people to do it… So those are the ways we help the business do more with open source. But we have to have a good business decision and argument, because businesses know business, and they need the return on investment or justification for why they should be involved.

Daniel Vassallo: But anyway that I could, without necessarily being very picky, very choosy. I don’t care about how sustainable it

is, or how high the return on investment is; whatever I can, to at least have some baseline income. And literally, the next day, after I went through this week of anxiety, I picked up my phone, I went to my personal contacts, and I found a friend who needed some programming help, and I started doing some freelancing with this friend. But the connections wasn’t a lot; I was doing like 20 hours a month, barely enough to cover my mortgage… But it was a huge, huge improvement in my peace of mind. Now I have some income. And I knew that if I wanted to, I could maybe find another client, or maybe add some more hours… It was incredibly liberating. Now, I wasn’t relying on this project succeeding anymore; it almost became like a bonus. Now, if it succeeded, it’s great. If it doesn’t, I’m not going to be desperate, as there’s already something there.

And immediately, things started to – one thing started to come after another. I was seeing these creators, the creator economy was flourishing, self-publishing, educational products, programming books, and whatever, and some of them making decent payoffs, some of them less so… And I said, “I have some technical content in my head. I worked at AWS for almost 10 years, I know certain products inside out… What if I do a brain dump of some of the things I know, and try to sell it online?” And this became another – now I call them another small bet. I spent like a month on this project, not a long time; brain dump of everything in a Word doc, minor editing, put it out on the internet… And within a few weeks, it made low five figures, and it was, again, eye-opening to me. Like, why am I bothering with this huge, ambitious project, highly speculative, that might never – yes, the upside might be huge. If all the stars align, I might make a big exit, or have a high income from it… But there are these small, much more achievable projects that are much more attainable.

And over time, I kept experimenting with different ideas, some things worked, some things didn’t, and I started to believe that this is a sound philosophy, and then, as I kept searching it more, I realized “This is nothing new.” The operators in this world, like book publishers, venture capitalists in Silicon Valley, movie studios in Hollywood, operate on this strategy, and some of them have been doing it for centuries, like book publishers, even VCs… I mean, not the modern kind, but investors in general, since probably the Middle Ages have had some portfolio of activities; they only invest small amounts of their funds, they never go all-in on one thing… They treat ideas – I like to call it treating ideas like cattle, not like pets, so you don’t fall in love with your projects… Yes, you want them to succeed, you’ll be disappointed if they don’t… But if they don’t, it’s just business; you move your attention to something else.

[17:57] And I started to see these parallels with how these industries operate, and I think many of their principles apply even to us as individuals. Yes, we’re not investing capital, unlike a book publisher, or a venture capitalist who’s investing their capital; we’re investing our time. But it’s equally scarce and precious, probably more so… And I think we can sort of, again, reason the same way as a VC does with their funds, but in our case with how we invest our time.

So what’s our time? We only get a few hours a day, that is disposable to our projects… How should we invest it? Should we go all-in for many years on one project? Is that a wise strategy? That would be like a venture capitalist going all-in with all their funds on one idea. It’s almost never going to happen, right?

Daniel Whitenack: Talk about return on investment, yeah…

Aaron Schlesinger: I’m not gonna give one that talks about higher-kinded types, because it’s been a while, first of all, so I don’t think I would do it justice, or maybe even get some parts incorrect… Also, it would take me a while. But I want to mention one pattern from FP that I think is really great, and it’s not that hard to do yourself in your code. I also don’t think it’s too hard to imagine how it would work in a codebase; it’s called the lens. It maybe sounds like an intimidating thing to some people. It certainly did to me. The lens is essentially a tupple of two functions. One function is a getter for some data, and then the other function is the setter for some probably the same type of data. But you can have separate things, too. You can have a getter for one piece of data, and a setter for another piece of data in the same tupple. So usually, it’s the getter first in the tupple, and the setter second in the tupple.

Now, the first function takes no parameters and returns the result, the data; it might take one parameter if the data is multi-dimensional, like a map, and you want it to specify a key… And that’s up to you; that’s part of the design pattern. And then the setter potentially takes in the dimension of the data, again, like the key of the map… And then it also takes in the data that you want to set into that place. So it might be the value of a map, if we’re going to use the map, like the example of a map, right? And then it returns nothing. These things can also alternatively return an error if this is doing something over the network or something that might fail. Traditionally, they’re not.
So if you could kind of imagine how these functions would be created, they would probably be closures. They would close over your data, they would take some parameter… Let’s take the get case, for example - it would take the parameter for the thing to get, and then they would return the value of it. Same thing with the set - it would close over the data and then do its thing.

It’s a fancy term called a lens, but it’s a fairly basic, foundational thing. And that is a common theme in FP, actually. So I call this the lens, and it is in Wikipedia, and it’s in the textbook, and everything… But it just boils down to functions, like most things in FP do. They just boil down to one or more functions, maybe combined together in some creative way, or maybe one function applied across a data structure in some other way.

[43:59] So I would challenge everyone listening to just give it some thought. Can that reduce code somewhere in your codebase? If you have a bunch of getters, can you get rid of those getters and just replace them with a closure, the lens? That’s a very interesting one to me, because it is exceptionally simple… I’ll put it a different way - the return on investment is very high, right? It’s very simple, but it tends to be applicable and useful in a lot of places in the codebase.

Adam Stacoviak: [29:56] I think it comes down to a lot of factors. When FireHydrant had its first round of funding I brought on two people I deeply trust as co-founders. So Dylan Nielsen, who’s our head of product, and Daniel Condomitti, who’s our head of engineering. And when you have – let’s call them like startup accountability buddies, that helped a lot.

Another thing that was big is - I knew the problem, I deeply understood the problem as an on-call engineer that was fighting fires for so long, which is why the company is called FireHydrant. What do you need to put out a fire? A fire hydrant. And I think that’s still what excites me, is that this is such a large problem, with so much return on investment for companies, that I just want to solve it. I want to solve it as much as I can. And I don’t think it’s ever truly solved. We’re always going to have incidents in our systems, but the impact of them, how long they last, and the morale impact and the internal impact of them… There’s just so many facets of incidents that are problems that we are setting out to solve. And that’s what keeps me going. And just the quality of the customers that use our tool today. It’s just gotten more exciting every day.

Gerhard Lazu: Yes. So in trying to connect the two themes, what you just shared about DevOps is 100% true and matches my experience as well. And where the analogy between the two struggles a little bit, is that if I want to have that git push Heroku experience and it cost me $30 a month, it is very difficult, I think, to be like a human who works on infrastructure and do literally any amount of customization or custom stuff, and compete with that on price. But because of the way that we consider the cost of software development - like, a lot of companies out there, as soon as you’re a full-time employee, your marginal cost on an hourly basis is $0. It’s like they become blind to just the actual expense of people’s time.

So I think of the failures of DevOps as being a failure to recognize the time sink that a lot of teams find themselves dumping lots and lots and lots of hours into when there’s commodity services that if you would only adhere to a set of conventions, would get the job done close to, or as well.

Testing is kind of like same core fallacy, is that we talk a lot about the activity and the importance of it in a sort of boolean state, like “Are you DevOps? Are you not DevOps? Are you in the cloud? Are you not in the cloud? Are you tested? Are you not tested?” It’s sort of like the degree of sophistication, because these are secondary concerns to building a product that does the thing that it’s supposed to do. No one really has the mental and emotional bandwidth to consider, “Am I DevOps-ing good? Am I testing good?”

So simply, there’s usually some, if not a person, like a mood in the team that’s like “In order for us to be a moral and ethical and upstanding team, we should be able to check this box or that box.” And so I want to check the box that I’m doing DevOps on and check the box that I’m testing. And when we consider the bad advice about either, it’s often coming from people who either are operating under that sort of simplistic notion, or for some reason have an incentive to enable and perpetuate it.

And so what I think about the failings of either are when the team lacks an appreciation for the overall total cost of ownership, the overall return on investment of where their time is going, and what are they getting for that time, or you know, in terms of AWS, or if you’re running a bunch of server somewhere to automate your CI build, and money. And if you appreciate that, then you can have a lot of really fun and interesting conversations about testing. How often are you seeing failures? When you see a failure, does it indicate an actual bug? Does it indicate somebody forgot to update a test? Is it brittle and flaky? Like, how long does it take to fix them? How many places do you have to fix the code or the tests in order to get back to a passing build? Like, how much time is lost in terms of the waiting to run the tests locally? Do you run the test locally? Do you run them in the cloud? And if you run them in the cloud, how long does it take until you get notified? And how many people get notified? Does the whole team get notified or just one person?

[36:00] And unless you know, in a quite data-driven way, the answers to a lot of these questions, general context-free advice that you see about the right way to run a test or the right tool to use is not necessarily going to help put you closer to the end goal, which is like the tests serve the team to accomplish what they were trying to do either better or faster.

Stevenson Jean-Pierre: This is where we as systems people run up against some frustrating brick walls whenever we’re in organizations… Because Kelsey is saying it’s about maturity, and it’s about “These are the things that ideally we should have in place and that we should do”, but when it comes down to the business perspective, if it’s quicker to reboot that system and get customers back in and doing what they need to do - we’re not talking about data corruption, we’re talking about just a purely access type of outage; if you reboot that thing and you know you could do a 2-second reboot and you get the customers back in, the business will be more likely to be like “Hey - yeah, just reboot it.” If this thing happens once a week, for whatever the business cares, they’ll probably be like “Okay, this happens once a week. Let’s go ahead and reboot once a week, and just take care of the customers.”

As systems people, we care about making sure that we’re solving this more holistically, so it doesn’t happen once a week. Because if it happens once a week, at some point it’s gonna happen once a day, and then once an hour, and then once a minute if we don’t catch it.

So having that business context and understanding that return on investment that we get for some particular fixes I think is important to answer your question around when that trade-off needs to be made and what influences those decisions.

We’ve all seen the type of bugs where we’re like “Well, we now have a cron job that restarts Tomcat, because we know we’re gonna eat memory…” And of course, the devs need to work on getting better garbage collection or better resource allocation when they are newing up resources… But as a stop-gap, you might have to do that restart, and as you build that maturity and understanding the problem with your platform, and as you give the devs time enough to figure out what that thing is, sometimes you do take that cluegy solution, but it is in an effort to save the business or in an effort to keep the business afloat while they’re doing what they need to do to make things better… And you build up that maturity over time. It doesn’t come from day one.

Kitson Kelly: The thing with package managers - we’ve sort of advocated a lot of our responsibility, of us as developers, of saying “Oh, because it’s on npm, I can trust it.” And that is wrong. A model that bugged Ryan, and it was one of his big regrets - it was a centralized authority… And it was a centralized authority that wasn’t open to introspection. Obviously, npm has changed, it’s been acquired by GitHub, but that doesn’t necessarily mean that they have to be any more open.

We hope that they are, and history has been pretty good in that space… And again, people who are looking for a return on investment based solely on whether they manage packages or not - there’s a much bigger ecosystem that they’re now a part of, where they’re not the sole source of revenue. So that’s a good thing… But still, why do we put that trust in there?

[31:48] At Deno, we always wanna explore the model where we get to choose who we trust… And we don’t have to be dependent upon it. Because if the npm registry were to go away today – there’s been, even a couple weeks ago, a silly mistake on a package, and half the internet goes down. How we avoid getting in that situation is not to put trust in a single entity like that, and diversify that trust. Deno just has no assumption, and this is what a lot of people have – Deno just doesn’t assumptions. If it’s a module on a website, Deno can access it, and download it and cache it. Now, that means it’s up to us who host the website to put any logic in there about how we resolve things.

So instead of having to sit there and sort out all of this on our local machine, across thousands and thousands of machines, in the Deno model it’s the server serving up the modules who sorts out this stuff.

There’s two CDNs that are really useful for Deno at the moment. There’s pika.dev/cdn - that’s probably the most useful one. So you get all this sort of semantic versioning that you would want out of npm modules, and Pika.dev CDN in particular has two big advantages. One, it only deals with ES modules, so it goes through all the npm packages, it looks for ones that have ES modules… And then what it’ll do is it’ll bundle it all up as a single ES module, so that you don’t have other dependencies inside that module, and you have the whole package there ready to go. Obviously, it was originally intended for a browser, but because Deno works like a browser for modules, it’s basically perfectly packaged there.

The other cool thing is – Fred, who is kind of the ringleader behind it, worked with us… And we were like “We have all of this TypeScript code out there, that’s been transpiled to JavaScript and bundled up, and we have all these other types that are out there available in [unintelligible 00:34:00.19] How can we sort of allow Deno to access those types and allow people to safely develop off of stuff in TypeScript on code consuming those packages, but not have the overhead of doing all of that transpilation, and take the advantage of the fact that it’s already bundled up?”

So what we added to Deno is a feature where if the remote server sends a header called x-typescript-types, and it has the content of a URL, Deno will go off and fetch those types, and use that in place of the JavaScript when type-checking any work that’s consuming that module or package.

So if you go to pika.dev and you find an npm package that you want, most of the time - and there’s always caveats with it - it’ll work under Deno. You’ll just be able to import it, you’ll get type safety… It works beautifully.

The other awesome CDN out there for Deno is jspm.io. The biggest thing is that it doesn’t serve up the type headers at the moment. But what it does is it takes all the other non-ESM parts of npm and packages them up into a bundle that’s compatible with browsers, and therefore it’s largely compatible with Deno, and you get a lot of other packages out there. You can React off of jspm, you can do server-side rendering in Deno with that, and there’s other stuff there.

There’s all these solutions out there, that provide this information, including the semantic versioning resolution of the URLs. Why would we need the package manager…? [laughter]

Chris Benson: That sounds good. Another thing that I think is happening already, it’s starting to - I see a lot of conversations, and I’ve been a part of a lot of conversations about this, is the fact that we’re getting to a point where instead of deep learning and neural network development being a separate little shiny object, with dedicated people that only do the modeling, you get to the problem of “How do you implement this in real life?” And you can build a great model, but then people in organizations really struggle to get it deployed into production, and getting a kind of a DevOps and feedback loop associated with what they’re doing and those activities… So I think you’re gonna see a lot of effort into moving neural network development into existing software development lifecycle and workflows that organizations already have in place, and they’ll make adjustments to those workflows to accommodate these new technologies. I think that’s really important, for them to see a good return on investment for their efforts in this space.

Jerod Santo: Right. Worth noting - there is a cost to decoupling, there’s a cost to making something modular. So that’s worth thinking about… Although through time and experience I can attest to the fact that it’s almost always worth it. There are times when it isn’t worth it, and that’s subjective, and like you said, it’s hard to quantify these things and come up with an equation for TCO.

Well, there’s so many factors, but I think just us developers thinking about decisions in terms of total cost of ownership and return on investment - these business ideas, bringing them to our software… I would just say - I’ll add one more thing and I’ll pass this to Divya - a huge win for open source is it’s a lot easier to calculate total cost of ownership when you can inspect the internals of your dependencies, and you can say “Well, here’s two modules that provide the exact same functionality”, and I don’t have to guess at their cost, because I can see the approaches, I can see the software inside of those things, and I can say “Well, this one’s well-factored, it’s a pretty simple, straightforward thing, it’s well-maintained”, so I know what the long-term cost of that one is likely to be lower than this other one, because I can see their internals… Whereas the proprietary software - you hit an API, a sentiment analysis API provided by a service provider, and you basically are going off of the reputation of the service provider, because you can’t see how they went about solving that, unless it’s also open source.

David Yakobovitch: Sure. Galvanize was founded in 2012. We’re one of the bootcamp providers for software engineering and data science in the United States. We have three segments of the business - a consumer, a remote, and an enterprise. I’m on the enterprise corporate arm, and that plays a lot to my previous skillset of helping other individuals learn tech in corporate.

Prior to being at Galvanize, I was at General Assembly, doing the same thing on the enterprise side - working with financial clients, scaling hundreds of individuals in organizations to reskill and upskill in the Python programming language, in Jupyter, in working with return on investment projects for their divisions. At Galvanize we have all those divisions as well. We are both consumer and enterprise-facing, and we’re all over the U.S.

I think what’s most exciting is there’s been so much growth happening in 2019, and we’re seeing that even into the next three years, predominantly because everyone is wanting to reskill and upskill, and code is now the first thing that people are picking up.

André Staltz: So there is a big benefit here for users that, “Of course, I pay much less.” And that’s actually the real reason why people are so interested in Google, is that, “It’s free.” From their perspective, it’s free. And then, of course, when I say that it costs $10 per user per year, people are paying that still; not in the form of dollars, but in the form of privacy, because obviously, Google is getting that money because users exist. So people are paying still something, but with their privacy. And not everybody is comfortable paying that.

So I think that in order to disrupt that, you have to be ten times cheaper… So you really need to reduce that cost of going from $10 per user per year to something like $1 per user per year, and you need to do that in a way that they don’t need to pay with their privacy -So how are you gonna do that?

I actually did the calculations, and what I receive now for Manyverse through donations - out of thousands of users, there’s 53 backers. So it’s something like one out of one hundred users becomes a backer. Those backers donate approximately $5 or $6 per month. If you do the math carefully, it’s roughly meaning that each user represents through some backer $1 per year. So you kind of reach that 10x reduction, but it has to be through the value proposition that you don’t have to pay anymore through your privacy.

These things are actually sustainable in the sense that if you have 10x less revenue, well then it means that all your costs should be like 10x smaller, 20x smaller. You can do that with a smaller team of developers, of course. You’re not gonna have hundreds or thousands, you’re gonna have dozens maybe, at scale. It also means that you shouldn’t have costs such as infrastructure, like servers… So those things are also consistent with the Manyverse situation that we don’t have servers running, we don’t have that kind of cost, and we just have developers - basically just me now - and it’s still not sustainable. I don’t have that kind of sustainability to have myself funded, and I’m the only developer so far…

[12:12] But the point is that at scale, these things are sustainable; if there’s $1 per user per year, then you do the math and with a million users you’re probably gonna have some kind of yearly millionaire revenue, which means like a team of a dozen people, or something like that. The case for scale becomes even more important once you’re going down with these costs. I think that’s the big gist of it.

It’s not a business because donations don’t have that kind of return on investment (you don’t have investors), and I think that’s the real key to the problem - donation-driven software, if it would be investment-driven, it would actually be much bigger, because investors would make it big. But that’s not the case.

Ashley Baxter: Yeah, absolutely. I think that if you listen to the stories about the successes, then if you are somebody who’s thinking of starting a business or have just started a business, you’re doing so under false pretense. For me, I’ve found that there’s so much work and so much time of not making much money, and there’s really long periods of putting in so much effort, and blood, sweat and tears and money into something and not seeing a return on investment. There’s years of that part, but people don’t really talk about that, which is what worries me, because we’ve got great websites, like Indie Hackers, which I love… All of these websites where it’s a bit too easy to read those glamorous stories of success and get a bit too caught up in that… So that’s why I think it’s important to get the story out about “Well, here’s how it looks for me, and it isn’t that glamorous.”

Zed Shaw: Yeah. Actually, I think one of the other reasons why you can’t really make open source is I think the imagination is you can just be a dude in your bedroom, crank out some code and you’re gonna get a job, or cash, or something like that… And I think no. I think now to run a very successful, large open source project is so expensive that no small group of people being underpaid, as a hobby, can do it.

I think you’re now competing with a Google who has 500 billion dollars and can hire 1,000 people to work on their fopen source project… And you are not gonna make a competitor to that. You’re not gonna be able to fork it, either.

If this is a project that takes 1,000 people, you and your three friends who hate it are gonna fork it? No, you’re not gonna be able to do that. And then what, you’re going to get a job?

I remember Google interviewed the dude who made Homebrew, and they didn’t hire him. They used his software. And he can clearly make a piece of software and manage a team, which is way more important than any algorithm’s knowledge… And they turned him down because he couldn’t reverse a black tree. I’m thinking “That’s insane.” That’s not the important part of making software. You can look that up… Like, “What’s next? I don’t get a job because I don’t know the name of every king and queen in England?” It’s just weird. I can look that up. Why would I bother learning that? I can look it up.

So for me, I think if – yeah, Kubernetes totally made the right decision… Trying to run a project that large without support, without money… And then, venture capitalists won’t invest in a lot of these projects, because if you invest money in a Kubernetes, that Kubernetes needs to have a return on investment.

Adam Stacoviak: So I guess the question I have is like if you’ve got this 200-million-dollar computer, it’s gotta be something that’s pretty demanding, right? People are gonna wanna use this thing, because you’re not gonna wanna not get the return on investment for that thing… So what’s it like scheduling, managing a project – how do you schedule time for it? Do you have to predict how long the project will take, the compute time…? Give us a day-to-day operation of using one of these computers.