Ubuntu Everywhere

Welcome back everyone. This is The Changelog and I am your host, Adam Stacoviak. This is episode 207. Today Jerod and I are talking to Dustin Kirkland about the rise of Ubuntu. Ubuntu everywhere. We talked about Canonical and Microsoft collaboration to get Bash on Windows. We also talked deeply about the history of this project, where Ubuntu is going, Canonical (the company behind it) and what we can expect in the future of this Linux distro. We have three sponsors for the show today - Linode, Rollbar and Bling Sale.

Our first sponsor of the show today is Linode, our cloud server of choice. Get up and running in seconds with your choice of Linux distro, resources and node location, SSD storage, 40Gbps network, Intel E5 processors. Use the promo code ‘changelog20’ for a $20 credit, two months free. One of the fastest, most efficient SSD cloud servers. It’s what we’re building our new CMS on. We love Linode and think you’ll love them, too. Again, use the code ‘changelog20’ for a $20 credit. Head to linode.com/changelog to get started. And now, onto the show.

Alright we are back, everybody. We have a fun show lined up today. Jerod, how exciting is it to have Bash on Windows? That’s what teed this up.

Not just Bash, as I am sure that we will learn the details of here soon. But yeah, absolutely. Big announcement out of the Build conference back in March, that Canonical and Microsoft have partnered in some way to bring the Bash shell along with other things natively on Windows, not inside of a VM or anything like that.

It’s more than that, too. We have got Dustin Kirkland here from Canonical. He has got quite the resume, too. He created the service command, and a ton of other stuff.

Yeah, thanks for coming on the show Dustin.

Hey Guys. Yeah absolutely. Adam, Jerod, good to speak with you.

So what’s the quick details, not the deep details - we are going to go into that later - but what’s the quick details on Bash on Windows?

The quick details is that if you’re running Windows 10 today and you’re in the preview mode, you can very simply type Bash and launch a shell where you are running Bash natively inside of Windows, and the userspace utilities are provided entirely from our distribution Ubuntu. And that’s pretty cool. It’s in preview mode right now, but Microsoft will release this generally to Windows 10, some time later this year, I think.

I am definitely interested in not just how that works, but also how this collaboration came together between the two corporations. But before we get to all that, Dustin, let’s learn a little bit more about you and your journey to where you are here at Canonical. Give us a quick background, maybe even how you got introduced to programming and computers, and then the summary of how you ended up working for Canonical.

Alright, well programming goes way back. I was programming in high school. Actually I started playing MUDs, if you remember MUDs, the multi-user dimensions in the 1990s, and I got pretty bored of playing MUDs and started actually writing some of the code behind some of the MUDs. It was in about 1994 or 1995, I was 14 or 15 years old at the time in high school. I went to college and got a degree in Computer Engineering from Texas A&M university, and then out of college I got a job with Tivoli in Austin, Texas. Then I spent about 8 years with IBM, and then in 2008 I left IBM and joined Canonical on the server team. [03:56]

At the time Canonical was building Ubuntu, and Ubuntu was fairly well known at that point as an up-and-coming desktop distribution, but there really was no focus or no technology behind Ubuntu as a server platform. So I joined a team of about three at the time. Our initial goal was really to get Ubuntu server into Amazon, into AWS, into its cloud. We did that in about 2008, and that really change the trajectory of Ubuntu as a server platform. Ubuntu is now about 70% of all instances running in Amazon, in Linux instances and Microsoft Azure - we will talk about Microsoft in a few minutes - Google Compute, across the board. Ubuntu is overwhelmingly deployed as a server platform in public clouds.

It’s an interesting background with regard to the switch between the desktop server, and the focus on server. Just to give you a bit of context of Ubuntu around the Changelog. I actually ran Ubuntu a while back - I was looking at you guys’ release schedule and the first release was 2004, and I almost hopped on right after that, because I had Ubuntu on my laptop in college in about 2004, 2005… I graduated in 2006, so somewhere around there. I ran it on my laptop for a couple of years - I always ran Debian on my servers - I ran Ubuntu on my laptop and then Adam has Ubuntu running on the Changelog servers, so he has always used it as a server distribution more recently.

I love it!

That great! Changelog servers are running Ubuntu. That’s awesome!

14.04. We haven’t upgraded yet.

Yeah, so about that time - the timing lines up pretty much exactly right. We were very much a desktop focused distribution starting with the first release 4.10, Warty Warthog in October, 2004. Then the server really emerged for us right around 2008, which would have been the 8.04 release, Hardy Heron. That would have been the first release, which was I guess a really server-focused release, and we haven’t looked back. It’s been really incredible to see Ubuntu adoption at tremendous scale, especially by this sort of new wave of technology-focused companies. Things like Netflix, Uber, Lyft, Tesla - all of these are platforms or all of these are companies that have built entirely on Ubuntu.

One of the things that Ubuntu is known for, aside from or in addition to its interesting orange color of branding, is your guys’ naming convention. It follows this alphabetical alliteration with animal names. I assumed it would have started at A, but it looks like back when 4.10 was out, Warty Warthog, and then you went H, Hoary Hedgehog, and then to Breezy Badger. It seems that maybe the alphabet didn’t quite kick in until you guys had your feet under you, or how did all that work out?

That’s pretty much it, yeah. The first release was Warty Warthog, second release was Hoary Hedgehog, third release was Breezy Badger, and then it wasn’t until the fourth release, Dapper Drake, that I think that we sort of realized that we actually need to have some method to this madness, especially for listing files and directories, where the directories are dapper, edgy, feisty, gutsy - it started to make sense when those were alphabetical. So yeah, you are exactly right. Our release cadence has always been every 6 months. We always release in April and October. We are now working on the Y release of Ubuntu, which is the Yakety Yak. It will release in October of 2016, which will make it the 16.10 release. So the two numbers in an Ubuntu release - the first number is the year, and the second number is the month. And yeah, we are working on our 25th release of Ubuntu and we have released everyone single one of them on time, every time. [08:03]

Wow! How do you get that done?

It’s a discipline that we called time-based releases. We sort of borrowed it and adapted it from Gnome, Gnome being one of the main desktop managers that Ubuntu is built on. Gnome started this time-based release cycle, just before we did in 2004, and we sort of took that to the next level and applied it across an entire distribution.

The real discipline is that we set forth the dates upfront, and then we engineer a series of milestones. We always have three alphas, two betas, one release candidate, and then a release. It’s really important that we identify in those three alphas the features we are going to develop. Then we institute a feature freeze, which can be broken, but it’s a very exceptional process to break the feature freeze. Then we go into the stabilization period, the two betas, followed by the release candidate. Now, anything that didn’t make it into the release or is not yet baked or cooked, we tend to move it out of scope. If something has to give, the scope, the feature list is what gives. The date never ever gives.

I’m googling ‘time-based releases’, you’re number one, so that’s a good thing, right? You got that down. So I guess, was it you that started it? Is this a thing you began or is it something that you adopted from somebody else’s practices, like “What? That’s actually cool. That’s a good way to do it”.

Probably I need to give a good bit of credit to the Gnome project, which is the desktop manager in Ubuntu. But as I said, we have certainly taken it, adapted it and broadened it to the entire suite of 20,000 open source packages that we snapshot and release every 6 months in Ubuntu. Also about that time, we were coming out of… We were built off of Debian - a lot of credit goes to Debian. We use Debian as an upstream for most of our packages. But Debian takes a very different approach. Debian releases when ready - that’s the mantra at Debian, and that certainly has the ability to produce high-quality software, but the release cycle is extremely unpredictable. And in fact, when Mark Shuttleworth founded Ubuntu in 2004, Debian was in a bit of turmoil. It had been several years between Debian releases, or a very long time between Debian releases. Our releases aren’t perfect, but they are very predictable, and we do as much as we can to get the most features as possible in a high quality release of Ubuntu, and anything that does not make it, you’re never more than 6 months away from the next release of Ubuntu.

So who is behind all of this? Who is the ‘we’ you are speaking of?

Who’s the ‘we’? So Canonical is the company behind Ubuntu. Ubuntu is the operating system, the Linux distribution. Canonical is the company behind Ubuntu. Canonical is 12 years old. It was founded at the same time as Ubuntu. Today we are about 750 people in 45 countries. Most of us work from home; I work from a home office in Austin, Texas. We do have a headquarters in London and a couple of satellite offices - Taipei, Boston, Montreal - but for the most part, about 80% of Canonical works from home. We are hiring, by the way, if you are interested in really awesome industry leading technical work, either on the client desktop side or the server and cloud side. Certainly check out Canonical jobs and send me a message. I am on Twitter and Linkedin and so forth, I am sure we can get those contact details.

You’ve got a lot of them there. It’s kind of interesting that you call them vacancies too, versus jobs. I guess you can kind of use both nomenclature but you call them vacancies.

Yeah, interesting. That may be a bit of our British heritage. [12:00] We have certain words that are kind of funny in our world that differ from Texas or California where a lot of technology comes out of. I should also mention Mark Shuttleworth, who is the founder of Canonical. He is the founder and owner of Canonical. Today he is the VP of product and strategy. He is my boss, I report directly to Mark. We have an executive structure that includes a CEO, Jane Silber, and a sort of an executive VP of the cloud business, Anand Krishnan. Mark, for the most part, leaves that sort of administrative side of running a company and focuses on his passion, which is really the product and the strategy, and he is extremely involved. He is involved every single day, seven days a week with all things Ubuntu. Very active on the mailing lists and accessible, very accessible.

That’s very cool. I think this change - you guys had this shift, your Gnome for desktop and you started with a Debian base and you put polish and improvements upon that, over time. I think there used to be two releases. There was a desktop release and a server release - correct me if I am wrong there - and… Well, let’s just stop there. Are there two releases still or am I imaging things?

No, we still do roll a number of different flavors - we call them flavors of Ubuntu. The server release of Ubuntu is what we publish to the clouds. I assume that is what your Changelog.com is running on, server Ubuntu. The server Ubuntu is a pretty trim and lean and mean and fast, small, secure Linux distribution. It doesn’t include things like a graphical display manager. Why? Because typically in the cloud you are going to do everything over SSH anyway. Let’s carve out the X windows and reduce the security surface and how much data needs to be stored and updated. The desktop release of Ubuntu is separate - yes, you are right. There are a couple of different flavors. There’s Ubuntu with Gnome and Unity as its desktop, that’s what I am running here today. There’s also KDE Ubuntu. There is XFCE, which is yet another sort of desktop manager. But we also produce Ubuntu for phones and tablets, which is really interesting. We’ve got Ubuntu shipping on three different platforms, from Meizu, which is a Chinese phone manufacturer, and BQ, which is a European phone manufacturer. And you can take Ubuntu and actually put it on any of the Nexus line of tablets. So we’ve got a Nexus 7 and a Nexus 4 phone here, both running Ubuntu as well.

What’s interesting about that is that we had to adapt - a lot of what we love about Ubuntu is still certainly present on the phone and on the tablet, but the way that we deliver the operating system, especially how we update and upgrade it and ensure that rollbacks can happen safely, as well as how we add extra software - all of that we had to rethink completely when started putting Ubuntu on phones and tablets. And in doing so, we actually came up with something pretty cool, and that’s yet another flavor of Ubuntu that we call Ubuntu Core. Sometimes you’ll hear it referred to by its codename Snappy - Snappy Ubuntu Core. Snappy is Ubuntu designed for IoT devices. So that’s really cool, because now we are seeing Ubuntu today running on developer boards like the Raspberry Pi, for instance. But those developer boards and Snappy Ubuntu are now shipping in refrigerators from GE, in drones, in robots, in printers, network switches - all sorts of super interesting IoT hardware; [16:01] we are seeing Ubuntu in that. The cool thing is that it’s a really interesting sweet spot, somewhere in between a desktop tablet phone consumer style device, but it’s a whole lot like a server, because many of these - a drone or a robot doesn’t have a keyboard video mouse traditionally, but it does have an SSH shell for instance.

I love this new focus - or maybe you could say it’s this lack of focus, but this desire, this vision to bring Ubuntu to all of these different places, beyond the desktop, now even beyond the server, too… You’ve reached the cloud and you are looking at IoT, or you have IoT deployments or distribution - or flavor, I guess, is the term that you use - and then the news that we are here to speak about in some detail was the conversation or the announcement with Windows and Microsoft that came back at the end of March. In fact, we know that our audience is interested in this particular topic because we’d linked out to that in weekly number 99 which was… April 2nd is when that shipped out, and I am pretty sure that Microsoft had their own announcement, and maybe Canonical had an announcement as well, but Dustin, we linked to your blog post announcing the Bash on Windows and it was the number 2 top story of that week, and I will tell you why it wasn’t number 1 - that’s because Adam included a rant from Giles Bowkett, titled “Why Scrum should basically just die in a fire”.

And you can’t not click on that headline!

Everyone’s gonna click on that one, right?

Yeah. I would click on that one as well. [laughter]

And you know, Giles is always very reliable for a solid and entertaining rant. So that was our number one story of the week, but this was our number two story, and one that we had lots of conversation going on around Twitter and lots of reaction to. So we are going to look at that in a little bit more detail. We are hitting up against our first break, so let’s pause now and we’ll talk about Bash on Windows when we come back.

Alright, we are back with Dustin Kirkland, talking about Ubuntu and the proliferation of the distribution everywhere, or at least the goal to, specifically in this case, the proliferation of Ubuntu onto Windows. And Dustin, we are quite interested not just how this works, but also how it came to be. So this is a partnership between Microsoft and Canonical and, as much as you are able to, can you give us the inside scoop, the backstory of how this partnership came to be?

Yeah, you bet. It’s funny sometimes in life how things never happen the way you plan them out to happen, whether it’s meeting a wife or a spouse, or having kids, or where you end up in a career, or set out on an adventure in a vacation and where you end up. [20:08] The real irony here is that Canonical and Ubuntu was founded with a single goal in mind, which was basically to be the desktop that replaces Windows. In fact, one of the tools that Canonical created in support of creating Ubuntu is Launchpad. Launchpad.net was sort of GitHub before GitHub. It was based around Bazaar, another source code tracker, but it was the home for open source projects, it was exactly how we build, it is the build system for Ubuntu, but it’s also our bug tracker. And if you go to bugs.launchpadd.net/1, the number one bug, the first bug filed in Launchpad, was basically that Microsoft Windows is the only desktop that someone can really use today, and we are creating a vision to change that. So it’s really funny, ironic, unexpected somehow that here we’re talking about Ubuntu on Windows and it’s a good thing. And let me explain to you why I think that’s a good thing. But does that actually resonate? Have you been in that situation where something happened, but it’s so far away from what you expected and it’s a pleasant surprise?

Yes, and I can say, speaking to your situation, I was pleasantly surprised and I think many people were, because I was very much… In 2004, 2005, I was running Ubuntu on my laptop because I didn’t want to use Windows on my laptop. So I was there, I was ready for the year of Linux on the desktop, as you guys were trying to build it, and so it’s quite ironic and interesting. It’s like an odd couple, Microsoft and Canonical, coming together for a collaboration. It’s not bittersweet, it’s just sweet, but it’s so interesting that this is what has happened.

Microsoft has changed so much though, over the last several years. We have seen a new Microsoft in open source, we have seen a new Microsoft approach to its products… I mean, is that part of it, Dustin? Do you attribute some of their change to open source in that?

Yeah, so that’s exactly where I was going with this, it’s that a lot has changed in the last 12 years. Canonical, we found this huge focus and success on Ubuntu as a server platform. We haven’t abandoned the desktop, our desktop is now better than ever and we are bringing it to mobile devices. But the cloud did not exist - in that name anyway - in 2004 and Ubuntu server didn’t either. Microsoft was led by Steve Ballmer at the time, and the thought of Microsoft doing with anything open source was just unthinkable. He called Linux a cancer and had essentially banned any Microsoft team from working with open source. But there has been a regime change at Microsoft. Satya has lead a real revolution at Microsoft in the adoption of open source. Over the last several years Canonical and Microsoft have worked together, most notably on Azure, Azure being Microsoft’s cloud. It’s sort of in the big three clouds of Amazon’s AWS, Google Compute Engine and Microsoft Azure, and Microsoft through Azure has been a tremendous partner of Canonical’s, and we have helped deliver the best Linux experience inside of Microsoft Azure. The vast majority of instances running in Azure that are not Windows instances, like 80% of the Linux instances are Ubuntu. We’ve worked very successfully with the Azure team in doing so, and that really fostered an interesting set of business relations, as well as - and this is extremely important - technical developer relations with Microsoft. [24:08]

Now over the last couple of years I have gotten to know a couple of developers at Microsoft, especially the Windows kernel team and the Azure team, and these guys are a lot like me. They went to college 5, 10, 15 years ago and universally, their college experience was much like yours, running a Linux desktop or certainly running a Linux on servers. The newer kids are running it on Raspberry Pis or in a data center on campus. That was absolutely my experience, I was at Texas A&M from 1997 to 2001. That was when I really went all in on Linux. I used it across the board, stood up web servers on Linux, used it on a desktop, and then I sort of made the move wholesale, in about 2000. Microsoft has totally taken note of that, that the developer environment, while it certainly used to be very closely tied to Windows desktop, has moved away from it. And to neither Canonical nor Microsoft’s benefit, a lot of that desktop environment has actually moved towards Macs. I think you can walk into a coffee shop almost anywhere, certainly in Austin or San Francisco where I spend most of my time, and in that coffee shop there is a developer with a Mac, writing some code, and that code is almost always inside of a virtual machine, in a Vagrant or a virtual box - a shell essentially, where they are running Ubuntu. And Microsoft actually commissioned a study to sort of understand what kids in college are using as their developer environment. One thread that was constant through that entire study was that regardless of whether someone was running Windows or mac or Ubuntu on their desktop, they were almost certainly doing their development in a Linux shell, and the vast majority of those Linux shells were Ubuntu Linux shells. It’s something that I think we have all seen, and whether or not Forbes or Wall street Journal or anyone has taken note of that, you could talk to almost any developer in the Valley or anywhere across the world in fact, and I don’t think any of them would be very surprised by that story. You know, most web development being done in a Linux shell isn’t exactly a headline that - you guys are in the media business, that’s probably not gonna grab a whole lot of clicks, right?

It’s kind of par for the course.

Exactly, and I think we took that for granted in the years that we developed and worked on Ubuntu, and we thought we were gonna take over the world - or we hoped that we were going to take over the world’s desktop. And while we did not do that, we objectively did not do that - there are certainly more Windows and Mac desktops out there in the world than Ubuntu desktops - we have done something pretty incredible in bringing Ubuntu, and then I am going to talk a little bit more about the greater open source ecosystem in a second… We have done something pretty incredible by making Ubuntu available across that entire hardware spectrum, from the Raspberry Pi, to desktops, laptops, tablets, phones, physical servers, your commodity x86 servers, into IBM’s sort of deep thinking power machines that run Watson, to some of IBM’s biggest mainframes, the Z-series mainframe, and then into supercomputers and high performance computing across the board. You can find Ubuntu running on all of those, and while there are, relative to the 7 billion people in the world, relatively few of those 7 billion people are running Ubuntu on their desktop, I’d venture that over half of the world’s population is touching a server running Ubuntu right now… [28:12]

Wow!

Netflix, Wikipedia, I mentioned Uber, Lyft, Tesla, Instagram, Snapchat - all of these are services running on Ubuntu servers. Ubuntu isn’t up front in the spotlight there, but it is critical to the operations behind this podcast, for instance.

Yeah, it totally is. I know that the CDNs we probably have the mp3s being distributed through run Ubuntu or Linux, for sure.

Yeah, I mean I’ve always just put Debian on my servers just out of habit almost more than anything else just through the years, because maybe even back when I started with Ubuntu it didn’t have a server edition, but I’ve always just thought Linux powers the web, but I’ve never really thought it about as… Do you know what percentage of that is Ubuntu? Has Ubuntu gotten so big that it’s in the 80%, 90% of Linus distros that are running web servers out there, or is it just conjecture?

So right now… This is public data from W3Techs. I’m gonna pull this up so that I can quote the number right… But just in the past month, Ubuntu has surpassed all distributions for the plurality. More web servers are running Ubuntu than any other Linux. We have been running neck and neck with Debian for many years, and by the way, we love Debian. Many Ubuntu developers are also Debian developers and vice versa. Canonical employs a hundred plus of Debian developers who work on both Debian and Ubuntu. It’s a very close relationship. We certainly don’t pit Ubuntu against Debian; there is a lot of collaboration, and we want to see that collaboration continue. But right now, W3Techs.com has been tracking web servers; basically, they just sniff the web, they ping the web server with a couple of queries and then inspect the response and try to guess what web server it’s running. Ubuntu is now running 32.5 % of the web, as of may 25th 2016, which is today. Debian comes in a close second, followed by CentOS as a distant third. But yeah, we are quite proud of that. In fact, if you look at the trend line, it’s looking very strong for Ubuntu, and almost everything else is in decline at this point. So I think your premise here is on, and it’s certainly that Ubuntu is definitely winning the web.

Why do you think it is? What makes someone choose one distro over another? Is it in, Jerod’s case, where it’s maybe habit or comfortability? Why do people choose a certain distro?

With Ubuntu, I think it starts with what we opened the podcast with, which was the cadence plus the velocity, the stability and the quality, these the 3 things. The first one, the cadence, that we release Ubuntu every 6 months, gives the developers really fresh libraries, compilers… All of your dependencies are certainly baked into Ubuntu and available; app install - almost anything that you could possibly want, and it’s already there. Maybe it’ll be there in CentOS, maybe it’ll be there in another distro, but it’s almost certainly already there in Ubuntu. The second one is quality - we put a lot of effort into ensuring quality, and that the quality of Ubuntu is on a trend line that’s moving up and to the right at all times, that we’re always getting better than the last release. [32:05] And I think it’s choice, the fact that we have given you basically the entire open source world, it’s at your disposal.

So let me bring that back to the question about Windows, Ubuntu on Windows or Bash on Windows. We set out to dominate, to win the desktop, and while we didn’t do that, we put Ubuntu on 100s of millions of servers and server instances, possibly even close to a billion. And yet we’ve still managed to find new and interesting places to bring Ubuntu. I was in – I guess it doesn’t really matter… It just happens that I recall exactly where I was when I received a phone call from my colleagues at Microsoft, saying: “Hey we have got something we wanna show you.” I was in Heathrow airport, trying desperately to get on the Wi-Fi so that I could stream the desktop share over Skype, and see the demo of what the team at Microsoft had been working on. Essentially, it is the technology underneath the Bash on Windows, the Ubuntu on Windows. It’s something called the Windows subsystem for Linux, WSL. The Windows subsystem for Linux is a layer that runs basically in or just on top of the Windows kernel, that’s able to run native Linux binaries, elf binaries compiled, running on Linux, eventually call out to the Linux kernel for system calls - things like open, close, malloc. It’s the fundamental technology, the fundamental interface between the kernel and user space, and so the Windows team created this layer that was able to intercept Linux system calls and translate them to Windows system calls, which is really cool. That’s not virtualization. Virtualization is presenting a whole fake hardware emulator basically, and an emulator is also similar to virtualization in that it’s really emulating an underlying system. Both of those, while they have gotten faster, are still inherently slow. What this sys call translation is doing is extremely fast and it feels very, very native. And that’s it. So the team brought to Canonical this technology that they developed, and they’d identified Ubuntu as being the user space that would be most attractive to most Windows users, and basically invited us to collaborate with them on this, so we started working on that.

I kind of led the effort from the Canonical point of view. We are essentially delivering the exact same, byte for byte, package for package, user space, user mode. User mode is what the Microsoft guys call it, maybe it’s a Microsoft term. I would call it basically the root file system, the same root file system that we test and publish to all of the public clouds for an Ubuntu server, that command line environment that you’re familiar with if you use Ubuntu in the cloud. We make that same root file system available in the Windows store, and it’s downloaded just as part of running the Bash command, essentially. So you run Bash the first time, it downloads that small Ubuntu image, loads that up and then at that point you’re sitting in a Bash shell, and almost anything that you would do on an Ubuntu Linux shell should work inside of it, with a couple of caveats; the Microsoft teams are working to close those gaps as quickly as possible. And then to your point at the very top of the show, it’s more than just Bash, and that’s the beauty of it. [36:03] It is Bash, that’s the way you start running the technology, but Bash is really ls and find and grab, and that’s kind of the end of the shell. Actually, we ship apt. Apt is the package interface, how you install additional packages onto Ubuntu, how you have access to 50,000 plus binary packages freely available on Ubuntu. So you can apt-install almost anything, and I say almost, we’re really not focused on graphical apps right now, this is really just command line apps, but still there’s tens of thousands of command line apps that are at your fingertips through Ubuntu’s archives, through the Ubuntu repositories. And that’s why it’s so much more than just Bash. Its GCC and Golang and Ruby and Python and Perl and PHP, and everything that you would normally do on an Ubuntu system, is now right there of a Windows shell.

That’s awesome. So let’s get this personal for a second, because you have your own little - I guess that could be condescending to call it little… You have your own open source project called - and help me out here, Byobu? Is that how you pronounce it?

Yeah, Byobu.

Byobu, which is a text-based window manager and terminal multiplexer. This is something you’ve been working on over the years, and is this something you can get up and running inside of Windows?

Well, so they are working hard on the underlying system call that it depends on. It needs to be able to allocate a TTY, basically, it’s just part of the underlying terminal technology, but it’s one of the top bugs, top priority bugs that the kernel team is working on right now. So it doesn’t yet work, but it is absolutely on the roadmap and certainly planned. It’s one of the most asked for features missing right now. So yeah, Byobu is a passion of mine. I am surprised and delighted that you asked about it, actually. It’s kind of a part time project, it’s certainly not part of my day job, and for the most part it’s implemented technologies; it’s really just bug fixes I work on now. But Byobu started out as a better version of Screen - if you are familiar with Screen. Screen is a beautiful program, but it’s kind of old and a little bit neglected, and then eventually tmux came along to be a better implementation of Screen. On top of both of those, I created this project called Byobu, which ships a set of opinionated configuration defaults and key bindings that turns either old school screen or the much newer tmux into a really rich command line windowing environment, essentially. It’s all command line, but it has a number of advantages. You can run commands and then detach and come back later, which is really important when you’re doing some work in the cloud and maybe you’re on a low connection or your connection drops, but you wanna make sure your long running command continues, or maybe you wanna kick off this job and then drive home and come back and check on it later. So that’s all very natively baked in, as well as the ability to split the screen - horizontal, vertical split, chop it up, run multiple windows… Sharing the sessions - that’s actually really useful. We use it for pair programming across the world. I mentioned that Canonical is very widely distributed around the world. It’s a very common mean for us to share a screen, so we’ll have an instance running somewhere in Amazon or Azure or open stack, and then we’ll run SSH import ID and we’ll give someone’s GitHub or Launchpad user ID, we’ll pull down their SSH key and then that person can SSH into that system and then with Byobu, we are both looking at the exact same terminal. [40:09] We are both typing and we can see what the other is doing. It’s great sharing a screen, at much higher resolution than say Skype of Google Hangouts screen share. You are literally in the same session.

That’s awesome. We’ll definitely link up Byobu in the show notes. That’s something that I’m sure our listeners are quite interested in. I am a fan of Screen/tmux myself. Old school screen user, recently switched to tmux because of some problem I had with the screen. Pretty pedestrian user of both, but nonetheless, awesome tools and anything you can do to put those on steroids is much welcomed. So check out Byobu if you are out there listening - it’s in the show notes. Let’s get back to Windows a little bit here. So Microsoft teed this up, and it sounds like they have done a lot of the heavy lifting for you guys before they reached out for the collaboration. From the time that you got that call in the airport to the point where we had an announcement at the end of March, give us a timeframe and the amount of work you had to put in to get this up and running.

Right. So Microsoft absolutely has done all of the lifting on the Windows subsystem for Linux. That’s all proprietary code baked into the Windows 10. It’s low-level Windows kernel stuff; Canonical could not have offered any expertise there, for sure. I certainly would not want to diminish the role that Canonical and Ubuntu have played in building the distribution, that root file system that we publish and we make available.

Essentially, it’s a really interesting meeting of two worlds. It’s this world of Windows desktop users that may know a little bit about Ubuntu, and then there is this world of Ubuntu developers that may know a little bit about Windows, but now being able to use the two within the same environment I think is sort of, I don’t know, a dawn of a new era almost, where you really are crossing the boundaries of a traditional desktop and the world of open source software.

In terms of timeframe, you saw the announcement, March 31st, at Windows and Microsoft Build was when it was announced. We worked on it… I worked around the clock for a couple of months, from basically January, February, March, intensely dedicated to that. We had a small tiger team at Canonical working on it, who were closely tied into the team at Microsoft. We worked very closely, I flew up to Redmond a couple of times, and it’s been a fantastic collaboration. We have really enjoyed working with that team.

I am curious, the future of Windows in this case, because in the back I’m thinking to Jerod, asking questions like “Will this ease the common pain of setting up dev stuff on Windows?” Like when you set up Rails, or anything like that on Windows, there is always some sort of glitch, always some sort of problem, and since Microsoft came to you all as the motivated party, they had done the research, they had done all this extra work to kind of even tee it up, and then chose you as the Linux distro to do it all… There’s got to be some future plan there, and maybe even some reduced pain in terms of dev environments and people who want to use Windows but still be developers… As you mentioned, those in coffee shops, using Macs of some sort of flavor of Linux to work on might be the case.

Let’s take a break real quick. We’ll come back and we’ll start talking about the future and maybe even some loss of pain for those developers that are on Windows. So we’ll break now, and we’ll be right back.

Break: [43:56]

Alright, we are back from the break. We’ve got Dustin here and we’re talking deeply about this motivated party called Microsoft, who needed, wanted, desired, built the technology to put Bash/Ubuntu on Windows, and it just seems that they just came to you so motivated, and I am curious about what your thoughts are on why they had this desire so much. Obviously they did the research and if there’s a developer doing something on a Windows computer or any other computer, they are going to be doing something in a VM or some sort of Linux distro to do their development, but how does this play out to the future of Windows? What do you think Dustin?

The motivation I think is pretty obvious, particularly when and where Microsoft announced it, at their Build conference. The motivation is to ensure, from Microsoft’s perspective, to ensure that developers choose Windows, use Windows as a desktop platform. Making the Bash command immediately available and bringing into the Bash command the ability to use your space, all of the Ubuntu tools and libraries, was sort of the missing piece. Just having Bash alone… You know, you could have a BusyBox or something, but BusyBox, which is just basically Bash statically compiled, isn’t enough for a developer to use it. Having that entire Ubuntu user space apt installed, you mentioned Ruby, and then Ruby libraries, and then from Gem install whichever libraries - it certainly brings a very native development experience to a Windows desktop. Writing code in Visual Studio is fine and dandy, but a typical Windows developer would then need to git commit that and then push that, and then launch a VM or an instance in the cloud to actually test it.

So one of my colleagues at Microsoft who’s been super vocal about this, sort of a peer product manager at Microsoft, is a guy named Scott Hanselman. Extremely active, a great blog, and very active on Twitter. Scott has been quite vocal about how he is using this Ubuntu on Windows to do Ruby development, Rails development, Mongo, and all natively on his Windows desktop. Across the board, I’ve met a number of people who in some cases are relieved, thankful, appreciative of this work, in that in many cases they’ve got a company-mandated desktop, which is a Windows desktop, and they are so pleased with the ability to have that world of Ubuntu and open source natively available in a command prompt at their fingertips.

That resonates with me. I think once I found the Linux shell and was exposed to that in college, there was just no way that I could go back to command.exe on its own, even with Powershell and these other tools. It just wasn’t the same, it didn’t feel right. [48:02] So that has kept me away from Windows honestly, as a developer, for a long time.

Let’s talk about the practical use of this. Now that it’s still in beta and all that, and I’m sure there are things that are being worked on and bugs to iron out and system calls that aren’t quite bridged yet, or whatever the term we use for that layer is, but what about practical things where the two operating systems or the two environments meet? You might find an uncanny valley at times; I’m thinking of like system paths, file naming conventions. Well, first with this - is the shell isolated inside your environment or is it exposed to the Windows file system?

It goes both both directions, actually. The Microsoft team did a really nice tasteful job of this. In the Linux shell itself, if you cd to /mount/c you can see your entire Windows file system. You can see all the files and directories, you can vi a file, you can edit a file, you can touch create, you can remove files. You can do anything that you would be able to do natively as the user who is logged into the Windows shell. And in fact that goes both directions. You can also open the Windows explorer. I have forgotten the exact path, but you can navigate to the entire Ubuntu shell and do the same thing. You could open up a C file in Visual Studio for instance, or create a directory. It’s actually the same file system in both directions.

Very cool. Now, you admitted in your announcement blog post that even this announcement is odd for you… Or not the announcement is odd, but the using of it is odd, because you hadn’t used Windows in nearly 16 years, which is kind of funny. But for people who have, like Scott Hanselman and these other people who started to like use it for Ruby development or for whatever happens to be that they’d normally be using, either a virtual machine from a Mac or using a cloud machine, or even just a Linux desktop - have they seen the uncanny valleys, or is the grass really greener when you bring the two together?

I think it’s clearly greener. Find, grab, xargs, sed, awk - all of that right there, it’s not sigwin, it’s not a virtual machine, it’s a operating within the file system.

I’ll sort of recount the anecdote I shared in my blog in that it’s so poignant and it’s so literal and true to actually build… So we take the Ubuntu image, that root tar ball, and then we have to repack it into an appx file, which is basically just a zip file wrapped around some metadata. And to upload that to the Windows store I needed to use Visual Studio, and I am not lying when I say that I haven’t used Windows as a desktop since Windows 2000, since the year 2000.

So I installed Windows and that was an interesting experience, and then to build that appx, I had to do a bunch of work in Visual Studio. Visual Studio has a tremendous reputation, it’s this huge environment, and I think once you’re comfortable with it you can probably use some incredible things at a very large scale. Well guess what, I really didn’t need to do anything at a large scale; I needed to make a couple of copies of a couple of files, some metadata. I needed to replace all instances of ‘sampleapp’ with Ubuntu, essentially. And I sort of fought and struggled, clicking my way around Visual Studio until at some point I just decided, “Hey, you know what? Let me actually just try this.” [52:03] So I dropped to the Bash shell, I cd-ed into mount/c/users/settings/dustinkirkland, my source directory. I found the directory that I was working in, I renamed the directory using the move command, I did a deep a find, find . in that directory, pipe grab for “sampleapp”, found all the files I needed to edit, piped that through xargs and renamed all those files in a fraction of a second, and then I needed to do a deep grab through the data in that directory for all instances of the word “sampleapp”, or whatever it was, and then do a sed replace; another grab, another pipe to xargs, another sed, and I replaced all of those instances of “sampleapps”, and then I flipped over, alt-tab back over to Visual Studio, reloaded and hit compile, and it just worked. It was really neat to do what I was comfortable with - find, awk, grab, sed, xargs. Then, for various reasons, I needed it to be in Visual Studio, in Windows, and those two really worked seamlessly. I didn’t have to ftp some files around or git push or scp or copy. Speaking of scp, the other thing that was neat was when I needed to SSH from that Windows machine to another Ubuntu machine, and I can natively run on that Windows system, directly to that local filesystem, rsync, scp, SSH, without setting up PuTTY or some other third party tool or going and fetching another third party tool.

That does sound like pretty green grass. It sounds pretty good. I think that this could be a huge boon for open source projects, especially when the burden of maintenance between multiple operating systems is such a problem for many developers who even would love to support Windows users; they don’t even have environments that they can test against Windows in order to provide support. So it’s often a pain point. People, when they are calling for help, it’s like, “Please if you have Windows or if you are on Windows and you use this, provide help. Give us good bug reports or even help us fix these things.” Do you think it’s feasible that once this comes out and is out there in the wild, readily available for developers, that it would be socially okay to say, “Our Windows support is via Bash on Windows or via Ubuntu and Windows, and we don’t have any first party support for the operating system”?

That’s a leap. [laughter] Possibly… I should also caveat this with the fact that Microsoft has said that this is Windows 10 desktop only. They have not announced any plans to bring this to Windows, the server, so running a rails web application or some sort of server application, at least at this time, is not natively supported. There is no commitment from Microsoft that that’s even coming. But let’s relegate your question to desktops… Possibly. I’ll tell you this, the next time someone asks me to help fix their Windows machine, traditionally I just plead….

Don’t use it!

Yeah, exactly. The next time someone asks me to fix their Windows machine, I will actually fire up a Bash shell and see if I can poke around and figure out what’s wrong maybe. Now, there are limits to how much of the Windows hardware can be seen from Linux. For instance, direct access to some of the storage drivers or the video drivers and stuff is still going through a Windows layer. [56:00] But yeah, I think there is an opportunity to certainly make, I don’t know, scanning the file system, backing up data to some of the tasks that are strictly file-driven, that we’re really good at doing things against file descriptors in Linux - I think some of those tools may make it over to Windows in a hurry. I think it’s a huge opportunity.

I don’t know if this is on your radar, Dustin, or not. I imagine it probably is, but I think about what kind of developer typically is running Windows, and if you look at the landscape of available, consumer desktop, laptop machines out there, you’ve got various flavors or Windows, you’ve got various flavors of Linux, and then you have Macs out there. But typically Macintoshes are more expensive, and universally, Windows tends to be less expensive machines. So by doing this, it seems to me like you’ve… We always talk about the accessibility to software, to being a developer, to learning to code on this show, and we have talked about that on the last several shows, with Sara Chipps with JewelBots and things like that, bringing access at a younger age… It seems to me like I could buy my daughter a Windows machine that runs Windows 10, pretty expensively, and she has access to everything I would want her to have access to, to be a developer, with this new access to Bash and the shell in Linux and Ubuntu. That’s awesome. It’s now become a much lower barrier with still great tools.

I think it’s amazing how many more people open source software will touch through this development. I think that’s the most amazing part about it. The internet is full of all sorts of opinions. I saw a handful of sort of to-be-expected negative opinions of “Why would Canonical do this? Microsoft’s the enemy. Don’t we build good desktop environments? Why would you do this?” and to most of them that I had the opportunity to reply to, I really tried to speak to their heart and say, “Guys, your love of open source… Just think about how many more people open source can touch, how many people will be exposed to GCC, Perl, Ruby, Shell…” The fact that that’s now available on Windows desktops or will soon be available on any Windows 10 desktop - we have made the entire open source world of software - and to their extent - our mission, our somewhat religious mission that open source is just a better way of doing things. I still believe that in my heart.

I dual boot my laptop now to demo Bash on Windows but I still run Ubuntu as my primary desktop and I still develop open source software because I love it. I am just thrilled that over the next five, ten years, that all of this open source software that I have written over the last 20 years will be available to even more people than ever before.

I love the sound of all that, by the way. Quick question on limitations before we get to some of our closing questions… You mentioned there’s certain system calls that aren’t quite there yet; obviously Microsoft is not too interested in Windows X, or Window managers, or graphical applications coming from Ubuntu over… You also mentioned that it’s desktop-only, so server deployed things, which I am not sure why you would want to do that anyways, because you could just deploy it to Linux server, but I guess via the existing infrastructure that is based on Windows licensing or something… What are some other limitations that either are going to disappear or they’ll just kind of always be there in this environment that you know of? [01:00:06.02]

Yeah, let me clarify a couple of the first ones first. So I think we both, Canonical and Microsoft, and yourself as a potential user, we all understand that bringing this to Windows server doesn’t make as much sense. If you’re in a Windows server, you’re probably a power shell user anyway, which our shell is great and amazing work as well. And if you’re really gonna run a server workload, by all means, please run it in Linux and ideally, from Microsoft’s perspective, that’s a Linux running in Azure. That’s a Linux running in a hyper v. We all kind of looked at that and went, “Yeah, if you’re doing it that way, you’re kind of doing it wrong, so let’s not focus a whole lot of effort onto Bash on Windows servers.” That’s sort of the color behind that first one.

The second one you mentioned, or the other one you mentioned, was about X applications. What we call in Linux X applications are graphical applications. Believe it or not, a super motivated - and this is just the beauty of open source; kids will be kids, I guess… There’s actually X Windows applications now running on Ubuntu on Windows, believe it not. I have myself launched Firefox - now, Firefox is a silly example because you can run Firefox natively on Windows, but I have actually run the Firefox binary from the Ubuntu archive, exported on the same system, over a local export display calls local host, to the local machine… As well as a couple of other minor x applications. That’s using an open source project called Xming server. It’s basically an X server for Windows. So you go out to the Xming server website or GitHub, you pull down the .exe binary, install the X server on Windows and then run that, and then you can apt-install some X applications.

Some things still don’t work, and most of them come back to basically the same system calls we know aren’t currently working. D-Bus right now is sort of the low-level message bus, it’s got some problems. For instance, I wanted it to run Gnome terminal, because I hate the Windows command.exe terminal. So I wanted to run the Gnome terminal or the Terminator or any one of the X terms from Ubuntu, and all of those kind of depend on that D-Bus, which is related to the other things that are still not yet working. But the team at Microsoft working on this, they are fully- they are well-staffed, it’s smart guys. This wasn’t a proof of concept that gets announced and then abandoned; I’ve got every bit of confidence that this is getting better and better over time.

So we’ve all had the Bash on Windows or Ubuntu on Windows announcement. The next question is when are we going to hear about the Windows on Ubuntu announcement?

[Laughter] That’s kind of… Some people describe this work as the reverse of Wine. If you are familiar Wine.

Absolutely.

Windows emulators - it’s kind of how you sort of run some Windows applications on Ubuntu. I know that Wine has saved my bucket on a couple of, I don’t know, updates to firmware… I updated my firmware on my Plantronics headset using Wine not too long ago. So yeah, this is sort of the reverse of Wine. I’d argue that without this… Wine is not obviously Microsoft-driven, but Windows emulation on Linux has been around, and while not fantastic, it’s certainly functional for many cases.

Right. But it’s always been emulation, whereas this seems like it’s a little bit more native. [01:04:02.00]

Yeah, indeed. So we are not working on that right now…

[laughter] Yeah, I know… It was a joke.

It was a joke, pal. It’s like, “He’s taking it seriously.”

[Laughter] Sorry…

That’s good though, you were on point. That’s what we want, we want that. This is a serious closing question though, so don’t take this one as a joke, but there are a lot of listeners tuning in… You know the numbers, users of Ubuntu, lovers of Ubuntu, and I’m kind of curious for those out there who care about this mission, this mission of not so much just using Ubuntu where it’s at, but enabling Ubuntu on Windows even better. What things out there, what ways could people or the open source community step in and help push this mission along with you? How can they step in?

Yeah, so there are a ton ways to contribute to Ubuntu. I got started as a contributor to Ubuntu before I became an employee of Canonical, it’s just sort of my background in open source. There’s a way of doing open source and that’s great whether you are young or old or anywhere in between. Open source community is a brilliant way to learn and get better and contribute to society, to the technical world. It can start as easy as reporting bugs. It’s something that myself as a developer, I spend hours every day reading bugs, replying to bugs. Reporting a bug, a good high-quality bug with how to reproduce these instructions, not just the “this pissed me off and I hate you”, but like, “Hey, here are my logs. Here’s what happened. Here’s how to reproduce it, I’ve tried it on two computers”. That’s a huge contribution to open source and requires no programming skills necessarily. Contributing to the mailing lists… We have a Stack Exchange channel, it’s actually the third most popular channel on all of Stack Exchange, it’s called askUbuntu.com. You can get involved asking questions or providing responses and voting up or down responses. That’s a great way to get involved that again requires a little bit of technical interest, but not necessarily technical expertise. We’d love to see more questions and answers around Ubuntu on Windows at askUbuntu, of course.

Then from the developer perspective, man, the possibilities are endless. From contributing to source code on GitHub or Launchpad… You should absolutely have accounts on both GitHub and Launchpad, try to follow the projects you are interested in, branch or fork the projects that you’re interested in and provide patches or contributions, or even just using it and providing feedback is an incredibly valuable. Especially for your younger listeners, it’s a great way of building out a resume, long before you even create the resume. I’ve hired dozens of people who barely have a resume, but they have a GithHb profile and a LinkedIn profile that show me what they have worked on. Your code comes across as more important to me than a ten-page resume or a huge CV, or something like that. It’s really the way to get started. I’ve hired interns, I’ve hired new hires straight out of college, I’ve hired high school kids that just know what they are doing and they’ve contributed to open source, and their contributions are very public and visible.

I’ll end with one last thing. I’ll say that this is - you asked about some the reasons why Ubuntu has been successful. I’ll tell you one that’s at our core, and it’s one that doesn’t get noticed and in fact that’s kind of a good thing. At the core of Ubuntu we have this thing called the code of conduct, the ability to code of conduct. All Ubuntu developers are required to sign it and it really governs the way we behave with one another. [01:08:05.00] While there are arguments, we tend to focus on technical fights as opposed to personal fights. You can attack the code, but never attack the person, and it’s so important; it’s really part of the friendliness of the Ubuntu ecosystem. I would say that whether you’re working in Ubuntu or the Linux kernel or anywhere, treat one another well, treat one another like humans, because all of that code that I just said is public and has become part of the public record, your actions on those mailing lists are absolutely part of the public record as well. So treat one another well. It’s important.

On the note of open source, we are commonly asked this question, and you might have gotten it in the email when we teed up the call and prepped you for everything, but we’d like to ask, especially someone like you who comes from a company that leans on and supports and cares so much for open source - I imagine you have got an interesting radar, meaning if you had a free weekend, what’s something that you haven’t played with or you’d love to play with that if you had a free weekend and nothing else to work on, this would be something that you would hack on, what would be that for you?

I’ve got a lot of nice weekend projects, I’m working on two right now. One is an Alexa skill sets… I recently bought an Amazon Echo. “Alexa, play Fish on Pandora”, and while she is playing the Fish radio station, “Alexa read me the news”, “Alexa, what’s the weather?” I am actually working on an Alexa skill for man pages. I have something that I wrote a long time ago that actually parses all of the Ubuntu deb files every night and looks at any that have changed, and if it finds one that has changed, it cracks open that deb, extracts any man pages and then publishes them, renders them to HTML, formatted and linked and everything, and it publishes them at manpages.ubuntu.com. As you can imagine, I’ve got a wealth of man pages there. I am actually working on an Alexa skill where you would say: “Alexa man SSH” and then Alexa would come back and read you the first line of the SSH man page and say, “Hey, would you like to know more?” Is that useful? I don’t know, but it’s useful for me to learn how to write an Alexa skill, to sort of get into that AI, machine learning. Yes, that’s what I am looking to learn.

The second thing is closely related to that… Because I have this nightly job that cracks open every deb published to the Ubuntu archive, it’s very easy for me to extract other files from that deb. So I have another side project that’s also in that sort of machine learning space, and I say it’s pertinent to you guys because every package in the Ubuntu repository has a file called “Debian changelog”, which is the history of that Debian package. Some of those packages got back to the early ‘90s. In fact, some are more recent than that, but they follow a very consistent, parsable format. Part of that nightly script that I’ve written also indexes the changelogs, and right now I am rendering that to basically a huge blob of JSON that I can then import into MongoDB or Postgres. I guess the part of the project that I need a night or a weekend to work on is actually developing some intelligence, deriving some intel from the changelogs. Who’s changing what? How often is it changing? What things have been abandoned, obviously, haven’t been touched in a long, long time? Those are sort of the interesting questions I’d like to ask that SQL or NoSQL database. Makes sense? [01:11:58.08]

Makes sense. It’s interesting, the Alexa piece, the skill learning… I don’t know if I would use it, but I could see if anybody out there… I guess it could be used. I would talk to Alexa about man pages, that would be cool. [laughter] What about you, Jerod? Do you care about that?

Asking Alexa to RTFM, that’s worth it right there, right?

It’s funny you say that… As you close the Alexa skill, she thanks for reading the friendly manuals. [laughter]

Nice.

I don’t know, Adam. I think we have a kindred spirit here with regards to changelogs. I feel like that data dump could be very interesting for us.

Yeah.

It’s one very large data dump of literally changelogs.

We are 8 minutes past the show time for this show. Anything else you wanna say in closing Dustin?

I did want to touch on the business side of Canonical. I’d love to share our experience about what Canonical is as a company, and tee that up with our listeners, and some of them building businesses or interested in building a business around open source - how does Canonical do it? That’s one that I’d like to have two or three minutes to talk about.

Let’s do it. We are always curious about that. I mentioned in the break earlier we were in a pull into this show, we just didn’t find time too, because we were kind of talking about necessary things, but I am curious with a staff, like… Is it an actual staff of 750 people all across the world? Are they all paid employees? And I’m really curious how you sustain and make money.

756 employees across 46 countries. Paid employees. And a much bigger open source community of friends and foes that write open source software that filters into Ubuntu.

Friends and foes… So what’s the story behind the company side of things? How do you make money?

That’s a good question. I mean, it’s extremely important to the long-term sustainability of Ubuntu. The success of Canonical is very importantly tied to Ubuntu being continuing to be available to everyone in the world. Canonical itself has a couple of businesses, a couple of parts to the business. First of all the company is really split into two halves. One half works on laptops, desktops, phones and devices, now sort of IoT connected devices. The other half of Canonical works on servers and clouds and virtual machines and containers. But the two work… We share a lot of technology. Our businesses are a little bit different.

At its core, Canonical is a services company. We offer suites of services that surround Ubuntu. We don’t charge for Ubuntu. The bits that make up Ubuntu on your laptop or desktop or in the cloud are always free to you; you’ve never paid for Ubuntu itself. But there are many enterprises and organizations and some individuals that choose to buy auxiliary services from Canonical. And those services, at its very core, often start with support services, so we do have commercial support where we try to help ensure large institutions, typically, that have built a business around Ubuntu and they heavily depend upon Ubuntu. We can try to help ensure their uptime, the security of their packages. They have a phone number to call, they have an interface to connect with our support engineers, who can then help troubleshoot or debug maybe a kernel performance regression, or a security issue, or something like that. That’s our support services.

Then we also provide consulting services. Some of the biggest telephone companies in the world - AT&T, Deutsche Telekom - are building their data center infrastructure services around Ubuntu OpenStack, and we engage with them at a very large scale to provide consulting services where we help design and implement that cloud. [01:16:12.28]

And then we have a large number of customers where Canonical provides what we call level 3 support. So that customer might take the first and second phone calls from the customer. IBM is a good example. IBM has its own customers, let’s say the financial industry, Wall Street or something like that, where they’re using Ubuntu. They call IBM for the first level and the second level of support, but when that IBM engineer needs help from a Canonical engineer, IBM can then pick up the red phone, call Canonical and receive first-rate Ubuntu support from an Ubuntu expert. So those are sort of the support services.

Then we also do… Let’s call it OEM work, or work with OEM, which was originally hardware manufacturers, ensuring that Ubuntu, as it ships on Dell PCs and HP servers and IBM servers and lots of others, now network switches, as Ubuntu ships those devices, we help that engineering team build the image of Ubuntu that runs on that switch or on that server. But we’ve also seen that same model work very well with the major public clouds where Canonical curates, secures, provides tests, QAs, the images in those clouds. So all of those are commercial engagements that help sustain Canonical, helps pay the salaries of the 700 of the 756 of us working all around the world.

That’s amazing. I think it’s interesting to think about Mark too and his past, to get the gumption to even think about Canonical and how that can benefit Linux and ultimately benefit open source. Certainly a longer topic for another show which we might call on you for, who knows. We’re reviving and relaunching Founder’s Talk, which is a show I did that kind of peeks into the life of an entrepreneur, a founder to look at what they did, why they did it, the fails the successes, all of those things. It would be interesting to talk to Mark about the deeper details if he’d be game, for sure.

Mark would regale you with stories from the International Space Station, to his trip to Antarctica. After you go to space, where do you go next? He took a three-month trip to Antarctica where he printed out a thousand pages of Debian mailing lists and he went through the mailing list with a pen and a highlighter and circled a bunch of names, which were the first 20 people that he contacted and said: “Hey, I’m an African billionaire and I want to start a company. Come help me make Ubuntu.”

Wow…

I love this guy already.

Yeah… I wanna have that conversation. It makes me want to start Founder’s Talk again, right away. Soon though, very, very soon. We have a lot of things happening here at the Changelog to make all that happen. Very soon, listeners, if you are a past listener of Founder’s Talk, it’s coming soon, and that conversation with Mark will probably happen very soon. But we are out of time for this show.

Dustin, it was an absolute pleasure to deep dive into this Linux, Ubuntu, Windows history and future, and the new abilities and access to so many the developers out there; everything from Linux to open source is available now. Thank you for all the hard work you do and to the listeners, we thank you as well for tuning into this show. That’s it for this week, fellas, so let’s say goodbye.

Goodbye! Thanks, Dustin.

Thanks, gents.