Kurt Mackey is back for a deep dive into what it takes to build the developer cloud. Kurt joins Adam to discuss the alliance between companies and cloud, something Kurt refers to as the “Rebel Alliance,” cloud complexity vs usability, Fly’s future with Postgres and why they’ve waited, thoughts on Neon and Supabase (Kurt shares a hot take), and our CDN saga and plan to build a simple CDN on Fly called Pipely (still a Pipedream).

Adam was out when Bryan made his podcast debut here on The Changelog, so we had to get him back on the show along with his co-founder and CEO Steve Tuck to discuss Silicon Valley (the TV show), all things Oxide, homelab possibilities, bringing the power of the cloud on prem, and more.

Service Weaver is a programming framework for writing, deploying, and managing distributed applications in Go. With Service Weaver, you write your application like it is a traditional, single-process Go executable that runs on your local machine. Then, you deploy it to the Cloud, and the framework breaks it down into a set of connected microservices and integrates it with the cloud provider (e.g., monitoring, tracing, logging).

The rough math goes like this: We spent $3.2m on cloud in 2022. Just under a million of that was on storing 8 petabytes of files in S3, fully replicated across several regions. So that leaves ~$2.3m on everything else: app servers, cache servers, database servers, search servers, the works. That’s the part of the budget we intend to bring to zero in 2023. Then we’ll worry about exiting the 8PB from S3 in 2024.

David spoke at length on this decision when he was on Ship It! late last year.

StackQL allows you to create, modify and query the state of services and resources across all three major public cloud providers (Google, AWS and Azure) using a common, widely known DSL…SQL.

I have to admit, it does look pretty darn simple:

SELECT * FROM google.compute.instances 
WHERE zone = 'australia-southeast1-b' 
AND project = 'my-project';

tRPC helps you move fast and break nothing, Michael Muré embeds a bug tracker in git, Fatih Kadir Akın curates some awesome ChatGPT prompts, Mike Nikles thinks dev environments in the cloud are a half-baked solution & Georgi Gerganov ports OpenAI’s Whisper model to a lightweight, portable C/C++ program.

Valerian Saliou’s Sonic search backend, Brandon Willett on how to build software like an SRE, DHH on why they’re leaving the cloud, Amos’ HTTP crash course nobody asked for & Sam Magura tells why he and the Spot team are breaking up with CSS-in-JS.

TLDR: AWS is super-expensive at scale.

Of course it’s expensive to rent your computers from someone else. But it’s never presented in those terms. The cloud is sold as computing on demand, which sounds futuristic and cool, and very much not like something as mundane as “renting computers”, even though that’s mostly what it is.

But also:

It’s also about what kind of internet we want to operate in the future. It strikes me as downright tragic that this decentralized wonder of the world is now largely operating on computers owned by a handful of mega corporations. If one of the primary AWS regions go down, seemingly half the internet is offline along with it.

Methinks this will become even more of a trend in the coming months and years.

Maybe it’s the Californian sun. Or perhaps it’s the time spent at Disney Studios, the home of the best stories. One thing is for sure: Taylor Dolezal is one of the happiest cloud native people that Gerhard knows.

As a former Lead SRE for Disney Studios, Taylor has significant hands-on experience running cloud native technologies in a large company. After a few years as a HashiCorp Developer Advocate, Taylor is now Head of End User Ecosystem at CNCF. In his current role, he is helping enable cloud native success for end-users like Boeing, Mercedes Benz & many others.

As it gets easier and easier to deploy to the cloud, and with increased concern about data sovereignty, regional architectures are becoming more common. This article gives you an overview of how we easily maintain our regional services on AWS at Apptrail.

Rich Burroughs:

When you hear the phrase “cloud native,” is Kubernetes the first thing that comes to your mind? It is for me, and I expect I’m not alone. Kubernetes is now the second-largest open source project after Linux, and it’s the big fish in the cloud native pond. But there are many other projects in the CNCF landscape and the broader cloud native community.

So, I thought I’d list some cloud native tools that can be very useful for teams that aren’t using Kubernetes or aren’t using it for every workload. Here are 7 of them that I like a lot.

If Rich’s name rings a bell, that’s because he was just on Ship It! last week. 😉

One of the things we discussed with Paul Copplestone from Supabase was what, exactly, might a cloud native Postgres look like? Well, perhaps it will look like OrioleDB:

A new storage engine for PostgreSQL, bringing a modern approach to database capacity, capabilities and performance to the world’s most-loved database platform.

OrioleDB consists of an extension, building on the innovative table access method framework and other standard Postgres extension interfaces. By extending and enhancing the current table access methods, OrioleDB opens the door to a future of more powerful storage models that are optimized for cloud and modern hardware architectures.

Troy Hunt (of Have I been Pwned fame) has been a vocal proponent of cloud-first services for awhile. Last December, that strategy came back to bite him:

It all started with my monthly Azure bill for December which was way over what it would normally be. It only took a moment to find the problem…

He goes on to tell the tale in excruciating detail. Be careful out there, cloud natives.

In our first 2022 episode, Alexis Richardson, co-founder and CEO of Weaveworks, is talking to Gerhard about going fully remote, what a great team looks like, and GitOps. While you may have heard of GitOps, now is a good time to check out opengitops.dev.

The most interesting part of today’s conversation is the missing cloud native App Store. While Apple revolutionised the world with the App Store and the iPhone, we don’t yet have something similar for cloud native apps. You may be thinking “But what about OperatorHub?”, or all the Helm registries out there? The registry fragmentation, operator deprecations and lack of curation are not what people have in mind when they think App Store. But there is more to it, so let’s hear how Alexis thinks about this.

This cool open source project provides transparent client-side encryption to be used with your cloud file storage of choice. Which choices, you ask?

Works with Dropbox, Google Drive, OneDrive, MEGA, pCloud, ownCloud, Nextcloud and any other cloud storage service which synchronizes with a local directory

On this special edition of The Changelog, we’re talking with Cory Wilkerson, Senior Director of Engineering at GitHub, about GitHub Codespaces. For years now, the possibility of coding in the cloud seemed so close, yet so far away for a number of reasons. According to Cory, the raw ingredients to make coding in the cloud a reality have been there for years. The challenge has really been how the industry thinks, and we are now at a place where the skepticism in cloud based workflows is “non-existent.”

After 15 months in preview, GitHub not only announced the availability of Codespaces for Teams and Enterprise — they also showcased their internal adoption, with 600 of their 1,000 engineers using it daily to develop GitHub.com.

On this episode, Cory shares the full backstory of that journey and a peek into the future where we’re all coding in the cloud.

This is an article about exploring Inlets, a cloud native tunnel, with various use cases (including the Apple Silicon RC version of inlets-pro) in my home lab along with its surrounding open source ecosystem. Also some tips about using inlets-operator and MetalLB at the same time, and what could be done in a Kubernetes cluster.

Spending the time and effort to consider what happens after your cloud credits run out, including how much storage you’ll need to scale and how you plan to manage your data, pays off in the long run. Every developer wants their project to succeed.

So, how do you plan for survival and success when your free credits run out?

This week Adam is joined by Mitch Wainer, previously CMO at DigitalOcean and a member of the founding team. They talk about his journey as an entrepreneur and marketer, the early days at DigitalOcean, and everything that went into disrupting the cloud with blazing fast SSDs. Back in March (2021), DigitalOcean started trading on the New York Stock Exchange (NYSE) — this obviously earned Mitch and many others a very large payday. They also talk about the work Mitch is doing now with Welcome and Sponsored.

Why Cloud Native? What are the guiding principles that you should keep in mind as you are choosing a project from the Cloud Native Landscape? How do you build & ship an app in a Cloud Native way? Katie Gamanji, Ecosystem Advocate @ CNCF and former cloud engineer for American Express, Condé Nast and Microsoft, joins Gerhard to cover these topics in the context of the Cloud Native Fundamentals course that she developed. 15,000 students have already enrolled, and the initial feedback has been great. Tune in if you want to know why you should too, how to do it and when the course will become available for free.

What is cloud native? In this episode Johnny and Aaron explain it to Mat and Jon. They then dive into questions like, “What problems does this solve?” and “Why was Go such a good fit for this space?”

Cryptomator works with Dropbox, Google Drive, OneDrive, ownCloud, Nextcloud and any other cloud storage service which synchronizes with a local directory. Since it’s open source, you can check for backdoors. Since it’s entirely client-side, you don’t have to trust anybody else’s machines.

Rclone acts like its source of influence, rsync, but instead syncs files and directories to many cloud storage providers. It runs on all the Linux/macOS/Windows flavors. You can even install with Homebrew on macOS.

brew install rclone

Based on what I read in the HN comments there are tons of users/uses for Rclone.

Craig Wiley, from Google Cloud, joins us to discuss various pieces of the TensorFlow ecosystem along with TensorFlow Enterprise. He sheds light on how enterprises are utilizing AI and supporting AI-driven applications in the Cloud. He also clarifies Google’s relationship to TensorFlow and explains how TensorFlow development is impacting Google Cloud Platform.

Kubernetes has won and the cloud is a moving target. But, one thing that often gets lost in the mix with all the Cloud Native talk is the productivity costs associated with keeping up.

In the US alone, over 70% of enterprises have adopted or are currently adopting cloud-native architecture, causing a surge in developers who are trying to learn the stack.

It’s called the “cutting edge” for a reason…

Staying on the cutting edge…one critical area of productivity loss is keeping up with all the changing technologies.

Cloud-native architecture is still being developed and learning the latest technologies is a moving target. While at the same time, most computer science and software engineering programs don’t delve into the heart of these technologies. At best, graduates will have limited experience working with a handful of these cloud technologies…

Depending on your perspective or seat at the table, these hidden costs could be a good thing.

The commercial VPN industry is a minefield to navigate and many open source solutions are a pain to use or ill-suited for the task. Algo VPN, on the other hand, is a self-hosted personal VPN designed for ease of deployment and security. It uses the securest industry standards, builds on rock-solid solutions like WireGuard and Ansible, and runs on an ever-growing list of cloud hosting providers.

On this episode Dan Guido –CEO of security firm Trail of Bits and Algo’s creator– joins Jerod to discuss the project in depth.

The linked article is an excellent introduction to Algo, which is effectively a set of Ansible scripts that set up a Wireguard and IPSEC VPN for you.

Algo automatically deploys an on-demand VPN service in the cloud that is not shared with other users, relies on only modern protocols and ciphers, and includes only the minimal software you need. And it’s free.

For anyone who is privacy conscious, travels for work frequently, or can’t afford a dedicated IT department, this one’s for you.

Algo’s list of features (and anti-features) is compelling and most VPN services are terrible. 👀

Johnny and Mat are joined by Kris Nova and Joe Beda to talk about Kubernetes and Cloud Native. They discuss the rise of “Cloud Native” applications as facilitated by Kubernetes, good places to use Kubernetes, the challenges faced running such a big open source project, Kubernetes’ extensibility, and how Kubernetes fits into the larger Cloud Native world.

Developers and IT decision-makers should not be surprised by the recent Capital One data breach: Misconfigurations have long been the top cloud security concern. A new StackRox survey of IT decision-makers supports this finding as 60% of respondents are more worried about misconfigurations or exposures, as compared to attacks and generic vulnerabilities.

We’re not 💯 on what exactly happened, but the evidence is pointing toward a misconfigured firewall.

Cloud lock-in isn’t an issue until it is — “any amount of backups are just more eggs in the same basket if everything is under one account.”

Most of the popular one-click server providers (including DigitalOcean, as well as Linode, Vultr, and OVH) provide their own backup offerings for an additional monthly cost (usually proportional to your plan). But as Nicolas learned the hard way, any amount of backups are just more eggs in the same basket if everything is under one account with one credit card on one provider.

Luckily, crafting a DIY automated backup system using a second redundant storage provider isn’t as daunting (nor as expensive) as it might sound. The following steps are how I backup my various VPSes to a totally separate cloud in the sky.

Jake Jarvis writes on his personal blog how to backup various VPS’s to a totally separate cloud provider.