Amazon Web Services Icon

Amazon Web Services

Amazon's cloud computing platform.
48 Stories
All Topics

Rust github.com

A high-throughput file client for mounting Amazon S3 buckets

With Mountpoint for Amazon S3, your applications can access objects stored in Amazon S3 through file operations like open and read. Mountpoint for Amazon S3 automatically translates these operations into S3 object API calls, giving your applications access to the elastic storage and throughput of Amazon S3 through a file interface.

Early alpha. Written in Rust by the AWS team.

Corey Quinn lastweekinaws.com

AWS is asleep at the Lambda wheel

Corey Quinn:

Countless volumes have been written about the various benefits of serverless, a task made even easier by it being such a squishy, nebulous term that’s come to mean basically whatever the author wants it to mean. This has been a boon for AWS’s product teams, who’ve gone from creating services that are clearly serverless such as DynamoDB, Route 53, IAM, and others to instead slapping the “serverless” moniker on things that are clearly not very serverless at all, like OpenSearch and Aurora.

Go benhoyt.com

From Go on EC2 to Fly.io: +fun, −$9/mo

Ben Hoyt shares his experience switching two of his side projects from on an EC2 instance to Fly.io:

It took me about an hour to figure out the basics of Fly.io and move the simpler project, and a couple of evenings to move the more complex one. Fly.io handles the annoying reverse proxy and SSL stuff, deployment is as simple as fly deploy, and there’s a nice dashboard on Fly.io to show me what’s going on…

I’m only a few weeks into using Fly.io to host my side projects, but I’m very happy with their product so far. I was quite happy to delete the 500 lines of Ansible scripts, systemd unit files, and Caddy config files.

It also made me smile to finally stop the EC2 instance and bump my AWS bill down from $9 per month to about 10 cents per month (I still use S3 for user-uploaded images and for backups). I have nothing against EC2 and would use it again for certain things, but for small web applications, Fly.io seems like a great fit.

Ship It! Ship It! #89

Rust efficiencies at AWS scale

Tim McNamara is known as New Zealand’s Rust guy. He is the author of Rust in Action, and also a Senior Software Engineer at AWS, where he helps other builders with all things Rust.

The main reason why Gerhard is intrigued by Rust is the incredible resource frugality. Fewer CPUs means less energy used, which is good for the planet, and good for the monthly bill. This becomes most noticeable at Amazon’s scale, when S3, Lambda, CloudFront and other services start adding Rust components.

Ruby rubyonjets.com

Ruby and Lambda had a baby and that child's name is Jets

Ruby on Jets allows you to create and deploy serverless services with ease, and to seamlessly glue AWS services together with the most beautiful dynamic language: Ruby. It includes everything you need to build an API and deploy it to AWS Lambda. Jets leverages the power of Ruby to make serverless joyful for everyone.

I’m not (yet) big on serverless things, but if I were, I’d love to run some Ruby code there.

Ship It! Ship It! #83

🎄 Planning for failure to ship faster 🎁

Eight months ago, in 🎧 episode 49, Alex Sims (Solutions Architect & Senior Software Engineer at James & James) shared with us his ambition to help migrate a monolithic PHP app running on AWS EC2 to a more modern architecture. The idea was some serverless, some EKS, and many incremental improvements.

So how did all of this work out in practice? How did the improved system cope with the Black Friday peak, as well as all the following Christmas orders? Thank you Alex for sharing with us your Ship It! inspired Kaizen story. It’s a wonderful Christmas present! 🎄🎁

History allthingsdistributed.com

The Distributed Computing Manifesto

Amazon CTO, Werner Vogels:

Today, I am publishing the Distributed Computing Manifesto, a canonical document from the early days of Amazon that transformed the architecture of Amazon’s ecommerce platform. It highlights the challenges we were facing at the end of the 20th century, and hints at where we were headed.

25 years later. This is super cool!

Ship It! Ship It! #73

A modern bank infrastructure

Matias Pan is a Staff Software Engineer at Lemon Cash, a crypto startup based in Argentina. Lemon infrastructure runs digital wallets & physical cards, which technically makes them a bank. How does Matias & his team think about enabling developers get code from their workstations into production? Remember, we are talking about a bank - a bad deploy is a big deal. And when a bad database migration goes out, what happens then?

Amazon Web Services Medium (via Scribe)

We reduced our server costs 80% by moving away from AWS

Zsot Varga explains how Prerender saved $800k annually by removing their reliance on AWS and building in-house infrastructure to handle traffic and cached data. This was no minor migration, and it took months to pull off, but it’s a solid lesson in testing your assumptions.

The cloud (which is AWS in most cases) is the default for most businesses today. That’s a good starting place for many reasons, but once you get up and going you may find it’s not the best choice for your business, like the folks at Prerender learned.

PostgreSQL github.com

An open source alternative to AWS Aurora

When we had Paul Copplestone from Supabase on The Changelog I asked him what a cloud native Postgres would look like. He replied,

Decoupled compute and storage. So the idea is that you should be able to attach the compute part of it to a storage, hopefully like an infinite storage; you know, anything that is infinitely scalable. If you can do this, and in particular, if the compute can start up very fast, maybe in, say, a hundred milliseconds via some sort of HTTP response, then that’s cloud native, yeah.

Enter Neon, which sells itself as:

The multi-cloud fully managed Postgres with a generous free tier. We separated storage and compute to offer autoscaling, branching, and bottomless storage.

Behind this new business from MemSQL co-founder Nikita Shamgunov is the linked (Apache-licensed) server written in Rust.

Ship It! Ship It! #49

Improving an eCommerce fulfillment platform

Alex Sims, a Senior Software Engineer at James & James, an eCommerce fulfilment company, reached out to us about the Kaizen story of the third-party logistics (3PL) platform that he has been involved with for several years now.

The system delivered 16 millions of orders in 10 years, and 4.5 million in the last year alone. All the numbers are going up, and there is only so much that a single PHP monolith deployed as VM images can handle. So how do you even start thinking about the architectural improvements, and inspire everyone involved to move towards better?

We encourage you to look at the architectural diagrams in the show notes, especially the 10 year roadmap, and ask Alex for a blog post follow-up. While today’s episode was a good conversation starter, there is a lot that we did not have time to cover.

Ship It! Ship It! #38

Go for the bananas

Gunnar Holwerda (Engineering Manager) and Tom Pansino (DevOps Team Lead) share with us a few stories about how the teams at opensesame.com manage AWS operational complexity. The first link in the episode show notes are the slides that Tom & Gunnar prepared for this conversation. Check them out as you hear us speak about the Inverse Conway Manoeuvre, and why you should always go for the bananas.

If you like this episode, and have a similar story to share, please reach out to us. We all love real-world stories that we can learn from, and perhaps contribute to.

Cloud swyx.io

AWS is playing Chess. Cloudflare is playing Go

Shawn (swyx) Wang lays out Cloudflare’s strategy to disrupt the cloud from the outside in:

While the tech industry is used to come-from-below disruption, and the software industry is increasingly grasping class-for-the-masses atomic concepts, I believe Cloudflare is writing a new playbook that is the little-guy counterpart of the embrace, extend, extinguish model used by Microsoft.

Chris Fidao laravel-news.com

Mistakes I've made in AWS

Chris Fidao:

I’ve been using AWS “professionally” since about 2015. In that time, I’ve made lots of mistakes.

Other than occasionally deleting production data, the mistakes all arose from ignorance - there’s so much to know about AWS that it’s easy to miss something important.

Here’s a collection of the most commonly missed things when using AWS with Laravel Forge!

Learning from your mistakes is powerful. Learning from other people’s mistakes can be just as powerful without the major drawback of, you know, feeling all that pain!

Amazon Web Services github.com

You can now run Amazon EKS on your own infra

Amazon EKS Anywhere is a new deployment option for Amazon EKS that enables you to easily create and operate Kubernetes clusters on-premises with your own virtual machines. It brings a consistent AWS management experience to your data center, building on the strengths of Amazon EKS Distro, the same distribution of Kubernetes that powers EKS on AWS. Its goal is to include full lifecycle management of multiple Kubernetes clusters that are capable of operating completely independently of any AWS services.

Ship It! Ship It! #13

A monorepo of serverless microservices

In this episode, Gerhard talks to his Skyhook Adventure friends: Alan Cooney, Saul Cullen & Wycliffe Maina. They are the ones that introduced Gerhard to the world of serverless in the context of Amazon Web Services. Gerhard shared his experience with remote work, how to ship software with confidence and consistency, and what to look for in infrastructure as code.

At the heart of Skyhook Adventure are adventure trips, and 2020 was not a good one for this business. As you can already tell, code and infrastructure was not the biggest challenge for this team. Having said that, serverless, microservices, a monorepo and the event-based architecture played a big part in successfully navigating the challenges.

This is a story about what happens when a good team allows itself to be guided by solid experience and keeps doing the right thing, long-term. It’s fun, real, and it applies to many.

API pirateweather.net

A weather forecast API to replace Dark Sky's

You may have heard that Dark Sky (the beloved weather app for iOS) was acquired by Apple and its accompanying developer API will be shut down soon, which left a lot of devs scrambling for alternatives. Enter the PirateWeather API:

Weather forecasts are primarily determined using models run by government agencies, but the outputs aren’t easy to use or in formats built for applications. To try to address this, I’ve put together a service (built on AWS Lambda) that reads public weather forecasts and serves it following the Dark Sky API style.

It’s in beta at the moment, but it appears the author has put a lot of thought into it.

Amazon Web Services aws.amazon.com

Introducing OpenSearch (renamed from Amazon Elasticsearch Service)

From the AWS open source blog:

Today, we are introducing the OpenSearch project, a community-driven, open source fork of Elasticsearch and Kibana. We are making a long-term investment in OpenSearch to ensure users continue to have a secure, high-quality, fully open source search and analytics suite with a rich roadmap of new and innovative functionality. This project includes OpenSearch (derived from Elasticsearch 7.10.2) and OpenSearch Dashboards (derived from Kibana 7.10.2). Additionally, the OpenSearch project is the new home for our previous distribution of Elasticsearch (Open Distro for Elasticsearch), which includes features such as enterprise security, alerting, machine learning, SQL, index state management, and more. All of the software in the OpenSearch project is released under the Apache License, Version 2.0 (ALv2).

Did you listen to our epic Elastic vs AWS episode on The Changelog? I’d really love to hear from the community on this subject…is this a good thing or a bad thing for open source at large? Why didn’t AWS just work with Elasticsearch (the company)?

We plan to rename our existing Amazon Elasticsearch Service to Amazon OpenSearch Service.

Elasticsearch aws.amazon.com

AWS forks Elasticsearch and Kibana as license changes

Ever since AWS took Elasticsearch and decided to sell a managed version of it there has been controversy around AWS and Elasticsearch. Now that the software created by Elastic is being switched to the Server-Side Public License
(SSPL), which is not a very permissive license, AWS is going ahead and forking the projects.

The debate rages around this. Few people feel sympathy with the behemoth that is AWS, but they don’t seem to be in violation of any licenses. Elastic have definitely worked hard on Elasticsearch and arguably deserves an opportunity to profit from their work. This new license raises significant concern though.

I don’t think we’ll see this settle anytime soon, just like the issue of open source sustainability is neither easy nor straightforward.

Player art
  0:00 / 0:00