As it gets easier and easier to deploy to the cloud, and with increased concern about data sovereignty, regional architectures are becoming more common. This article gives you an overview of how we easily maintain our regional services on AWS at Apptrail.
Alex Sims, a Senior Software Engineer at James & James, an eCommerce fulfilment company, reached out to us about the Kaizen story of the third-party logistics (3PL) platform that he has been involved with for several years now.
The system delivered 16 millions of orders in 10 years, and 4.5 million in the last year alone. All the numbers are going up, and there is only so much that a single PHP monolith deployed as VM images can handle. So how do you even start thinking about the architectural improvements, and inspire everyone involved to move towards better?
We encourage you to look at the architectural diagrams in the show notes, especially the 10 year roadmap, and ask Alex for a blog post follow-up. While today’s episode was a good conversation starter, there is a lot that we did not have time to cover.
Gunnar Holwerda (Engineering Manager) and Tom Pansino (DevOps Team Lead) share with us a few stories about how the teams at opensesame.com manage AWS operational complexity. The first link in the episode show notes are the slides that Tom & Gunnar prepared for this conversation. Check them out as you hear us speak about the Inverse Conway Manoeuvre, and why you should always go for the bananas.
If you like this episode, and have a similar story to share, please reach out to us. We all love real-world stories that we can learn from, and perhaps contribute to.
Welcome to Song Encoder, a special series of The Changelog podcast featuring people who create at the intersection of software and music. This episode features Pwnie Award-winning songwriter Forrest Brazeal.
Avi Lumelsky shares his journey finding and reporting databases with sensitive data about Fortune-500 companies, hospitals, crypto platforms, startups during due diligence, and more.
His major entry point: misconfiguration
This is super interesting to see bare metal Mac mini’s attached through Thunderbolt to the Amazon EC2 infrastructure. Really makes me wish I had a reason to use them. 😂
Shawn (swyx) Wang lays out Cloudflare’s strategy to disrupt the cloud from the outside in:
While the tech industry is used to come-from-below disruption, and the software industry is increasingly grasping class-for-the-masses atomic concepts, I believe Cloudflare is writing a new playbook that is the little-guy counterpart of the embrace, extend, extinguish model used by Microsoft.
I’ve been using AWS “professionally” since about 2015. In that time, I’ve made lots of mistakes.
Other than occasionally deleting production data, the mistakes all arose from ignorance - there’s so much to know about AWS that it’s easy to miss something important.
Here’s a collection of the most commonly missed things when using AWS with Laravel Forge!
Learning from your mistakes is powerful. Learning from other people’s mistakes can be just as powerful without the major drawback of, you know, feeling all that pain!
Amazon EKS Anywhere is a new deployment option for Amazon EKS that enables you to easily create and operate Kubernetes clusters on-premises with your own virtual machines. It brings a consistent AWS management experience to your data center, building on the strengths of Amazon EKS Distro, the same distribution of Kubernetes that powers EKS on AWS. Its goal is to include full lifecycle management of multiple Kubernetes clusters that are capable of operating completely independently of any AWS services.
In this episode, Gerhard talks to his Skyhook Adventure friends: Alan Cooney, Saul Cullen & Wycliffe Maina. They are the ones that introduced Gerhard to the world of serverless in the context of Amazon Web Services. Gerhard shared his experience with remote work, how to ship software with confidence and consistency, and what to look for in infrastructure as code.
At the heart of Skyhook Adventure are adventure trips, and 2020 was not a good one for this business. As you can already tell, code and infrastructure was not the biggest challenge for this team. Having said that, serverless, microservices, a monorepo and the event-based architecture played a big part in successfully navigating the challenges.
This is a story about what happens when a good team allows itself to be guided by solid experience and keeps doing the right thing, long-term. It’s fun, real, and it applies to many.
You may have heard that Dark Sky (the beloved weather app for iOS) was acquired by Apple and its accompanying developer API will be shut down soon, which left a lot of devs scrambling for alternatives. Enter the PirateWeather API:
Weather forecasts are primarily determined using models run by government agencies, but the outputs aren’t easy to use or in formats built for applications. To try to address this, I’ve put together a service (built on AWS Lambda) that reads public weather forecasts and serves it following the Dark Sky API style.
It’s in beta at the moment, but it appears the author has put a lot of thought into it.
From the AWS open source blog:
Today, we are introducing the OpenSearch project, a community-driven, open source fork of Elasticsearch and Kibana. We are making a long-term investment in OpenSearch to ensure users continue to have a secure, high-quality, fully open source search and analytics suite with a rich roadmap of new and innovative functionality. This project includes OpenSearch (derived from Elasticsearch 7.10.2) and OpenSearch Dashboards (derived from Kibana 7.10.2). Additionally, the OpenSearch project is the new home for our previous distribution of Elasticsearch (Open Distro for Elasticsearch), which includes features such as enterprise security, alerting, machine learning, SQL, index state management, and more. All of the software in the OpenSearch project is released under the Apache License, Version 2.0 (ALv2).
Did you listen to our epic Elastic vs AWS episode on The Changelog? I’d really love to hear from the community on this subject…is this a good thing or a bad thing for open source at large? Why didn’t AWS just work with Elasticsearch (the company)?
We plan to rename our existing Amazon Elasticsearch Service to Amazon OpenSearch Service.
Serverless Stack Toolkit is an extension of AWS CDK that makes it easy to build serverless apps. It features:
Ever since AWS took Elasticsearch and decided to sell a managed version of it there has been controversy around AWS and Elasticsearch. Now that the software created by Elastic is being switched to the Server-Side Public License
(SSPL), which is not a very permissive license, AWS is going ahead and forking the projects.
The debate rages around this. Few people feel sympathy with the behemoth that is AWS, but they don’t seem to be in violation of any licenses. Elastic have definitely worked hard on Elasticsearch and arguably deserves an opportunity to profit from their work. This new license raises significant concern though.
I don’t think we’ll see this settle anytime soon, just like the issue of open source sustainability is neither easy nor straightforward.
JuiceFS is an open-source POSIX file system built on top of Redis and object storage (e.g. Amazon S3), designed and optimized for cloud native environment. By using the widely adopted Redis and S3 as the persistent storage, JuiceFS serves as a stateless middleware to enable many applications to share data easily.
serverless.tf is an opinionated open-source framework for developing, building, deploying, and securing serverless applications and infrastructures on AWS using Terraform.
This started as a response to the “accidental complexity of many existing tools”, which I definitely felt when I tried the Serverless framework awhile back.
Infracost shows hourly and monthly cost estimates for a Terraform project. This helps developers, DevOps et al. quickly see the cost breakdown and compare different deployment options upfront.
The primary objective of this boilerplate is to give you a production ready code that reduces the amount of time you would normally have to spend on system infrastructure’s configuration. It contains a number of services that a typical web application has (frontend, backend api, admin panel, workers) as well as their continuous deployment. Using this boilerplate you can deploy multiple environments, each representing a different stage in your pipeline.
there are a lot of AWS services available. And I do mean: a LOT. Currently, there are 163 (!) different services that are available from the Amazon Dashboard, each with their own way of working, difficulties, catches and best practises.
What follows is one-line descriptions of all 163 AWS services. MSK? Kafka as a service. Amazon Connect? AWS call center platform. And so on.
You’ll learn DynamoDB, Elastic Beanstalk, Serverless and more. Quincy and the gang keep cranking out the hits with free online training videos for developers of many stripes. They now have free courses for 3 out of the 12 AWS Certifications.
Do you have an AWS account? Great. Do you want to run cron jobs in the cloud?
Cronyo provides A simple CLI to manage your cron jobs on AWS.
In addition, Cronyo can instantly deploy a couple of super-simple, helpful and secure lambda functions to perform HTTP GET/POST requests for you. So if you need to trigger any webhooks on schedule, an AWS account and Cronyo is all you need :)
This stack was created out of frustration due to the fact that to this day there’s no easy way to have a full email server without the overhead of installing and configuring all servers needed to handle incoming and outgoing messages. We wanted something simple, with no interface and no server management, so we came up with S3-Email. This included AWS SES as our email server (receive and send) and S3 as our database and interface. Then we tied everything together with a bit of code via AWS Lambda.
All of this functionality and the repo is just some JSON, Yaml, and text files. Maybe 2020 really is the year of #nocode… 😉
A story of an internal tool – Replicator. It involves AWS Lambda, one of the hardest things in Computer Science, and Star Trek.
Jean-Luc Picard quotes FTW. I’d love to see this tool open sourced.
LocalStack looks like an excellent way to develop & test your serverless apps without leaving your local host. It appears they are basically mocking 20+ AWS services which is undoubtedly a lot of work and I would expect to be error prone. Is anybody out there using LocalStack on the regular and can let us know if it actually works as advertised?
Today we are happy to announce PartiQL, a SQL-compatible query language that makes it easy to efficiently query data, regardless of where or in what format it is stored. As long as your query engine supports PartiQL, you can process structured data from relational databases (both transactional and analytical), semi-structured and nested data in open data formats (such as an Amazon S3 data lake), and even schema-less data in NoSQL or document databases that allow different attributes for different rows.