Zach Leatherman on the tension and future of the Jamstack community, Chenxin Li helps you avoid 13 bad practices in data visualization, Laravel Pulse is coming real soon, Max Chernyak develops a new way to accomplish long term refactors & Spencer Baugh makes the case for more libraries and less services in our software stacks.
Shawn “swyx” Wang is back to talk with us about the state of DevRel according to ZIRP (the Zero Interest Rate Phenomenon), the data that backs up the rise and fall of job openings, whether or not DevRel is dead or dying, speculation of the near-term arrival of AGI, AI Engineering as the last job standing, the innovation from Cognition with Devin as well as their mis-steps during Devin’s launch, and what’s to come in the next innovation round of AI.
This open letter by the Future of Life institute has been signed by 1380 people (so far) including notable technologists such as Steve Wozniak, Stuart Russell, Emad Mostaque (Stability AI) & Elon Musk.
Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable. This confidence must be well justified and increase with the magnitude of a system’s potential effects. OpenAI’s recent statement regarding artificial general intelligence, states that “At some point, it may be important to get independent review before starting to train future systems, and for the most advanced efforts to agree to limit the rate of growth of compute used for creating new models.” We agree. That point is now.
Therefore, we call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4. This pause should be public and verifiable, and include all key actors. If such a pause cannot be enacted quickly, governments should step in and institute a moratorium.
Rachel Stephens, trying to reconcile the claims that the tech industry is vastly overstaffed with other claims of vast resource scarcity:
Both things can be true: it’s not a clear over- or under-staffing problem. It’s a problem of proportionality. There can definitely be organizational bloat… At the same time, there are also areas in tech that remain chronically understaffed and under-resourced: OSS project maintainers, people working on-call support, and people writing documentation to name a few.
An interesting analysis with a much-needed call for kindness to our fellow nerds as they face these tumultuous times.
And (I can’t believe I’m writing this, but the jokes about Twitter layoffs in particular keep coming at a surprising rate) be kind to people facing layoffs. Losing your job is awful in the best of circumstances; going through it in such a public and charged situation must be emotionally grueling. Be kind.
The speed record for data transmission using a single light source and optical chip has been shattered once again. Engineers have transmitted data at a blistering rate of 1.84 petabits per second (Pbit/s), almost twice the global internet traffic per second.
For context, if you’re fortunate enough to be living the “gig-life” (as my local carrier markets it) with a 1-gigabit connection at your house… a petabit is a million times faster than that. And that record-setting 1.84 petabit speed is just the start:
But the new chip is far from finished breaking records, according to the team behind it. Using a computational model to scale the data transmission potential of the system, the researchers claim that it could eventually reach eye-watering speeds of up to 100 Pbit/s.
Shut up and take my money?!
Why do we think StatefulSets are broken? Well, if you run through the operational needs of a stateful workload in your head, there’s one key component that you might notice is missing:
What do you do when you need to resize the underlying disk?
The dataset is a common database store that typically grows at a pretty constant positive rate. Unless you support horizontal scaling and partitioning, you’ll need to add headroom in the disk as that dataset grows.
This is where Kubernetes falls flat on its face.
Reading other people’s code is a sure-fire way to improve as a developer. But what about as a database designer? The same process applies!
This post by Shekhar Gulati is him sharing what he learned by applying that principle to GitLab’s schema.
I learnt a lot from the GitLab schema. They don’t blindly apply the same practices to all the table designs. Each table makes the best decision based on its purpose, the kind of data it stores, and its rate of growth.
More posts like this, please!
In this Fully-Connected episode, Chris and Daniel explore the geopolitics, economics, and power-brokering of artificial intelligence. What does control of AI mean for nations, corporations, and universities? What does control or access to AI mean for conflict and autonomy? The world is changing rapidly, and the rate of change is accelerating. Daniel and Chris look behind the curtain in the halls of power.
HUBFS is a file system for GitHub and Git. Git repositories and their contents are represented as regular directories and files and are accessible by any application, without the application having any knowledge that it is really accessing a remote Git repository. The repositories are writable and allow editing files and running build operations.
So if you hubfs mnt (on macOS/Linux), it will set up a file hierarchy inside /mnt that follows this pattern: / owner / repository / ref / path. Cool idea! It is affected by GitHub’s API rate limiting and I’m not sure if/how it syncs (commits) back to the remote repos…
As the editor of the Elixir Radar newsletter, I read lots of articles related to Elixir every single week. Along the year I read probably more than 700 articles, so I could curate the best ones and send them to Elixir Radar’s subscribers.
In this article, I share the 11 most popular articles on Elixir Radar in 2020. Those are the ones that had the biggest engagement from Elixir Radar subscribers in each month of 2020, in terms of CTOR (click-to-open rate).
ffmpeg is an incredibly powerful tool, but its many flags and options make it not the easiest thing to wield (especially if you use it just infrequently enough to forget the magic syntax you ginned up last time).
vdx makes ffmpeg more approachable for many of the common video processing operations you may need on a regular basis. Examples!
$ vdx '*.mov' --crop=360,640 # Crop to width 360, height 640
$ vdx '*.mov' --format=gif # Convert to GIF
$ vdx '*.mov' --fps=12 # Change the frame rate to 12
$ vdx '*.mov' --no-audio # Strip audio
$ vdx '*.mov' --resize=360,-1 # Resize to width 360, maintaining aspect ratio
$ vdx '*.mov' --reverse # Reverse
$ vdx '*.mov' --rotate=90 # Rotate 90 degrees clockwise
$ vdx '*.mov' --speed=2 # Double the speed
$ vdx '*.mov' --trim=0:05,0:10 # Trim from time 0:05 to 0:10
$ vdx '*.mov' --volume=0.5 # Halve the volume
We’re big fans of what Hacktoberfest represents, but maybe it’s time to rethink the model. The burden falls primarily on maintainers, as Domenic Denicola outlines in this post – going as far as to describe Hacktoberfest as “a corporate-sponsored distributed denial of service attack against the open source maintainer community.”
For the last couple of years, DigitalOcean has run Hacktoberfest, which purports to “support open source” by giving free t-shirts to people who send pull requests to open source repositories.
In reality, Hacktoberfest is a corporate-sponsored distributed denial of service attack against the open source maintainer community.
So far today, on a single repository, myself and fellow maintainers have closed 11 spam pull requests. Each of these generates notifications, often email, to the 485 watchers of the repository. And each of them requires maintainer time to visit the pull request page, evaluate its spamminess, close it, tag it as spam, lock the thread to prevent further spam comments, and then report the spammer to GitHub in the hopes of stopping their time-wasting rampage. … The rate of spam pull requests is, at this time, around four per hour. And it’s not even October yet in my timezone.
This screenshot of issues on whatwg/html labeled as spam was taken moments before posting this.
This seems like a natural counter-weight to the go-big-or-go-home strategy of many venture capitalists:
Over five years, Indie.vc has backed 34 companies — half of which are women-led companies and 20% are Black. And while there haven’t been any big exits yet, the companies that receive Indie.vc funding seem to be much more robust than their peers, especially in a challenging economic climate. On average, they’re growing 100% in the first year, and 300% the second year, says Roberts. Plus, the fund’s mortality rate is 10% — compared to about 44% with traditional VC-backed companies.
Indie.vc’s next application window is “Fall 2020.”
This calculator lets you tweak things like R0, incubation time, and hospitalization rate to see how affect the results. From the author:
At the time of writing, the coronavirus disease of 2019 remains a global health crisis of grave and uncertain magnitude. To the non-expert (such as myself), contextualizing the numbers, forecasts and epidemiological parameters described in the media and literature can be challenging. I created this calculator as an attempt to address this gap in understanding.
Emily Robinson, co-author of the book Build a Career in Data Science, gives us the inside scoop about optimizing the data science job search. From creating one’s resume, cover letter, and portfolio to knowing how to recognize the right job at a fair compensation rate.
Emily’s expert guidance takes us from the beginning of the process to conclusion, including being successful during your early days in that fantastic new data science position.
This is an #oldread (2010), but it came up again in two contexts (Ron Evans’ Gopherbot and Chris Anderson’s Robocars) that made me give it another look.
The reason big new things sneak by incumbents is that the next big thing always starts out being dismissed as a “toy.” This is one of the main insights of Clay Christensen’s “disruptive technology” theory. This theory starts with the observation that technologies tend to get better at a faster rate than users’ needs increase. From this simple insight follows all kinds of interesting conclusions about how markets and products change over time.
In this post, Rich Archbold touches on something we discussed on a recent episode of The Changelog. Specifically, in the episode, we talked about contentment being the enemy of progress and how that might effect our industry psychologically — at-large. But when is what we’re working on ever good enough?
Rich has this to say…
Software can never be perfect, it can only ever be “good enough”…beyond a certain size and rate of change — it’s always going to contain bugs and experience outages. So how do you know if your software is good enough? … My opinion and approach is to codify your beliefs around what constitutes software that is “good enough” into a small set of engineering principles and build a culture, organization, and set of processes that reinforce them.
Manuel Bieh:
As an independent Freelance Developer I was wondering how I can support the Open Source community… so I had this idea: starting with my next project I will ask my clients for an hourly rate that is 1 Euro higher than I originally negotiated or I would usually charge. I will take that money (up to ~160 Euros per month) and support those projects on Open Collective that I’m basing my work upon in my client’s project.
I like the spirit of what Manuel is doing here, but I’d suggest a slightly different tactic: raise your rate by N euros/hr (where N is at least 10) and give that to open source maintainers whose software you use on the client’s behalf. No need to complicate the client relationship with additional line items or things to explain. Besides, you’re probably under charging as is. Most of us are…
From Kira Booth writing on the Plaid blog.
…we take an agile-like approach to how we think about process. If our team’s process isn’t working, we talk about it in a retrospective (aka “retro”) and figure out how to change it. Many companies don’t begin retros until they are large and have many processes in place, but we feel that retros are especially valuable at our size and rate of growth.
Plaid’s engineering organization is rapidly growing. In the Salt Lake City office where I work, we have plans to grow from 20 to 60 engineers this year. Processes that worked just a few months ago may not work now. A culture of continuous process improvement helps us to stay ahead of growing pains like inefficient collaboration, error-prone coding practices, and interpersonal conflict.
The adoption of application container technology is increasing at a remarkable rate and is expected to grow by a further 40% in 2020, according to 451 Research. It is common for system libraries to be available in many docker images, as these rely on a parent image that is commonly using a Linux distribution as a base.
In many cases, remediation is as simple as rebuilding the image or swapping out the base image, but it’s not always that easy. Click through for more analysis and advice.
We talk about Imposter Syndrome a lot around these parts, but a related phenomenon that gets less attention is a cognitive bias in which people with lesser abilities tend to rate themselves as more proficient than they are.
You’ll definitely want to read this if you’d like to know how Slack handles their job queue.
Robustly handling billions of tasks in milliseconds using Kafka and Redis.
On our busiest days, the system processes over 1.4 billion jobs at a peak rate of 33,000 per second. Job execution times range from a few milliseconds to (in some cases) several minutes.
Also, check out The Changelog #278 — we talked about blockchains and databases at OSCON, including a conversation with Tague Griffith on Redis.
Andrew Kane took a Heroku blog post about getting insights in to your PostgreSQL database and ran with it, creating a nice dashboard you can mount in your Rails app as an engine.
PgHero will show you long running queries, cache hit rate, and more. I installed it on one of my apps this morning and it worked well!
I used to be a big fan of rack-throttle for rate limiting HTTP requests to Rack applications, but it appears to be abandoned.
Kickstarter's Rack::Attack looks like a worthy replacement. Gotta love when the explanation of how it works is longer than the code it describes.
The massive success of the HTML5 boilerplate has inspired similar efforts in other domains. Michael Goreanski has started such an effort for Nginx, the web server employed by many of the Internet’s top sites (recent stats).
Nginx-boilerplate is only a couple months old, but it already boasts configuration templates that feature:
Boilerplate projects are great because they provide an opportunity for the community to rally around best practices and put them all in one place. The nginx-boilerplate already has a handful of contributors, but I’d love to see that list explode in the coming weeks.
The project is MIT licensed and hosted on GitHub.
If you’re writing a web service and you’d like to expose an API to the world, few things are simpler than running a Sinatra application to provide one. Grape, by Michael Bleigh, is one of those things. Grape builds on top of rack, but gives you versioning, automatically providing responses in multiple formats, and other cool stuff.
Here’s a tiny example for FooCorp, who wants to expose their paradigm-shifting, revolutionary Widget models to the world:
class Foo::API < Grape::API
version '2'
resource :widgets do
get :best do
Widget.best_list
end
get '/show/:id' do
Widget.find params[:id]
end
end
end
Since this is just another Rack application, you’d put this into your config.ru:
run Foo::API
This gives you these two calls:
I originally thought that Rails 3 did just about as much as they possibly could to eliminate extra noise with the new respond_with syntax, but it’s still more than Grape. Here’s what a similar Rails 3 controller would look like:
class WidgetController < ApplicationController
respond_to :html, :json
def show
respond_with(Widget.find(params[:id]))
end
def best
respond_with(Widget.best_list)
end
end
This also doesn’t take into account the versioning aspect, either. Now, Rails obviously can do a bit more than Grape can, but APIs are a great reason to use a microframework: You want the web server to be doing as little work as possible with an API. Serving up some JSON is a much more lightweight activity than the big, JavaScript-heavy pages that we expect from the Web today.
Grape’s roadmap shows some cool stuff is still in store. Rate Limiting is an important feature for any API, and filter support and OAuth are nice to have.
Preston Thorpe joins us from inside prison, where he awaits a hopeful release within the next 12 months. His journey has been anything but easy—marked by hardship and uncertainty. But over the past few years, Preston has undergone a profound transformation. He’s refactored not just his skills, but his identity. Today, he proudly calls himself a software engineer and an open source contributor. In this episode, Preston shares his story of redemption, resilience, and what comes next.
Matched from the episode's transcript 👇
Preston Thorpe: [01:25:48.26] Yeah, I think my main point there was - there’s a lot of people with similar mindsets to myself, in different facilities that don’t have any of the opportunities that they have here… That if given the chance, would flourish and would pursue these things, and give themselves a way out. Because without this skill, and without the network… I guess the skill is one thing, but without the ability to network and the ability to make other people aware of your story… To be in a place with no ability to better yourself, and if all you’re given is there’s like four phones on the wall, and 200 gang members surrounding you… You could be like even me right now, who’s so far from having that mindset and being involved in that lifestyle… If you don’t have the means to improve yourself, it’s really difficult to expect a different behavior.
I think that’s what I want people to consider, is if your expectations are that incarcerating people… You know, it’s the department of corrections, it’s expected that you’re going to learn your lesson and you’re not going to do it again… But oftentimes the reality is you take someone that maybe has far less exposure to some of the things they’re going to be exposed to in prison, and they spend a few years there and just end up absorbing the criminal mindset and the negativity, and they have absolutely no resources to improve themselves. There’s no access to meaningful education, or any way to find a way out of the hole that they dug for themselves. To then expect someone to get out and now have a criminal record, and no work history for N years… It’s now just orders of magnitude more difficult to actually improve your situation than it was before this punishment.
So when people – when the recidivism rate is 70% of people return to prison after I think like three years… The reason for that shouldn’t be a big surprise. I think making people aware that investing in education and investing in giving people opportunities to improve themselves, and in ways to find a career path, and meaning… That pays off significantly more.
There’s all kinds of statistics, I wish I was good at kind of reciting these things… One of my coworkers is. Shout-out, Jess. He can tell you all the statistics about how every dollar spent on education saves N dollars in recidivism costs, and all that. I’m horrible at remembering those things, but… Yeah, I think that’s the general idea. I think people need to be given a way out.
Nathan Sobo is back talking about the next big thing for Zed—agentic editing! You now have a full-blown AI-native editor to play with. Collaborate with agents at 120fps in a natively multiplayer IDE.
Matched from the episode's transcript 👇
Adam Stacoviak: You can test and experiment at such a rate, and fail sooner, too. Like, don’t waste so much time. Or find the thing that’s actually got a thread to pull and you pull on that thread, and you do what you’ve got to do to take that next step. I feel like that’s what’s happening, it’s like the ultimate numbers game increaser for everybody.
Kendall Miller is a bubbly extrovert who sticks his fingers in a lot of pies. He advises tech companies like FusionAuth, positions tech products like Civo & Tensorlake, organizes tech networks like CTO Lunches, and even sells whiskey & gin to tech people like us via his Friday Deployment Spirits brand. Kendall has learned a lot since he first entered the industry and he’s eager to share what he knows, and who he knows, with the world.
Matched from the episode's transcript 👇
Kendall Miller: I mean – so the first thing is… You’ve heard it said first time founders build, second time founders sell. There’s a lot of different variations on that, but it’s a common thing. The selling – what makes a salesperson a salesperson is not that they’ve learned this trick. “Oh, if I say this in this way, then they’re going to buy.” There are a few tricks like that. I’ve learned a few that I’ve been surprised actually are very effective… But it’s not like I’m going to turn my close rate from 5% to 90% because I learned to be a sleazy dirtbag sales guy. The truth of sales is number one, it’s a numbers game. You knock on enough doors with the world’s worst vacuum, eventually you knock on a door where somebody’s vacuum just broke, and they have companies showing up in a half an hour, and they are stressed the heck out and they don’t care how bad your vacuum is, they need it right now. So at the end of the day, it is just a numbers game. Having a great vacuum that does a great demo helps you sell more vacuums. But you’ve still got to knock on a million doors.
What makes a salesperson a salesperson is they’ve knocked on every door in the neighborhood, they have no idea what to do next, and they wake up and keep going. And that’s not in the DNA of the average technical co-founder or technical founder. And so it’s a thing that they have to learn the hard way, and almost always by the hard way, without putting in air quotes, what that means is they’ve failed once. They built a company and they couldn’t get enough people to use it, or they reached a couple million and they needed to reach a lot of million, because they were venture-backed. And then they realize in the second round that they have to go bigger, harder.
So what do I do? Well, so you asked a couple of questions there. First of all, general advice - sell, sell, sell, sell, sell. Sell it before you build it. Also, build something much, much smaller than you think. Because a lot of companies go build a product this big. Huge. And then they find out after talking to everybody that the product should go this way, and 10% of it is relevant. And it’s really hard to pivot when you’ve boiled the ocean already.
[00:11:50.15] If you build a product, that really should be a feature, and you could build in a weekend, then when you go out and talk to the market and find out “Oh, actually, the interest is over in this direction” or “I need to build it that direction”, it’s not that stressful to pivot, it’s not that hard. You can listen to the advice from customers and actually build a thing that they want.
My favorite companies are the companies that when I talk to technical people, they’re like “Why does that even exist? I can build that in a weekend.” Sure. You could build that in a weekend. You’d build it poorly, it would be not feature rich, and you wouldn’t be able to sell it, because you have no design sense and you don’t know who to talk to. Right? But if you build it in a weekend and then you add on, you can add on in the right direction without being stressed about it because you already boiled the ocean.
So that’s the first thing - build a lot smaller than you think, sell it a lot sooner than you think. If you can’t get somebody to pony up money for it really early on, it’s not going to be easier when it’s bloated and big and heavier, and you’ve put 10,000 times more of your opinions into it.
The other thing that I see a lot of technical founders do is they don’t do the math. You have to build a business around something that actually makes sense. One technical co-founder I worked with a long time ago built a B2B SaaS product that engineering leaders were going to buy, and he priced it as such that - I was talking with him saying, “Well, now hang on a second… You’ve sold to one of the world’s biggest companies, and it’s a $5 a month deal?” “Well, more engineers are going to take it up.” Okay, but how many of these do we have to sell before you can earn a living? And this is a bootstrapped business, it wasn’t even venture backed… And he’s sitting there and doing the math, and like “Well, I guess I’ve got to sell about 10,000 more…”
So either you can sell it for $500 a month, or $1,000 a month, or $10,000 a month, or whatever it’s going to take you to actually get to the sustainability that you need within a timeframe that makes sense, or you need to not sell it. But selling it for $5, patting yourself on the back that you got a success story, and finding out that you could do this for 25 years and you might be making $2,000 a month… That doesn’t work.
Anthony Eden, Founder & CEO of DNSimple, joins the show to talk about the world of managed hosting for DNS and more.
Matched from the episode's transcript 👇
Anthony Eden: Yeah, we could say essentially languages that have enough use at DNSimple that an engineer needs to know them, or at least if they’re going to run the whole stack. The answer is most of the engineers spend most of their time in Ruby. The vast majority of changes – the Ruby code, what the customers see, changes a lot faster. So it has the highest rate of change. The other two languages have a much slower rate of change. And so the reality is they don’t get constant practice. And then now what we’re trying to do is get it so that they can ensure that locally they can stand up the whole stack, so that they can get practice with each one of those things and make changes with each one of those things. If an engineer needs to do it every once in a while, most good engineers will be comfortable switching languages. They’ll be able to pick it up. It won’t be instantaneous, they’re not going to write the most idiomatic code, but they will be able to pick it up and do what they need at the most basic level with it. And especially if there’s good tests that they can run and ensure that whatever they’re doing doesn’t break things horribly.
