Import your gitlog into sqlite to mine interesting data from it
Thereâs a lot of information that you are ignoring from your VCS. SQL is a great way to access it.
Thereâs a lot of information that you are ignoring from your VCS. SQL is a great way to access it.
Aaron Francis:
The goal is not to become famous, the goal is to increase the chances of luck finding us. For me, one of the most helpful ways to think about this has always been the concept of the âLuck Surface Area,â described in an old post by Jason Roberts. He wrote (and note, the emphasis is mine):
The amount of serendipity that will occur in your life, your Luck Surface Area, is directly proportional to the degree to which you do something youâre passionate about combined with the total number of people to whom this is effectively communicated.
Going further, he codifies it into a formula where:
Luck = [Doing Things] * [Telling People]
This reminds me a bit of John Gruberâs Obsession Times Voice. Not exactly the same, but thereâs a healthy overlap that I find provocative.
Your first thought when reading that headline mightâve been _WebAssembly. Mine was! But no, that is not how Browsix does its thing:
Browsix is a framework that bridges the considerable gap between conventional operating systems and the browser, enabling unmodified programs expecting a Unix-like environment to run directly in the browser. Browsix does this by mapping low-level Unix primitives, like processes and system calls, onto existing browser APIs, like Web Workers and postMessage.
They have examples of this tech enabling a terminal in the browser, a Latex editor, and even a client/server-based meme generator written in JS and Go.
Opensource.com asked the community to share about a time they sat down and wrote code that truly made them proud. Ten people responded with their experiences and learnings. Hereâs an appetizer from Greg Scott:
One of mine around coding goes back to college in the 70s. I learned about parsing arithmetic expressions and putting them into Reverse Polish notation. And then, I figured out that, just like multiplication is repeated addition, division is repeated subtraction.
You can think of Julia Evansâ new dnspeep
tool as similar to tcpdump
but specifically for watching your machineâs DNS queries.
One thing I like about this tool is that it gives me a sense for what programs on my computer are using the Internet! For example, I found out that something on my computer is making requests to ping.manjaro.org from time to time for some reason, probably to check Iâm connected to the internet.
A friend of mine actually discovered using this tool that he had some corporate monitoring software installed on his computer from an old job that heâd forgotten to uninstall, so you might even find something you want to remove.
It also probably comes in handy when debugging those pesky âcould it be DNS?â issues, but this might be a limitation on that front:
One thing this program doesnât do is tell you which process made the DNS query, thereâs a tool called dnssnoop I found that does that. It uses eBPF and it looks cool but I havenât tried it.
Documentation. You can treat it as a dictionary or reference manual that you look up things in when you get stuck during your day-to-day work OR (and this is where things get interesting) you can immerse yourself in a subject, domain, or technology by deeply and purposefully consuming its manuals cover-to-cover to develop expertise, not just passing familiarity.
In this episode we pull in perspectives and anecdotes from beginners and veterans alike to understand the impact of RTFM deeply. Also Sweet Filepath Oâ Mine?!?!
Carl Riis:
How do you advance your Linux skills when you are already comfortable with the basics? My solution was to come up with 10 subjects to learn and create an accompanying mini-project.
I learned Linux the slow/hard way: by fumbling around in the dark for months/years. This intentional approach will probably serve you much better than mine did.
Sean C Davis writing on CSS-Tricks:
A colleague of mine built a static site generator evaluation cheatsheet. It provides a really nice snapshot across numerous popular SSG choices. Whatâs missing is how they actually perform in action.
Sean set out to test 6 of the most popular SSGs on the market today. The results are somewhat expected (Hugo is the super fast), but there are some surprises in there as well (Hugo scales poorly, but it doesnât matter so much because itâs super fast)
Martijn de Haan:
Itâs been almost 3 years since I started working on this query builder idea of mine. Today is finally the day Mammoth hits 1.0. Mammoth is a no-batteries-included type-safe Postgres query builder for TypeScript. Hooray!
Congrats on shipping, Martijn! Hereâs a peak at the API:
const rows = await db
.select(db.foo.id, db.bar.name)
.from(db.foo)
.leftJoin(db.bar)
.on(db.bar.fooId.eq(db.foo.id));
Earlier this year on February 2nd, 2020 Jon Evans and his team of archivists took a snapshot of all active public repositories on GitHub and sent it to a decommissioned coal mine in the Svalbard archipelago where it will be stored for the next 1,000 years.
On this episode, Jon chats with Jerod all about the GitHub Archive Program and how theyâre preserving open source software for future generations.
Built in Go, askgit
is an open source CLI and coming soon web interface (linked above). With this tool in your toolbox, you can mine your repo for info like commit count by author on each day of the week:
SELECT
count(*) AS commits,
count(CASE WHEN strftime('%w',author_when)='0' THEN 1 END) AS sunday,
count(CASE WHEN strftime('%w',author_when)='1' THEN 1 END) AS monday,
count(CASE WHEN strftime('%w',author_when)='2' THEN 1 END) AS tuesday,
count(CASE WHEN strftime('%w',author_when)='3' THEN 1 END) AS wednesday,
count(CASE WHEN strftime('%w',author_when)='4' THEN 1 END) AS thursday,
count(CASE WHEN strftime('%w',author_when)='5' THEN 1 END) AS friday,
count(CASE WHEN strftime('%w',author_when)='6' THEN 1 END) AS saturday,
author_email
FROM commits GROUP BY author_email ORDER BY commits
Iâm not sure what makes this lightweight (their word, not mine), but it does load pretty fast from where Iâm accessing it. I definitely see what they mean by powerful, though, as wiki.js boasts many features: multiple editors, multiple auth schemes, search functions, comments, multiple locales, the list goes onâŚ
The demo is worth a thousand words.
gitqlite
is a tool for running SQL queries on git repositories. It implements SQLite virtual tables and uses go-git. Itâs meant for ad-hoc querying of git repositories on disk through a common interface (SQL), as an alternative to patching together various shell commands.
Mine your repoâs history for goodies. Hereâs how to get commit count by author email:
SELECT author_email, count(*) FROM commits GROUP BY author_email ORDER BY count(*) DESC
Love this humble transparency from Jennifer Yip on her journey so far with Lunch Money.
This whole journey is a learning process and Iâm grateful to be able to share mine transparently. Itâs important to not only celebrate the highs and stay humble, but also face the lows and forgive yourself.
Iâm sure there are many more mistakes to be made on the horizon and I look forward to writing and laughing about them in another 6 months!
Why a rotary cellphone? Because in a finicky, annoying, touchscreen world of hyperconnected people using phones they have no control over or understanding of, I wanted something that would be entirely mine, personal, and absolutely tactile, while also giving me an excuse for not texting.
This reminds me a bit of the finding your analog conversation we had at the end of The Changelog #378. Whatâs super cool about this phone is that it is super functional!
So itâs not just a show-and-tell piece⌠My intent is to use it as my primary phone. It fits in a pocket.; Itâs reasonably compact; calling the people I most often call is faster than with my old phone, and the battery lasts almost 24 hours.
Iâve had never really come into contact with hardware programming, working mostly in python or C#, until a friend of mine asked me for some help with programming a simple controller for RGB strips using Arduino Nanos.
Weâd, of course, fail spectacularly.
Not only did our hardware not work quite like intended and a few Nanos died in the process(but thatâs a story for another time), but I actually learned a lot from this and similar projects.
And I want to tell you some of my mistakes, what I learned by making them and how to prevent them.
Mark Christian, being đŻ% accurate:
Hello! This is my personal web site. Itâs not much, but itâs mine. After nearly a decade of just barely existing, Iâve spent quite a bit of time in 2019 trying to breathe new life into it. At this point, I think just about everyoneâbut especially folks in the software engineering universeâshould have a personal web site of their own. Let me tell you why.
As with any research Nadia Eghbal shares, this is a deep dive into understanding the user support systems present in todayâs open source. Itâs very detailed, highly researched, and more importantly itâs actionable.
Hereâs a sample of Nadiaâs closing remarks:
I barely scratched the surface on user support systems: thereâs a gold mine of data waiting to be played with. Iâd love to see more research on how support communities form and maintain themselves (particularly Stack Overflow, mailing lists, forums, and synchronous chat).
Why do some have only one or two answerers, while others have many? Does the growth of these communities mirror that of the code contributor community? Implicitly, a deeper understanding of support communities would help validate the growth model and hub-and-spokes model presented above.
Betteridgeâs law of headlines declares: âAny headline that ends in a question mark can be answered by the word no.â But first, letâs see what CJ Chilvers has to say (emphasis mine):
Weâre just numb to the Buzzfeed-ification of podcasts, even (especially?) in outlets like NPR.
Then came popular YouTuber CGPGrey (one my favorite podcasters) and his Project Cyclops. In short, this is a well-known, well-liked podcaster who is now advising people to stop listening to podcasts. He has promised to stop listening himself as well â he will only create.
Nothing against any of the people involved in this article, but all of it feels like an overreaction. If you feel that a podcast is wasting your time or that itâs not the quality production you want, the solution is simple: unsubscribe. You donât go and start a movement calling for everyone to stop listening to podcasts. Thatâs just â as Jerod Santo put it when we were chatting about this story â asinine.
In further irony, GCPGrey still wants you to listen to the podcast heâs creating, obviously.
Per Vognsen has started live streaming daily while he builds a computer from scratch. The backstory:
After working as a game developer and systems programmer for over 15 years at places like Epic Games, NVIDIA, RAD Game Tools and most recently Oculus, I decided it was time to take a break from professional programming and spend a few years pursuing a long-time dream of mine, a project Iâve dubbed Bitwise, where I want to share my passion and try to demonstrate by example how to build systems from scratch, with a low-level computing focus.
Sounds super cool, but also overwhelming. Did he really say, âspend a few yearsâ? For a project that ambitious, you gotta have goals:
My goal with Bitwise is to show that these things can be done much more simply and quickly than people realize if we strongly favor simplicity over marginal gains in feature completeness or performance. The goal is not to outdo or compete with any existing product; the goal is to show how things work with real hardware and software.
Iâm sold. I subscribed us to his Twitch channel and look forward to following along!
Seizure warning!
Earlier this year a friend of mine rented an analog video synth. A group of us sat in his basement while he projected weird visuals onto the wall â plugging in and unplugging patch cables, twisting knobs, and feeding in VHS source videos. It was mesmerizing.
Pixel Synth is a tiny, web-based alternative by Donald Hanson and is almost as mesmerizing. Itâs also neat that you can control it with a midi device, thanks to the Web Midi API and WebMidi.js.
A simÂple one-off page to browse multi-variant typeÂfaces. Google Web Fonts is full of awesome free, open source fonts. Itâs also full of fonts that may not be worth your time. Iâve been using this to mine Google based on his metrics, and start there.
I quickly discovered that a good metric for higher-quality fonts was the presence of four or more alternates. The Google Web Fonts directory does not allow that type of filtering, so I built this simple one-off page that allows you to browse typefaces that are true families. - Matt Wiebe
Would be neat if each font in the index had its own page with Dribbble shots tagged with âPT Serifâ and an email with updates on these worthy fonts.
The source is on GitHub if you want to contribute.
Fun fact. We used Yanone Kaffeesatz in our logo.
If our conversation with Gordon Williams about Espruino on episode #104 got you excited about hacking on Espruino or Arduino check out Coinâs Arduino-BLE Developer Kit.
A year ago, we were software developers venturing into the world of hardware. Not surprisingly, we encountered difficulty as we integrated Bluetooth Low Energy (BLE) into our product. At that time, existing BLE modules were confusing to integrate and didn't meet our requirements. So we decided to build our own.
If you havenât ordered your Coin yet use my referral link and help me get mine free!
Hey everyone! Itâs Friday, and a good friend of mine (@DeMarko) just tipped me off to a really interesting repository.
If youâre not aware of the Demoscene, you should be. Basically, they make awesome crazy videos with computer graphics. That sounds boring, until you realize that they work within crazy constraints.
For example, hereâs a ten minute Demo made on the Amiga:
http://www.youtube.com/watch?v=k8JbAdEzvMA
Hereâs one on the Commodore 64, in 1982:
http://www.youtube.com/watch?v=8Nfgdr4fOS8
While these videos are old, the Demoscene certainly isnât dead. Hereâs another C64 demo, this time from 2010:
http://www.youtube.com/watch?v=L8onlB0F1_A
Anyway, I could go on and on. Point is, many modern demos are made on modern computers, but with a different constraint: file size. Generally, itâs âmake an awesome video with graphics and sound thatâs less than 4k.â Thatâs smaller than a blank word document. Hereâs an example of one of these:
http://www.youtube.com/watch?feature=player_embedded&v=0w_xEUoK79o
Make sure to tell YouTube you want the 1080p (!) version.
Anyway, Iâll stop with the history. Hereâs the point: Thereâs a repository thatâs basically just a raw dump of the code people have written to make these kinds of demos. An archive of the last ten years of a bunch of people making awesome things.
The README warns that much of the code is hard to compile, and if you donât know C or C++, you probably wonât be able to make heads or tails of it. Note also that these are tools used to make demos, not necessarily the code of the demo itself. Point is, there be dragons in this repo. Lots of dragons.
You can check out the Farbrausch tools repo here.
UPDATE: One of my other friends (@davewilkinsonii) pointed me to this article, which refers to the above repository, but also has links to download source code of a TON of demos: http://www.displayhack.org/2012/the-great-demoscene-sourcecode-giveaway/
You folks create a lot of events on GitHub. Trying to mine and report on that data is definitely a big data problem. Ilya Grigorik wants to help the community get a handle on the GitHub firehose with GitHub Archive. With an API call you can get a range of GitHubâs public timeline data for all seventeen event types:
require 'open-uri'
require 'zlib'
require 'yajl'
gz = open('http://data.githubarchive.org/2012-03-11-12.json.gz')
js = Zlib::GzipReader.new(gz).read
Yajl::Parser.parse(js) do |event|
print event
end
The source is on GitHub. Be sure and listen to Ilya on episode #55 talking about Goliah, EventMachine, and SPDY.
If you read this blog, you obviously care about open source. If youâve never contributed to an open source project, though, you might have some cold feet about it. So, inspired by the Ruby 1.9.3 Documentation Challenge, I wrote up a post for my blog about how to contribute documentation to Ruby. I got some feedback like this:
@steveklabnik Hey this is awesome. It's time for me to solider up and pitch in. Thanks for the added incentive!â Chris Baglieri (@chrisbaglieri) May 10, 2011
So I figured something more general might encourage you to get involved with whatever open source project youâre using, even if itâs not in Ruby. Every project can use a hand, especially the small ones.
If youâre not contributing because you think youâre not ready, donât worry about it! I know that this is easier said that done, but really, youâre ready. A friend of mine posted an article about why he doesnât contribute, and Iâm sure that many people share these kinds of fears. Greg Brown responded and addressed some of his concerns, but most of the people Iâve talked to object for two basic reasons:
Letâs talk about each of these, in reverse order. Itâs true, you may have a busy life. I donât know your personal schedule. But Iâm sure you could find a spare hour or two, maybe on a weekend? Thatâs all it takes to get started. Most projects are built on the backs of a thousand tiny commits. You donât need to make a large contribution, even small ones are valuable.
If you have fears about the quality of your code, well, the only way youâll get better is with practice. So fire up that editor, and submit a patch! Generally, if something isnât quite right about your submission, thereâll be a discussion about it on GitHub, and everyone learns. Take this pull request, for example. Colin originally submitted a patch that linked to the wrong URL, wilkie mentioned this, and Colin updated his code. Itâs going to get merged as soon as I stop writing posts for the Changelog. :) But, this is generally what happens if your first submission is a bit off the mark. Donât be scared! This is how we all learned, from each other.
The âItâs too hardâ complaint usually comes out of âIâm not good enough.â But it can also happen if you try to contribute to a large project, where there are a lot of rules. Contribution guidelines, code coverage requirements, updating AUTHORS and CHANGELOG files⌠big projects need to have process to manage the large number of contributors, but this can certainly create a barrier to entry for newcomers. If process intimidates you, I have a suggestion: start small! Smaller projects often have little to no process whatsoever. Plus, youâll make someone feel incredibly good. Think about it: Python receives a bunch of patches every day, but if you had a little tool you wrote up on GitHub, and all of a sudden you got an email, âHey, someone has a patch for you,â I bet youâd feel pretty good about it!
When contributing to an open source project on GitHub, thereâs a very basic workflow that almost every project follows. Three steps: Fork, commit, pull request.
GitHub makes the fork step really easy. Just click on the âforkâ button found on any project page. Letâs use Ruby as an example. The project page is here. You can see the fork button on the upper right. It looks like this:
Click that, and youâll see some âhardcore forking action,â and then youâll be at your own fork! This is your own version of your project, and it appears on your GitHub page. For example, hereâs my fork of Ruby. Youâll see a URL on that page, and that lets you clone the project down.
$ git clone git@github.com:steveklabnik/ruby.git
This creates a ârubyâ directory with all of the code in it. Next, letâs add a remote for upstream, so we can keep track of changes they make:
$ cd ruby
$ git remote add upstream https://github.com/ruby/ruby.git
$ git fetch upstream
Now at any time, we can grab all of the changes the main Ruby repository by doing a rebase:
$ git rebase upstream/master
A small note: ruby still uses both svn
as well as git
, so they call the master branch trunk. If youâre doing this for ruby, youâll need git rebase upstream/trunk
.
Now that youâve cloned, you can do you work! I like to work in feature branches, as it makes things nice and clean, and I can work on two features at once.
$ git checkout -b feature/super-cool-feature
$ vim something
$ git add something
$ git commit -m "Fixed something in something"
Once youâve got some commits that fix your problem, push them up to GitHub:
$ git push origin feature/super-cool-feature
And then you click the pull request button:
Select your branch, change the description of you want, and youâre good to go! The maintainer of the project will look it over, you might end up with a discussion, and youâll soon get something accepted in somewhere!
The best way to contribute is to help out with a project that you actually use. That way youâll get to take advantage of the fruits of your labors. Youâll be more motivated, you already understand the project and what it does, and thatâll make it easier on you.
If you donât want to or canât figure out how to work on something you use, the next best way is to start using some new software! Keep reading the Changelog and pick a project that looks interesting, use it for a few weeks, and then contribute!
I hope this encourages you to get your hands dirty, roll up your sleeves, and contribute. Even the smallest patch has worth, so please, make some time out in your schedule, pick a project, and give it a shot. You might just find yourself hooked.
In a passing comment on IM Wynn told John Nunemaker how he loved the 404 page on Gemcutter.
The next morning, the world woke up to gemwhois. Now you can see who owns or is squatting on your favorite gem name:
$ gem whois httparty
gem name: httparty
owners: John Nunemaker, Sandro Turriate
info: Makes http fun! Also, makes consuming restful web services dead easy.
version: 0.5.2
downloads: 40714
$ gem whois somenonexistantgem
Gem not found. It will be mine. Oh yes. It will be mine. *sinister laugh*
We bring you back to Microsoft Build 2025 to nerd out with Craig Loewen on Windows Subsystem for Linux and Mads Torgersen on leading the design of C#.
Matched from the episode's transcript đ
Craig Loewen: Your home drive, right? So if you do pwd, youâre like at home/craig would be mine.
Welcome back to #define, our game of obscure jargon, fake definitions, and expert tomfoolery. Weâve gathered some awesome friends, new and old, to see who has the best vocabulary and who can trick the everyone else into thinking that they do.
Matched from the episode's transcript đ
Jerod Santo: This is mine. Yeah.
Derek Collison â creator of NATS and Co-founder & CEO of Synadia â joins the show to dive into the origins, design, and evolution of NATS, a high-performance, open-source messaging system built for modern cloud-native systems and part of the CNCF. Derek shares the story behind NATS, what makes it unique, and unpacks the recent tensions between Synadia and the CNCF over the future of the project.
Matched from the episode's transcript đ
Derek Collison: Yeah. And I knew the rug pull stuff, Iâve been listening, and I know Joran was talking about it, and others have been talking about it⌠And so Iâm totally aware. And what my hope is is that while weâre focusing on license change and rug pulls, that we up-level to have a dialogue of âWhy is this happening? What are we as an ecosystem failing at, and what can we improve upon?â
I said earlier on in the podcast, charity is not a business model. Well, until youâre a company that is self-sustainable, and youâre not just running on VC dollars, thatâs still kind of a charity model. So in other words, right now, our customers are funding a lot of the development, but the VCs who have funded Synadia are funding a lot of the development that everyone feels entitled to for free, right? And so my hope, and my sincere hope is that we can up-level and have a dialogue about âWhy do we think this is happening? And are there things that we could potentially improve from an ecosystem perspective, that could do that?â And again, Iâm not a fan of this, but one might be âNo, we donât want any projects that run critical infrastructure to be from just a bunch of folks, or a single company.â I donât think thatâs right, but that could be one answer. Or âHey, if youâre a big, well-known Fortune 100 company and youâre using something in critical production, give back.â Kind of like your rug pull thing, mine is âNo pay, not okay.â [laughs]