Anthony Starks: So the first one, we can start with SVG Go. That was created – that’s the oldest one. Started in March 2010. So Go was sort of introduced to the world in November 2009, and a few months later SVG Go was born. So that is the ability to, again, take data from whatever, and take those high-level concepts in SVG and make a picture. So sort of the subtype I like is programming pictures. So that’s sort of what we’ve been doing.

Subsequently, I created other tools… So I use that a lot for generating SVG. So just one more thing that I want to say is, in all my packages I really spend a lot of time on the documentation. And because these are visual tools, it’s really important to make sure that we’ve got really nice illustrations for everything. So I just wanted to put that in there. So that’s SVG Go.

[00:09:44.04] So let’s go back to 2013-2014, something like that… Again, in the corporate world things don’t exist unless they’re on a PowerPoint. Right? [laughter] So a lot of the world’s knowledge is sort of locked up in PowerPoint. And I’ve been doing a lot of PowerPoint and that kind of thing, and again, I’d use SVG Go for doing, again, architectural diagrams, I had my own sort of markup language that was specific for those kinds of things… Again, you could have used Visio and those kinds of things, but whenever I wanted to make a tool, I would make a bespoke tool for a particular kind of visual using SVG Go. But it didn’t really quite work to make that as a PowerPoint, so I said “What are the things that build a nice PowerPoint?” I became sort of a student of what makes a good presentation, looking at Steve Jobs’s keynotes and those kinds of things, using things like Keynote on the Mac… I said “What are the things that really make that?”

So one of the insights was start with a blank canvas. Throw out all of the defaults that the tools give you, and start with a blank canvas. And again, that’s another sort of learning, is the defaults that you get with things like PowerPoint, and Excel, and those kinds of things are terrible. They’re really bad. Maybe they’ll be better in this age of AI, but right now they’re bad. And a lot of bad visuals and things have been foisted on the world because people just click on the default, and it’s not their fault.

Anyway, so pulling back, I said “Hm…” Remember the Present tool that the early Go developers created? I said “Well, you know what? Let me make a parser for that, so that I can sort of build those kinds of things.” And then from there, I said “Wait a second, what am I doing?” Because it was not really specified anywhere, the grammar or anything about that, I said “Let me pull back, and let’s create a bespoke markup language just for presentations.” So the whole thing is a deck, and then within the deck you’ve got slides, and then within slides you’ve got text and graphics. Okay, that’s sort of the basics.

So given that, I created this tool to do that, and - it’s interesting, the first client was built for Raspberry Pi. Because it had nice graphics, and I had sort of a 2D graphics library that I was working with at the same time. So I could create nice presentations around that on the Raspberry Pi. But because you had a markup language, you can take that as a basis… So hm… Let’s make an SVG out of it. Let’s make a PDF out of it. Hm… Let’s make a PNG out of it. So from there, you can create a whole deck, and then boom, you’ve got your PDF, and then nobody knows that you didn’t do it with PowerPoint, or whatever… It’s just a PDF and slides, and you can walk through it. So that’s sort of the beginning of that.

And then after that, I built another tool on top of that called Dchart, that uses the elements – so it turns out those fundamental elements that you can use are also perfectly great for making charts and graphs. So for example, if you wanted to make, let’s say, a bar chart, what are the elements of a bar chart? We’ve got numbers along the bottom, and you’ve got lines that make the bars, that can go this way or this way… And you just lay them out.

[00:13:54.04] So one of the things that I also want to mention is all of these tools use a simplifying thing called a canvas. But that canvas is percent-based. So it doesn’t matter what the aspect ratio is, everything goes from x, from y, from 0 to 100. And it uses the coordinate system that you learned in elementary school. X goes from left to right, increasing, y goes from bottom to top increasing… Which is sort of flipped from typical graphics programming, but from sort of a user’s perspective, or from my perspective, that was a fundamental sort of insight. Then, you can just sort of say “If I want something in the middle, it’s at 50/50. Boom, there it is. And it doesn’t matter what size the canvas is. The size of the can be this way, it can be this way… It doesn’t matter, it scales to it.

So you only have to remember numbers from 0 to 100. So that makes it easier to think about and to reason about. And it also simplifies your layout, because again, you can think in those very simple terms, of where things are going to go.

So that was – so we were talking about Dchart. So Dchart is built on Deck, which is the markup language… So therefore, your charts and your other elements are no different. There’s no sort of pasting it in, or whatever. It becomes a fundamental part of it. And then from there, that became a library that you can embed. And then again, you can have programs that work together… Again, all of my tools are command line-based. So you work on the command line, you can build your own tools around that… And again, from there – so before, I’d been writing sort of the markup language in raw XML, like sort of a caveman. But I realized, “Wait a second, I can create a domain-specific language, that will abstract –” In many ways, it’s a one to one. So if I want to draw a line, the line command looks like sort of the line markup. But you have other larger things, like if you wanted to draw star, that’s actually a polygon, for example. So you sort of work in those kinds of terms.

And I sort of built up the things as I needed them. So if I needed something that wasn’t there, I would just create an element in the markup language, and then create a way to express it, and away I went. So that was Deck, Decksh… And again, in - let’s see… In 2019 I learned about the work of W.E.B. Du Bois at the 1900 Paris Exposition, where he wanted to tell the world the progress of African Americans since emancipation. And he used a series of charts and graphs to do this, and it was all documented in a book. The name of the book is W.E.B. Du Bois Data Portraits Visualizing Black America. And in this book, they have each of these – they were large poster-size things, and I said “You know what, developing these tools concurrently with learning about that, why don’t I start and try to recreate them?” So I was on vacation, I said “Let me start, see how far I can get.”

So there are around 60 visualizations in the book, and I started in July of 2019, and I finished by October. So it took a few months to do it… But once they were done, I had this nice collection, I had learned a lot about how do you develop a domain-specific language… Again, the philosophy is let’s start from the minimum, and then let’s just build only as we need it, going up further. So that’s sort of the journey of those sort of sets of tools.

Mikeal Rogers: It’s a whole ecosystem. There’s a bunch of other blockchains as well, there’s chains that are very compatible, versus ones that have some difficulty with compatibility… It’s important here to separate what is a technical limitation because we just haven’t figured out what to do yet, but there’s an incentive to fix it, versus issues that are actual incentive problems. The players in this industry don’t want to do this, so it’s not going to happen. There have been things floated to do in Bitcoin that would be good for the people that use Bitcoin, but maybe not good for the miners… So that stuff in Bitcoin is probably not gonna happen, and may happen in some of these other chains that are being created.

Transaction fees - there’s nobody who wants all the transaction fees to be super-high, actually. The business model here is to mine new tokens, and the utility of those tokens is somewhat affected by how difficult they are to transact… So there’s a real incentive to try and get that down over time, rather than up, because that’s just not where you’re making money; nobody is making money on that. They wanna get it down.

The same thing with the power consumption. Nobody wants to pay extra power bills. Everybody’s trying to get the power consumption down. It’s just literally a technical problem that we’re iterating towards.

And Flow, for instance, like we just talked about, and NEAR Protocol for that matter - they already use proof of stake, and use way less power, and they have much lower transaction fees. Not because of that… Because of other reasons, mostly.

Anyway, if you look at the ecosystem as a whole, you don’t really have a transaction fee problem because you have all these other chains that you can use that have very low transaction fees… And if an asset gets to a certain price, you can actually move it to Ethereum. And not every protocol has worked out exactly what this is, but one of the really popular things right now is called Polygon; it’s the sidechains that are Ethereum-compatible, which means that they look more or less like Ethereum. And so if you ever want to transfer an asset from there to Ethereum, you can.

So there you can create assets for sub-penny, do all these transactions, and if some of them get to be like a million dollars and you really wanna put them in the more secure, stable, broadcasted network, you can then move them to Ethereum and pay that transaction fee. And I think the way to look at this evolving is that the way that you have to scale it is through more decentralization, not more centralization. So the solution is not to get everything into Ethereum and make Ethereum so good that it has sub-penny transaction fees, and is doing a billion ops a second, or something; that’s not realistic. It’s fine to just have these sidechains and to have the assets move around.

Protocol Labs, Juan Benet who started the company - he’s always had a lot of vision about how this stuff works in the future and about this protocol interop, and we’ve been working on standards and protocols that are just open source, open protocols that we implement. A lot of people, I think, for a long time, thought that we were over-engineering. They tended to be like “Oh, why don’t you just do your own custom thing, like we’re doing over in this chain?” And we were like, “Because it’s not about your chain, it’s about the compatibility between the things”, right?

Carl Johnson: His name is Carl Johnson. So if you search for my name without the m, you get pictures of Carl Johnson posing in front of cars in very low polygon resolution.

Craig Kerstiens: I think Postgres just moves forward each year with something new. JSON, JSONB as the shiny ones… I can go back in probably each release and there’s some interesting, new thing. One big area for me is indexing. Most databases have an index type; when you do “create index”, it’s creating a B-tree index. If you have a CS degree, this is what you learned, the basics of a B-tree. Postgres has had that for a while, and it’s also getting more exotic with those indexes.

With JSONB, when you index it, you use a GIN index (generalized inverted index), which flips it on its head and basically indexes every key and value inside that JSON document. So really interesting that I don’t have to say “Index this column or this little piece”, it indexes everything for me, so when I query it, it’s fast. Postgres has five different index types. For a long time, I would read the docs and look, and I just kind of like got over my humility and asked the core engineers “Can you actually explain this to me in plain English? I read the docs and I don’t get this…”

So you’ve got GIN, which is really useful when you’ve got multiple values inside the same column. So if you think about that, like JSON - you’ve got multiple keys and values, you’ve got arrays, that sort of thing. Postgres has an array data type; that’s super-handy. If you’re doing something with tagging or categories, it’s wonderful. Please don’t go build the whole categories table and join against it. You don’t need it. Just go ahead and use the array type. So GIN - really useful on arrays, JSON, those sort of things.

You’ve got a GiST index (generalized search tree). The way I best describe it is when you think about – you’ve got record that overlap values between rows… So if you think about full-text search; so you’ve got like a sentence, and maybe you wanna index on “the dog”, but not “the”. So you need both parts, and hey, “the” may appear in a bunch of other places, but it’s not gonna be in the index. So you’ve got parts of the values that span across rows.

Geospatial is another one where GiST is really useful. If you’ve got polygons - how do you find the dot within the polygon? So it’s useful for things that can kind of overlap in that sense.

You’ve got SP-GiST (space partition GiST). I only know it’s useful for phone numbers. I keep asking the question “Give me other examples”, and they’re like “Um… Phone numbers. That’s one.”

Rob Fletcher: Yeah, and I’d also like to just say that one of the things that really attracted me to this problem set in general is that using geospatial information in especially deep learning models - it’s not always clear how to do that. You mentioned extracting these into the standard inputs that you would normally get, and that’s definitely something that we do sometimes… But I think also we try to put a lot of thought into “Well, how can we use the objects themselves as input?” And this is, in general, a very difficult question, because you take something like a polygon, for example… You know, try putting a polygon into a neural network, and it could be hard, because if you have some arbitrary number of points that make up vertices of the polygon, it’s really difficult to fit that into something that’s like a confined or a predefined neural network that has a certain number of inputs.

So I think in general it’s actually a pretty difficult problem to tackle in the general sense, and that’s one of the things I think is really exciting about this field.

Chris DeBellis: So if you’re familiar with the COCO dataset, over the last few years folks have been taking the COCO dataset and providing the masks; so they’ve been going in and annotating, providing that polygon around the individual elements or the individual objects within each of the images… So that allows the people that created the original Mask R-CNN network to do transfer learning, which is you start with an existing set of weights. So they were able to use an existing set of images that were already annotated, and create their algorithm. Then what we do now is we take those weights that they use to create the original Mask R-CNN network from, and we use that as the starting point to train for images that we want to now detect.

So let’s say there’s something else that we want to detect that’s not part of the original COCO dataset. So we train with new images. We have to go out and obtain those images, annotate those images, and then apply the training on those images with the COCO weights as our starting point, and that’s actually called transfer learning.

Rachel White: I think that the people that primarily do data viz programming, it’s helpful for it; it probably makes a lot more sense. I’m looking at it and a lot of it makes sense to me, but also why do you have me put in a semi-regular polygon – oh! Those make sense…

Andrew Godwin: Yeah, and even better - keeping with the theme - it has admin widgets. If you make a point field or a polygon field and load the admin up, you get a map for the polygon that you can drag around and edit. So all of that, you can just start using it straight away; like, “Oh, I want to see what’s my shape of london looks like and it happen to show it on a map which is super nice.

Jordan Eldredge: [00:12:07.10] So like many file formats, it’s really just a zip archive by another name. That’s the wrapper, I guess you could call it. And then there’s a convention of files inside of it, which are actually just sprite sheets. So they’re Bitmap files, and each file contains all the buttons for the main window, but in a very particular layout. And that’s the main bulk of it, is these sprite sheet images. But then there’s also some INI files for “What colors should the text be?” And there’s even a region.txt file, which says “Define some polygons of which areas of the skin should be transparent.” So there’s a wide variety of just little different kinds of things that are in there, and they all have file names that they’re expected to be located at.

Suz Hinton: There was a product called the 3D Streaming Toolkit, and this is something that I worked on during my time at Microsoft because of a customer need. Essentially, it uses WebRTC with virtual reality and mixed reality devices, and also mobile phones. So let’s say you have someone with a low-power Android phone who’s trying to use Google Cardboard; or let’s say you have a HoloLens, which the entire processing unit is actually in the headset, so it’s not connected to some [unintelligible 00:15:04.17] even though the HoloLens is actually a pretty performant device.

Let’s say you’re trying to manipulate lots of polygons in space; it’s either a complicated game, or it’s for a medical application such as being able to rehearse or understand some surgery you need to perform before you actually do it… A lot of these devices are just not gonna be able to crunch those polygons in real-time, so something that I worked on is called the 3D Streaming Toolkit.

What it essentially does is there is a central server that you have a WebRTC peer relationship with, and you’re able to stream in real-time; it will render the 3D based on the data channel feedback you’re giving it on pitch, and role, and how you’re actually using the accelerometers on the device… And it is so fast that you don’t get that motion sickness, because the feedback in the data channel going back and forth and saying “Okay, this person’s now moved their head this amount” - it’s able to do enough look-ahead and prediction as well, in order to render a 3D video in real-time and stream that back through the video channel back to the VR device.

[16:12] I hope that I’ve done that explanation justice, because obviously there are a lot of details that went into this… But I worked on the web client sample; even though my name is not in the commits, I was pairing with people on that… So that was a really exciting thing to see, because it means that people don’t have to have these really high-end, expensive devices if you can use this protocol in this way, which I thought was really cool.

Jerod Santo: Yeah, which is so nice when it works. But with D3 you’re actually working at a lower layer, where maybe that library is actually telling D3 what to do underneath the covers, but you’re drawing polygons, you’re drawing lines. You’re actually programmatically or imperatively telling it what to do at maybe the drawing layer, if you think of it that way. So that’s powerful, but difficult to pick up. Charting libraries at higher levels are easier to pick up, but less powerful… How do you help folks make the decision of like “Where do I fit in? Do I go for D3 and just dive in, and I can be a craftsperson, and just get underneath there? Or am I happy with a higher-level thing?”

[12:15] Do you have to make those decisions on a regular basis? I mean, you’ve learned D3 very well, so you’ve grabbed the power tool, but what about somebody new? How can you help them make that decision?

Chris Benson: Yup. I’ve gotta say, I would love to have somebody from DeepMind who is directly working with this AlphaStar implementation onto the show, and just talk through some of the details of this… But it feels like the implications of how this could be used – this is a great test case, but it really feels like we’re turning a corner at this point.

And I guess while we’re mentioning turning a corner, there was another article that I ran across… The article is on Polygon.com, and it was entitled “An unbeatable poker bot offers glimpses of video game AI’s future.”

Suz Hinton: Yes, it’s so true, and it was like an Easter egg way of doing things, but it also meant that instead of having to have all of these separate pieces of an image an tile them all together in some kind of table layout in order to create these different hot spots you could click on, you could essentially apply a map HTML tag to a matching image tag, and you could set up these regions (I guess); it was kind of similar to SVG, where you could either draw like a rectangle, or you could draw polygons, and then each little hot spot would end up being a href just to go somewhere else, which was really awesome. So you could have a map with lots of different cute little buildings and landmarks, and that could be the navigation for your website.

[03:45] The reason why I really liked it was because looking back over it, it was kind of weird, but it did solve a pretty awkward problem that we had, where we couldn’t lay things out as nicely, like what we have with grid layout these days… And it just forced people to be really creative. The navigations that people made with that especially was really entertaining, and there were all these really cool online tools that allowed you to generate these image maps, so you could actually draw the polygons instead of having to hand code them… So there was this really cool culture of being able to push the limits of this HTML tag, for sure.

Brian Ketelsen: The neat thing about it is that you can choose to output a regular file, or if you choose a .gif, it will create an animated .gif for you and show the process while it creates all those polygons, and that is just so cool to watch. I’m really enjoying that.

Brian Ketelsen: Polygons everywhere.