Making web art the hard way

Hello, world. It is your internet friend. I’m Jerod, and I am joined today by Amelia. What’s going on?

Hey, hey. I’m so excited about our episode today.

I am as well. I’ve never felt more out of my league or out of my depth than when I’m with like a digital/real world artists… We’re joined today by Alex Miller, a.k.a. Space Filler. What’s up, Alex?

Hi. Thanks for having me.

I appreciate you joining us. So I came across your work, as one does, on the internet, by following a link from somebody else, in some other place… And I landed on Grid World. And I immediately thought “We’ve got to get this guy on JS Party, because this is so cool.” And let’s talk about it. But before we start there, in reading this Grid World piece, which is kind of a - what’s the word? Not literature… An essay? It’s a –

Yeah, I think I’d call it an essay…

It’s an essay/art hybrid, which kind of is one of the things that you do… And in it you write about your dad. And one thing that you say about your dad is that he and his brother created a computer game called Mist. And then you said that nobody under the age of 30 knows what Mist is. And if you can tell by my gray hairs over here, I am not under 30, I know Mist, I freaking loved Mist in the ‘90s, and I feel cooler just by - the transitive property of mathematics…? I don’t know, just by knowing you, whose dad created Mist. That is so rad, dude.

It’s really cool. Yeah, I mean it’s obviously it’s hard for me to have perspective on it, being his son, but that is the reaction that a lot of people over 30 have to it. I think it touched a lot of people.

So Amelia, as we date ourselves here… Mist - know it or not?

Oh, yeah, I am. This is actually something I played with my dad. When me and my sister were kids, we’d sit on his lap and he’d play Mist. And one, it was terrifying… A lot of trauma associated with the world of Mist. And two, we never finished it.

Oh, it was hard.

It was long, it was hard, it was spooky… But also very cool.

Yeah, there were moments of spookiness, and there were also moments of just like awe, as it was just this beautiful world, this kind of first-person puzzle… Alex, how do you describe Mist to those who haven’t played the game?

It’s like a point and click puzzle adventure… Yeah, it has kind of like an atmosphere to it that I think some would describe as spooky, but it’s just a very specific atmosphere, that I think a lot of folks relate it to.

Yeah, you’re dropped into this world, and you’re just left – like, there’s very little explanation or tutorial. And you’re just kind of like “Well, here we are, in a room, or on an island”, I can’t remember the first part… And just start clicking on stuff and seeing what happens. And I think prior to Mist, the only PC game or computer game that I’d played - I was always an NES guy - was like King’s Quest 4, or something, which was very much in the world of animation… And this felt, as a young person in the ‘90s, a real new world. Like, it didn’t feel drawn. It felt almost like you were just there. Of course, it was pixelated, to a certain extent. Nowadays the graphics probably wouldn’t hold up as realistic, but back then, as a kid, I was like “This is a whole new world.”

I’m curious, did you get like sneak peeks of the game, or did you have to play it?

So for Mist, that was kind of – I forget the timeline exactly, but I was born in ’91, and I think Mist came out right around then. Maybe it was ’93 or something, so I was still a wee babe. But I do have like a very specific memory of my dad showing it to me… And he and his brother made other games as well, that were more targeted towards kids, and I actually have much stronger memories of playing those earlier games, that never became as popular, but those really made an impact on young Alex.

That’s so cool.

Yeah. So when did you realize that Mist was like beyond something that your dad made, or like something that other people liked? I mean, in the piece you write about how you came across it at this museum, and there was this section about Mist at the museum… And it was a very big deal, because it was kind of groundbreaking in certain ways. I wasn’t even a PC gamer kid, but it was like, I heard about Mist and I had to go and play it. So you were maybe a one-year-old, or a two-year-old at the time… When did you realize that Mist at this time in the ’90s was like this big deal that your dad made?

It’s really interesting, because I’m sure if I would have been older, it would have been more obvious to me… But I didn’t – I don’t think I quite understood it until I was maybe in my 20s or so. And then when I’m talking with my friends, a lot of them don’t necessarily know what it is, right? Because when it came out, they were also one and two. So it was more like their parents would know about it. And so yeah, it’s kind of tricky, because I don’t think I ever quite understood the impact until it was already kind of a little bit over.

[06:07] And not to say that – you know, folks have continued to enjoy it, and there’s still a certain amount of fandom about it… But I think it’s true that a lot of younger folks – you know, the video game industry moves really, really quickly, and it was quickly eclipsed by things that came after it.

There was one moment I remember - I think I was in elementary school, and I checked out from the school library a Guinness Book of World Records, and Mist was in that book. There was like a little blurb about it.

Wow.

And that I think was kind of - that was the moment that I was like “Oh my gosh, this thing that my dad worked on is in this book that’s at school. That’s crazy.”

That is crazy. So your dad was very influential in your life, especially with regards to this piece that you wrote about grids. So this also spoke to me as a fan of grid paper, graphing paper… And this particular bit that you say at the beginning really resonated with me: “The emptiness of a totally blank page intimidated me by demanding that I make the first move. But graph paper invited my participation by steering my pencil in the grooves of its strictly regular lines. The grid was like a friend who had already done half the work for me”, and I feel this exact same way, but I never really thought about it in those terms, of like “Why do I like graphing paper?” Or “Why do I hate blank pieces of paper?” Or an empty, white canvas that is a browser viewport before you’ve actually put some HTML onto it. And it’s like, there’s just – you can put anything, anywhere. It’s like the ultimate paradox of choice, right? But with graphing paper, with grid paper, kind of just like “Hey.” It’s approachable. Here’s a starting place, right here where these two lines cross. That’s pretty poignant.

Thank you. Yeah, I think it’s something that I have thought about over the years, my love for graph paper. And it’s something that a lot of other folks - it resonates with them when you talk about it. Like, isn’t graph paper so fun? Yeah, it is. But there’s never – I think we just take it for granted, that it’s just this tool that exists in our world. And you use grids for everything, and you use graph paper for all sorts of stuff, and we never really think about where it comes from, or why we’re using it… And so the essay is kind of an attempt to talk about that stuff, but definitely with like a personal framing, because the first thing I asked myself was “Okay, when was the first time that I actually encountered a grid? What is my first memory of that?” And that kind of became like the starting point for the essay. And then I go on to explore kind of like different ways that grids have influenced the world, and computation, and user interface design, and such.

Amelia, you are quite artistic, I will just say; at least very good at design. And I’m not sure if you consider yourself artistic in the sense of like drawing, and stuff, but probably - you can speak to that. I think you are. Do you like a blank page? Do you prefer a grid? What are your feels on the subject?

Oh, I love this… Sorry, I was just scrolling up and down this piece…

I know.

There’s so many good visuals. I hate blank pages. I like grid paper. I also like recycled newspaper sketchbooks, because - not that there’s a grid, but it doesn’t look like a pristine, blank page.

I see.

It’s like, you can mess it up, and it’s not going from like 100% new to completely destroyed… So it’s less intimidating. But I like this concept of “Constraints are super-helpful when you want to be creative.”

I love that. Yeah, constraints are everything, I think.

I’m curious, is there something specific about a grid? Or would a triangle – or like a square grid. Would a triangle grid being just as interesting?

[10:05] Oh, that’s such a good question… I thought about that when I was writing this essay, because there’s – yeah, you can think of it, like lots of different shapes that can tile the plane regularly like that… And I do think that there is something special about a square grid, because for whatever reason, the space that we live in has this dimensionality to it, it’s a three-dimensional space, and the axes of that space are orthogonal to each other… And the two-dimensional grid kind of goes with the grain of that orthogonality; the axes are perpendicular to each other on a two-dimensional grid. So I’m sure somebody who is better at math and physics and understands this stuff could say it more poetically, but that’s always kind of been my impression, is like the square grid just describes the world a little bit better. It just fits into your brain a little bit better, for whatever reason.

Oh, that’s interesting. I have enough psych and neuro background to know that there’s something about –

There’s something to it…

There’s some brain cells that specifically encode horizontal lines, and also vertical lines, and there are more of them than ones that go the other directions, but… Yeah, I’m completely out of my depth when I talk about it more.

I think that you’ve breached far enough that I think we can all have a takeaway, at least… And I’m not gonna go any deeper than that. In his piece, Alex, you talk about how – especially, now we talk about computers, right? So the 2D world that has been the interface into our computers, right? The monitor, which is very much just an X, Y axis, grid, if you will… It’s behind everything, and you go into some of the details of like early design of user interfaces, the Macintosh, and stuff… And then one thing you also say is that “If you want to know how something is made, you should look for the grids. They are the ever-present behind the scenes structure of our cities, our machines, our homes and our lives, our maps…” You start off talking about mapping… Our games that we play, our computers… Our charts that we love to look at and see “Are things going up or down, or left or right?” Or quadrants… We are grid-focused humans, aren’t we?

We are, yeah. Definitely. And that sentence of the essay was actually kind of like – that was like the first thing that I wrote, in my first draft. That was my first initial idea. And it came from - my partner showed me some art piece… The name of the artist is escaping me right now; maybe it’ll come to me… But she showed me this image of like this final art piece that this artist had produced, and then the plan for the art piece, side by side, and so you could see how the artist was designing the final piece. And in the plan for the piece, there was a grid visible. And I immediately – my first impression was “I think that the plan looks cooler than the final piece.” Like, taking away that grid scaffolding actually kind of makes it less aesthetically pleasing to me, personally. And I find that a lot – like, I love seeing the plan for the thing sometimes more than the thing itself. It’s really cool to see the plan for a design system, as opposed to seeing like the design system implemented on some webpage.

Yeah. I kind of have that going on right now at my house… So my oldest son is doing a pumpkin patch this year; he’s gonna try to sell some pumpkins. And so we went out and we tilled up a 50 by 100-yard space on our land, and he got out some graphing paper, and it was like “Alright, we’ve got to plan out this pumpkin patch.” He put each hill eight feet apart, we need 80 hills, he has bought all these seeds, of various kinds… And so he starts charting out, you know, each square is four feet, or whatever it is, and he draws out the little deal.

[14:07] And then I point out to him, “Hey, if you put like A through F on the left-hand side…” Because it’s like six rows, 12 columns, and one through 12 on the bottom. Now it’s like Battleship. You can basically just be like “Okay, this kind of pumpkin starts at A12, and this one ends at B7”, or whatever. And his eyes lit up with that. He’s like “Oh, it is like Battleship.” Now it was fun.

And the thing that we ended up drawing was really, really cool… And then we go out into the real world and we actually start to do it… And it’s still cool, but it’s not gonna map directly onto that perfect grid anymore. It’s gonna be off by this much or that much. There’s a rock here, there’s a hill that we misplaced… And also, the pumpkin patch is on a slope, and you have to account for the angle… Anyways, the point being is I think I’m gonna like the design on paper better than I’m gonna like the actual patch… That’s analog.

Totally, yeah. I love that you are passing on the grid to your son, as well.

It was actually just circumstantially… I hadn’t read the Battleship reference until after I told him about it, and I was like “I just used that!” Because in your piece also you talk about Battleship, and how we see grids in our games and don’t even realize it… And yeah, so that resonated as well with me.

We should probably get a little bit into the other aspect of what you do… You’re not just an artist or a writer, but you’re also a programmer, and you use software to make your art. And as Amelia referenced, as you scroll this piece, which of course is linked up in the show notes, there’s art involved with this essay. And it’s all grid-based art, and it’s all JavaScript driving things behind the scenes… So maybe give us a little bit of background into your work that you do beyond just this – this is just one of your pieces. You’ve got a bunch of work, so much that I had to ask you, “What do we talk about?” Because you have so many pieces that we could talk about on the air. Give us a background into some of the art that you do, both in the real world, and digitally… What drives you to make certain things?

Sure. So I actually started programming because I was interested in creating digital art. I came across a guy on the internet called Daniel Schiffman, who is an artist and programmer and teacher, and he has an amazing YouTube channel where he explains basically how to make art with code. And I’ve found him in high school. So this was before he had a YouTube channel; I think he was teaching at NYU, and he had put his course notes online, so you could see them publicly… And I was just totally transfixed by these course notes. I forget how I found them, but I was like “Oh my gosh, this is so cool. You can create these amazing images and interactive content using code.” I didn’t really quite know what code was, but I was like “I want to learn this so bad.” So that’s actually kind of the first, or one of the first encounters I had with programming code, was through a desire to create art.

And then I went on to study computer science in college, and that kind of turned into a professional career as a programmer, but the art side of it has always stayed with me. So it’s always something that I’ve pursued kind of on the side right after I come home from my day job. And I’ve done all sorts of art using code, I’ve created physical installations that are interactive… So if you’ve ever been to like an art gallery and seen something that’s projected, that you get to like play with, that sort of thing - I think it’s called like “media art”, is the term that folks use.

I’ve also done visuals for musicians, and for bands, and little music videos… And then I’ve also done kind of more web-based stuff. So like the Grid World piece that we were talking about is more of a piece that’s web-native, intended to be viewed on the web. So kind of a wide, wide range of work.

I work a lot with collaborators… So the SpaceFiller name is kind of – I think of it almost like a band, where there’s like different members of the band that join for certain gigs. So I have my primary collaborator, Alex Nagy, who I work with on a lot of the physical installations… And then - yeah, just a slew of other folks who help out on various projects.

I think I missed this… Is this still kind of like your side work? Or has this taken over for full-time what you do?

It kind of oscillates back and forth. But I would say there have been periods when it has been my full-time thing, but in general, it’s a side project.

Break: [18:51]

So this Grid World piece that we brought up already is interesting technically to focus in on, because one of the things you mentioned about it is how you eschew the use of frameworks. And before the show, Amelia is like “This is where I get shamed, that I use so much JavaScript, and Alex is not going to.”

[laughs]

And you kind of just went vanilla JS from scratch to build all of the tooling you need in order to animate these grids. It’s probably difficult for the listener to know exactly what we’re talking about if they haven’t scrolled the site… But interspersed amongst the prose are these grid animations, which are basically attached somewhere to your scroll. And so as you scroll them into the frame, they’re going to animate. They’re kind of grids of grids. Everything’s kind of a pixel square at the atomic level of this piece. And then they move – one’s even called Battleship, I believe, if you look inside at the DOM.

So you built that, and you didn’t use – I don’t know, Amelia, what would you grab? You’re like “Okay, I’m going to build something like–” Because you do really cool animations as well, and especially on your blog…

Oh, yeah. I go to npm, I download every single package… [laughter] I throw them all in there…

Right… But it works, right? I mean, you put out some really cool stuff.

It makes my life easy… [laughs]

Right. So Alex, why didn’t you just go to npm and just download the Grid World package and be done with it all?

Yeah, I’d love to talk about this, because this is something that I think is really interesting… So I just want to back up a little bit and describe that this Grid World piece that I made was for a publication that my partner, Shelby Wilson and my friend Max Neely-Cohen run, called the HTML Review. And it’s like a web-based literary journal.

Super-cool, by the way. You can tell your partner, it’s just super-cool. I love this whole concept, I love the execution… So just adding a little props in there; go ahead, continue.

It’s awesome. And so much of my work kind of comes from conversations with her… And I actually have like major impostor syndrome being on this podcast, because she’s like a proper frontend web dev, and I am constantly having to ask her questions about “How do I get the –”

Oh, I should have invited both of you had I known that this was a dynamic duo… We would have had two guests on the show.

You should have her on separately, but… You know, you described how the piece reacts to scrolling, and I definitely had to ask her, “How do I get the scroll position correctly?” Because I was having trouble with that. So she helps run this thing called the HTML Review… The entire ethos of the HTML review is getting back to the basics of the web. Like, getting back to the primitive building blocks, “What can we do with HTML and CSS, and maybe just like a little bit of JavaScript, or maybe not any JavaScript at all?”

So that’s been like a really inspiring thing for me, because I think all professional programmers can relate to just kind of like – I don’t know, npm exhaustion, where you’re constantly just like downloading libraries and frameworks, and reading through documentation… And it can just get a little bit like boring, or it can kind of feel like you’re detached from the process a little bit.

Now, obviously, those frameworks are incredibly useful, and in my professional work I’m definitely downloading all the dependencies from npm. But for this piece, because it was for the HTML Review, where the whole point is to kind of see what we can do with the old web, how can we make these web technologies more personal and more intimate, I decided, “Okay, I’m just going to not use any dependencies. I’m not going to have any libraries, I’m not going to use any graphics-rendering engines. I’m just going to use the basic browser API to do everything.” And that ended up being quite a challenge, because there’s a lot of things that these libraries offer you that you take for granted. For example drawing lines - it turns out drawing lines is really hard. And there is a primitive mechanism for drawing lines using the Canvas API, but I really wanted to use WebGL, because I wanted to do stuff with shaders… And so if you’re using raw WebGL, drawing lines that have a thickness is not something that you get out of the box. That’s something that your graphics engine would provide to you.

[24:17] And so I had to read up on algorithms for drawing lines… Luckily, there’s this amazing tutorial by - I won’t pronounce his name right, but I think it’s Matt Deslauriers, who has this tutorial called “Drawing lines is hard”, where he goes through the trade-offs of all these different ways of drawing lines… So that’s just an example of one of the things that I had to build to create this interactive essay, without using any dependencies.

I think one – I mean, I joke, right? Sometimes I throw everything at it. But I really am kind of a purist. I think you should, as much as possible and practical, do everything yourself on the web. I also think most web developers should do at least one project from scratch, because the field gets a lot into “Oh, do you know React? Do you know Vue? Do you know Svelte?” and it’s like, “I’m a good web developer, because I know React, and I can get all these jobs.” And I think there’s not enough appreciation for what the frameworks are doing, and then people run into all these bugs because they don’t really understand the underlying technology. So I really appreciate that message. I’m also curious, are you usually using WebGL, or canvas? Or what tools are you reusing over and over again for your projects?

I have done a couple of projects using WebGL. So for the graphics for this essay I was building on prior work that I had done. For example, the line drawing algorithm was something that I had already investigated for a different project, and I have a little particle system, like a very, very basic physics simulation framework that I have created as well, that I built on top of. So I do have kind of like a set of like personal, very, very small, humble personal libraries that I used. And because I have complete control and knowledge of the code there, I can rip things out to decrease the file size. I had this obsession with this project of trying to make the file sizes as small as possible.

So yeah, it definitely is building on prior work. And then for some of my other installation-based work, I’m using graphics, I’m using shaders, for example, so I have some experience there. So it wasn’t like starting completely from scratch in terms of the graphics side of things, but it definitely like pushed my ability a little bit to have this constraint.

We talked about constraints earlier in terms of like the constraints of the grid, and I think constraints are always where the coolest stuff comes from. Because for this essay, giving myself that limitation of not having any – you know, not depending on any external libraries, it really informed what I could do. Because I couldn’t do some of the ideas that I had, because I didn’t know how to implement them, and so that forced me to kind of be like “Okay, well, what can I do with the tools that I do have?”

What was harder, you think, or more time-consuming - the prose for the essay, or the code for the Grid World?

Oh, definitely the prose say.

Okay.

Yeah, that was like pulling teeth for me. I find it very difficult to write… Whereas the kind of purely technical challenges in the code are just – I feel like I have a lot more energy for that, or it’s just like a simpler… It’s a more well-defined problem. And then kind of the creative visual aspect of it I find really enjoyable as well; kind of like that feedback loop of making a tweak, and seeing what it looks like to me. I find that certainly easier than writing. I don’t understand how some people can write so quickly and so well.

[27:56] I really can’t either. We had Cory Doctorow on the Changelog a couple of weeks ago, and he talked about how he writes so prolifically… I mean, the guy has seven books in the works right now, and they’re all going to publish at some point. And he also blogs daily, and he writes deeply, daily. And I just had to ask him, “How do you –” Because for me, writing something of significance - you know, sure, I can write out a quick thing here or there, I can communicate. But writing anything of significance feels like such an undertaking that I avoid it, because I just know how painful the process is going to be. And yes, it’s usually worth it at the end, and all those things, but that doesn’t help me get started anyways, because I know how much it hurts to bring forth a thought in a final form.

And I agree that coding feels so much easier to start, because I guess it’s iterative, there’s like little wins along the way. I guess you can say with writing “Oh, I love this paragraph”, but I don’t tend to love any of the things that I put out. Whereas maybe this function I wrote, I do kind of appreciate how it turned out, you know? And so I wonder what it is about that. It’s all writing at the end of the day, but different types of writing have different difficulty levels. Amelia, you’re nodding along… Do you agree?

I hate writing. So I wrote a book on a JavaScript framework, and the only way I could do it is you start by writing the code, right? Like, “Okay, every chapter is going to have a focus. For this chapter, we’re going to write this code snippet.” And then you go through the code, you write it, and then you’re like “How did I know what to write?” And then you just go through the process. But anything creative, anything not linked to code - it’s very, very painful. And I wonder, if it’s a matter of like how many hours we already have in it - like, for drawing, people who aren’t good at drawing hate it, and they’re like “Oh, I’m not an artist” or “I can’t draw.” But the people who are artists have just drawn enough that they like what they’re seeing. And I think we all went through this with coding. Or maybe this is not true.

Something to that… I think it was – who’s the guy who does This American Life, the producer of This American Life, and the voice? I can’t think of his name right now. I’m blanking on his name. Both of you are blinking as well.

I think I know what you’re about to say, though…

Anyways, he talks about the talent and style gap. I’m also getting that wrong.

Yes.

But it’s basically like this gap between your taste… Ira Glass.

Ira Glass.

Ira Glass.

Okay, we’re all onto this now. I knew we’d get there eventually… Yeah, in his premise, which I think all of us have agreed with, because we’re all like “Yes, we know about this”, is that when you start a thing, your taste is in one place. You know what’s good, or what you like, but your skill is so far. The gap between the skill to bring that taste out, and the taste is so far away that you just despise whatever it is that you’re doing. But like you said, Amelia, it’s just the hours in between; it’s the toil that brings your skills up to a point where they can come to the point of maybe being close to your actual taste. We all tend to still discount, I think, our own creations… But overcoming that gap is just very difficult. And so yeah, I’m a guy who doesn’t draw. I never really put the time in. And I’m okay with that. We all have to have certain things that we do… And it wasn’t something that I really had much joy out of as a young person. But I just consider myself a guy who can’t draw. And I know that if I put in the requisite 10,000 hours that Malcolm Gladwell says everybody has to put in to be an expert, I could be a drawer. I could be a person who draws. I just haven’t crossed the gap, and I just don’t – we only have so many things that we can pick to like “I’m gonna put the time in”, because there’s only so much time. But I feel like I’ve written a lot in my life, and I still despise it. [laughter]

[31:54] My opinion is that - and this is completely unfounded, but I think that writing is just harder than other tasks. Because I think good writing is really good thinking, and there’s no way to like cheat that, right? There’s no shortcut to it, really. You can use flowery language to make something sound nice and pretty. But when it comes down to it, you have to have thought through your chain of thought… And to me, that’s the painful part, is like “Oh, crap, I actually have to think through what I’m saying.”

That’s so interesting, because you can say the same thing about coding, but you don’t publish your code. So you could have really nasty-looking code, that does the correct thing, but it’s not good code.

So that’s sort of true, and I agree with it. But I was looking at Alex’s code, and it was just a bundled version, of course.. And so he has to ship his code to a certain extent, but it’s all minified and bundled. And so I use the tools to at least pretty-print it, but all the variable names are changed, and it’s not the original thing that Alex wrote.

No, that actually is the original.

That is the thing you wrote.

I’m just kidding, I’m just kidding.

[laughs]

I was like, “Dang, dude. You are good!”

Next-level.

All these one-letter variable names, I’m not sure how you do it. And so you can kind of hide 00 I mean, in binary world this is 100% true. Like, I compile my code, I ship a binary, right? In the Web world, it has been, and in many cases it still is that you do ship your code into the browser, and it’s at least observable by the end user, to an extent. But a lot of us do ship our code in the form of open source. And I was actually looking at your code, Alex, and thinking it would have been cool if there was like a raw version of this, that’s like “Here’s what I actually wrote”, because maybe I don’t want to learn how to draw lines in WebGL… And 12 kilobytes is pretty small; maybe I could be okay with maybe ripping a part of your thing out and using it for my own thing. Did you consider putting the code out there? Or maybe it exists somewhere.

Yeah, this is funny, because this is exactly what my partner Shelby asked me, like “Why are you going to minify your code? That’s not in the ethos of the HTML review. It’s all about like View Source, and sharing knowledge.” And I do agree with that. I think in this case I’ve put so much work into these personal little libraries that I’ve created that it’s almost as much a part of my artistic style as my aesthetic sensibility. This technical, aspect of it. And so I just made the decision that, at least for this piece, I’m not ready to publish it open source.

I do have open source code on my GitHub… So one of the things that I’m really interested in in all of my projects are creating these simulated ecosystems, these LoFi plants and creatures that swim around… And that’s something that I used a little bit for the Grid World essay, but all of that, all of those algorithms are open source on my GitHub. Now, they’re completely unusable because of how crappy the code is… Because when I’m writing code for art, it’s like my brain switches into this other mode, where I’m just like no comments, no organization; just vomiting it out, right?

Means to an end.

Yeah. But if you want to try to pick it apart, it’s there. And then maybe in the future I would consider cleaning up parts of my code and publishing it. It’s just that’s almost like a whole other project, to clean it up to the point where somebody could use it, and understand it, and learn about it without getting frustrated.

I had actually seen these art pieces before around the web. And one, I’m obsessed with them. They’re super-cool. They have like collision detection, and they feel alive in a really interesting way, because they’re very - pixelated, I guess is the word. Like, they’re low-res intentionally. I’m curious if you have any specific inspirations? Because I feel like they’re pretty unique. I haven’t seen anything like them.

[36:03] Thank you so much. Yeah, the low-res thing is interesting, because I kind of fell into that. So the story is that I was commissioned to create a permanent installation here in Seattle, where I’m based, as part of like an apartment building that was being constructed. And so this was a huge break for me, because the art thing is mostly a side project. So to be commissioned to do “a real art installation” was a big deal. So I worked a lot on this proposal for them, where I proposed to create this virtual ecosystem on these big, bright screens on the side of this apartment building. And the developer was really excited, and signed the contract.

And then when I started working on it, I ran into a ton of technical hurdles, because I hadn’t ever really done anything like that before. And so when I actually hooked up the computer for the installation, and ran my code to simulate this virtual ecosystem, I found that it just wasn’t performing very well… Because the virtual ecosystem is based on a particle simulation where each particle has to compute its distance to all of its neighboring particles to implement collision detection, so that the particles can have forces applied, so that they don’t overlap. And there’s a lot of optimizations you can do, but when it comes down to it, this is like an N-squared thing. You’re comparing every particle against every other particle. And so I just couldn’t fill the space with enough particles to fill up the screen, because the algorithm was too slow. So I was like “What do I do? What do I do?” And then I was like “Well, if I just have the size of the simulation, and use 50% of the particles, and then scale the graphics up to fill the screen, then it can run well.” And that’s where I hit upon that low-res look, because when I was scaling up the graphics, I decided to try “Hey, let’s not do any smooth interpolation when I scale up the graphics. Let’s just do kind of like hard, pixelated interpolation.” And it looked really, really cool. And so then that kind of became a style that I’ve used throughout all of my work… But really, I fell into it because of technical limitations and constraints. That’s like the thing about constraints leading to creativity.

They’re good friends.

Yeah.

Yeah. I love that story.

I will say though that as I’ve done more pieces using that low-res look, I’ve become attached to it, because I really like how it focuses your eye on kind of like the underlying behavior, the underlying algorithm. It’s like the code or the algorithm is the subject of the piece a little bit more, rather than some sort of photorealistic effect. So it’s kind of like making surface of the piece more transparent, and you can see through it into the computation that’s happening. And that’s the thing that I want to emphasize, rather than trying to render a scene realistically, or like use the latest and greatest special effects, if that makes sense.

It does. It’s like there’s less flashy stuff to look at. So the less things there are, the more you’re looking at the thing under the thing, right?

Exactly. Yup. Exactly.

Do you think in pieces or in installations - and I’m asking that from the perspective of a developer, because we tend to… I don’t know, maybe like create toolkits, or create libraries, or create things that persist, or some of us work on the same thing for a really long time… I mean, at this point in my life, I have one software project that’s like the main thing I do, Changelog.com, which powers everything that we do here at Changelog, and I’ve been coding on that thing for eight years.

[40:04] I have other things that I work on, I have little side projects, and things like that, but that’s like a long-term thing where when you make a choice, you’re like “I’m gonna have to live with this for years.” I think some of the freedom that I would think comes with artistic pieces like the ones that you create is kind of the ability to start fresh every time, or to let the past be the past… And I know that in the world of like data journalism there’s a lot of that going on, where it’s like “I create a piece for this report that we’re doing.” Rich Harris talked about this when he was creating Svelte, and the work he was doing at the New York Times. They had these one-offs – each thing was a new project, and so it’s a new opportunity to start fresh and try new things, or build something that you reuse. How do you think about it, Alex? Do you kind of start fresh each time? The reason I’m asking is you have kind of this one aspect that has to be – it’s kind of a theme now, that’s coming through multiple pieces. But how do you look at it?

It’s hard for me to turn my engineering brain off totally when I’m working on art… And so obviously, I’m thinking about costs, time costs, and if I’m going to start fresh, then that means I have to build up a whole new set of like primitives to create the art with… And so I’ve found that just practically speaking, I ended up reusing code. Now, when I’m in the professional programmer mode, I’m much more clean about that reuse; I’m packaging things up in nice ways, I have a nice, clean Git history, and stuff like that… And then when I’m in kind of more creative coding mode, I’m way more messy, I have repositories that are just like 100 branches, and there’s – oh my gosh, it’s like a nightmare. I’m filled with dread whenever I have to like go back and remember what I was working on before… But there definitely is a lot of that reuse, and I’m always surprised at like how practical that can be in creative pursuits, because if you go back to material that you’ve created before, but you have like a new perspective on it… Sometimes I’ll use the code that I created two years ago, but I’ll have this new idea on how to remix it. You can get these really fresh, new things that emerge from that. Now, I say that, but I also have reimplemented the same styles across different technologies, and that’s kind of like the opposite mode, where I’m literally writing something from scratch to achieve the same look, or the same behavior that I had on another platform. So it kind of goes both ways.

That resonates a lot with me. Jerod, you brought up data journalism, and when I was at The Pudding, they’d do like these data-driven articles as a team, right? There’s eight people. And it’s always tricky finding the right balance of what primitives you reuse across projects, but you can’t have too strict of a template, because it makes you less creative. You feel kind of constrained to this framework. So I like the way you talk about it, Alex, where it’s figuring out what are the right pieces that you want to preserve, so you’re saving time, but you’re not constraining yourself too much.

Right. It’s like the balance of constraint, right? Because on one side, like this piece with Grid World, and your knowledge of WebGL, and your abilities or lack thereof, and how much time you want to put into it really helped produce the art. But sometimes you want the art to drive the technology, right? Or you want the story in the case of reporting. You don’t want to be tied to a framework, if the story is calling for something else. You don’t want to be like “Well, but we use this, and here it is. I used it last time, I’m gonna use it this time, and I’m gonna use it the next time.” But that is a constraint as well. So when is the constraint exactly what you need to produce the better thing, and when is it actually holding you back? I don’t know.

Yeah, it’s so fascinating. I think about that all the time. I think the data journalism side of it is probably way tougher to deal with, because that truly is like a creative pursuit. Like, you’re trying to reimagine how to visualize this data, and you’re trying to be novel, and interesting… But then you also have all of these – the constraints are very stringent, right? You have to present things correctly, and it has to work on all browsers, and it has to be accessible, and all this stuff. So I have a lot of respect for that sort of practice. Yeah, that’s pretty cool.

One thing I’m very curious about is for these projects, like when you think about data visualization, or generative art, it ends up being this kind of symbiotic relationship… And a lot of the time you can’t foresee what the outcome is because you don’t have complete control over what it’s going to look like. I’m curious, what’s your process like, and how true – or how strong of an initial vision do you usually have when you set out on these projects?

I have found that the way I work best is to not have too much of a vision at the beginning. So I’d characterize two approaches as being like top-down and bottom-up, where the top-down approach is you have this large, grand story that you’re trying to tell, you come up with what you want it to look, or what you want the experience to be, and then you follow up and say “Okay, what are the technical ways that I can implement this?” I think that this is how really great filmmakers work, for example.

I’ve found that for myself that is just not the way that I enjoy working, and I think it took me a while to give myself permission, to be like “Oh, I just like the bottom-up approach.” I like to tinker, I like to start really small, and build up, and I like the technology – I like to kind of like go with the grain of the technology, and let that lead me a little bit. And maybe a similar way to a painter really letting the paint kind of guide them, right? Like, having a conversation with the material… That’s just kind of the approach that I like to take. It’s not always practical, because sometimes if you’re commissioned to do something - and I’ve had this experience before, where you have somebody coming to you and saying “Hey, I have a vision. Can you bring your art to this vision?” And that’s a real struggle for me. I’ve had a hard time doing that in the past. It can be good, though. I think both modes are good, and it’s good to try both and switch back and forth… But I’m definitely a fan of like the bottom-up tinker approach.

What is like the seed usually of your bottom-up ideas? Is it like “Oh, I wonder how this interaction would feel”, and then you build it, and then you riff on it? Or is it really just playing around?

[48:00] It’s usually an idea about like a very low-level behavior of one of my algorithmic plants or animals… So again, these are all just particle systems, where you have all of these interactions between particles, and so sometimes I’ll just kind of be daydreaming about “Oh, what if you had a structure that was like a ring of particles, and it could grow by adding particles into the ring? What would that look like?” And then I program it, I see what it looks like… It usually ends up nothing like how I imagined it, but then you hit on something where you’re like “Oh, but this is kind of cool.” And then it kind of just goes from there.

So I think when you’re dealing with these dynamic systems, that have lots of feedback loops in them, there’s kind of a space of all of the possible dynamic systems you can imagine, and there’s little pockets of coolness in that space, and you’re trying to tweak the parameters to guide your ship to find those little pockets. And I think what I have found is that those pockets are more common than you think. So if you just keep tweaking, and keep adding, and keep exploring, you will hit one of those cool pockets. And it’s not like – you can kind of get there through random chance a little bit. And I’ve actually thought, maybe an AI could do this, because it’s not actually super-directed or anything. I’m really just kind of like randomly playing around with these ideas, and then “Oh, hey, this looks cool.”

I literally had a whole conversation in my head about “Oh, I really want to ask if you think AI can help with this”, but I don’t want to be the one to bring up AI in this episode, so…

Well, Alex - yeah, he broke it out. Well, do you mean generate the things and you look at them as the artist? Or do you mean actually – because the “Oh, hey, this looks cool”, that moment right there - I don’t think AI is there. I think that’s us. And so there’s a symbiosis there. But are you referring to Alex out of the loop, or are you referring to Alex in the loop?

Yeah, definitely if the artist is in the loop, I think it’s way more possible. And that’s something that like – that’s not even a super-sophisticated idea, to generate the same piece of artwork with your parameters tweaked a bunch of different ways. I actually think that – I’m not an expert in machine learning, but I think there’s this term “grid search”, where you tune your hyperparameters, or whatever, a bunch of different ways, and then see what the results are. And I do stuff like that all the time. I don’t think that that’s really AI… But I think that if you were to have an AI that could like model your aesthetic preferences, and there’s Alex out of the loop - that would be definitely more interesting, and maybe more scary.

Or even to knock you out of a rut. You’re like “Oh, just come up with these oddball inspirations from things that I wouldn’t necessarily think of”, and you could look at one and be like “Oh, huh, that’s cool.”

Totally. Yeah, yeah. I do think that for me - and I’m speaking as somebody whose livelihood does not depend on my art, because it is like a side project, so it’s definitely coming from a different perspective… But for me, even if AI could do what I can do, I wouldn’t use the AI to do it necessarily, because I just enjoy the process so much. Creating art for me is about learning and understanding, it’s not really necessarily even about the end goal. It’s about that tinkering process. I just find that enjoyable and fun. And so if an AI could do that part, it would be not as fun for me. But I do think that there’s like really interesting applications of AI for creatives, and so I think that lots of people will start using these tools in interesting ways. It’s just from this personal art practice I can’t see myself really using it that much.

Well, Alex, there are many things that we could talk to you about, but in the spirit of this episode, we’ll keep it constrained. We’ll just focus it in on Grid World. Of course, listener, there are many more pieces of art. He even has a shop where each design is generated at the moment you order it, algorithmically-generated goods… So definitely check the links. You can find Alex’s work at alex.miller.garden. And of course, we will have links for you in the show notes. Alex, I’m serious about doing a show on the HTML review. Let’s hook that up. Amelia, of course, you’re welcome back. I just love cool, indie, interesting web things. I feel like the web has gotten somewhat boring, so corporate - we’re starting to see a backlash against that; like, more small, independent, creative, web-based things. I love the idea of a web-native - not newspaper, but I guess magazine. I don’t know how it’s –

Yeah, I think magazine is a good word for it.

Yeah we’ll ask her about the details on the HTML review on a future episode… But we really appreciate you coming on the show. This has been a lot of fun. Alex, any final thoughts or final words before we call it?

No. Thank you so much for having me. Like I said, I’ve got major impostor syndrome being on this show and not being a super-great web dev myself… So it’s humbling to be on here with you guys. But yeah, thanks so much.

Well, you are welcome here. Everybody is welcome here. There are no imposters. That’s one of our mottos here at Changelog. We’re all welcome. So this has been lots of fun. Amelia, it’s always a pleasure. Oh, by the way, real quick. You wrote a thing, Amelia… You published this week about AI chatbots, right?

Oh yeah, that was surprisingly controversial… [laughs]

Oh… Give us 30 seconds. Give us the synopsis, and then the controversy.

It all started with Chet GPT… [laughter]

It sounds like a story…

I’m still shocked that it blew up the way it did, and so I have a lot of thoughts about why chatbots aren’t it, and we’re really leaning on them pretty heavily recently for AI interfaces, so…

Okay. And we’ll leave it at that. So there’s a teaser for you. I’ll link to that one in the show notes as well if you want to read Amelia’s words, and then you’ll probably have strong opinions about them. It seems like everybody else does… Which is good. Conversation is a good thing. Alright, let’s end it right here… I’m Jerod, this is JS Party, and we’ll see you on the next one.