Visual programming with hardware and Node-RED

Thank you for joining us for another episode of JS Party. This is the podcast where we celebrate all things JavaScript. This week I’m joined by excellent regular panelist Chris. How’s it going, Chris?

Arrrrr!

[laughs] We also have a very special guest this week that Chris and I are already familiar with at work, so we’re pretty excited to dig in about that today… We have Nick O’Leary with us. How are you doing today, Nick?

Hey! I’m very well, thanks.

Excellent. Now, Nick is involved and is the author of an excellent project called Node-RED. This is special to my heart, specifically because of IoT, and it relates to hardware, and I actually really love the idea of visual programming, as well… And I know that Chris has also used Node-RED in the past as well, so we’re pretty excited to dig into the details. But for those who are listening who might not know what Node-RED is, do you want to explain what is, Nick?

Sure. Node-RED (the strapline) is “low-code” programming for event-driven applications. I said that to someone today and they said “What does any of that actually mean?”, which is a fair question. So it’s all about programming without writing code. Using visual programming, flow-based programming in your web browser to draw logically the flow of events in an application, so you can very quickly express how you want events to be handled, what should happen when an HTTP request comes in, or a sensor reading from a device, whatever it might be. An event… You can then, in your browser, draw the sequence of steps that should be taken in response to that event. So it hides away all of that boilerplate code of “How do you handle an HTTP request? How do you access the device?” It abstracts all that away, so as a developer you’re just drawing your browser that logical flow of events.

That is super-cool. And what’s your background in IoT in? And does this relate to your day job at all? What was the back-story behind how you got into this stuff in the first place?

Going back a ways, in a previous role at this company, I worked on MQTT, the now fairly ubiquitous protocol for IoT. And this is probably about ten years ago, so before IoT was a term, before anyone outside of IBM knew about MQTT.

[04:09] I was working on a very small footprint implementation of MQTT and its clients, so doing a lot around - and this was in the Java world - what we then called “pervasive messaging.” At the time, I thought one of the ugliest pits of the API our broker had was how you define when a message arrives on one topic, what allowed you to define some code that would get run on that message to republish it on another topic. We had the Java API for that, and it was horrible.

At the time, I dreamt of “Wouldn’t it be cool to have some way to quickly and easily visually describe that in a UI?” At the time, I had a quick go trying to see how you could visualize that in the browser. I got so far in terms of just how you can draw in the browser ten years ago, so it probably only worked in one browser type, and barely even then… But the idea faded and I sort of stopped playing with that idea.

Fast-forward a few years and I was in our IBM’s emerging technologies group, where we do lots of client projects, proof of concepts, first-of-a-kinds, projects where you meet with the client one day and you have two weeks to work with them to deliver something to show some sort of result, particularly around emerging technologies; areas that mainstream bits of the business don’t work. This isn’t just taking IBM products off the shelf.

And we were doing a lot of what was then more IoT - devices, plugging in random sensors, doing random sorts of integrations between different systems… And after the N-th time of writing some code to open a serial port to get GPS data or whatever it might have been, it sort of rekindled this idea I’d had earlier about this visual way to map MQTT topics… And I sort of thought – I guess it must have been six or so years since I’d last tried, so “Let’s have another go. What can you do in the browser?” I stumbled across D3, which just made it trivial to do nice flow-like diagrams in the browser, and in an afternoon I knocked up this workspace you could drag something on and draw a Y between them. And I showed a colleague, saying “Now, just imagine if there was a button that when you clicked that button, that was then running, and it was actually doing something.” He looked to me completely deadpan and just said “Well, go on then.”

So - back to my office, and the next afternoon we had the very first version of what would become Node-RED; a tool in the browser that I could drag these boxes on and draw a mapping between two different MQTT topics, hit Deploy, and the Node.js runtime would subscribe and publish as needed.

Then a couple of days later my colleague needed to plug a serial port in, so he wrote a serial node. Well, in fact, he asked me to write it, and I put him off for 24 hours whilst I completely rewrote the whole code to allow it to be more pluggable, so that other people could write nodes… But a couple days later we had the GPS node, a few days after that we had a TCP node… And because the nature of the projects we were doing, we kept stumbling across “Well, wouldn’t it be great if we had a node that did X, or a node that did Y?” and that is the first snowflakes-turned-into-a-snowball-turned-into-an-avalanche of just “Here’s a tool that we’re developing because it’s helping us deliver real client projects. It isn’t just some side project toy.” That was a great place, in those early days, to validate what we were doing. It was useful.

Can you describe what a node is? People are gonna be like “Node… Node.js?”

Yeah. We shamelessly overload a bunch of terminology… And again, being a visual thing, it’s one of those things that I’m far more used talking about being able to point at a screen, and showing it, and letting a picture speak for itself.

[08:06] So in Node-RED you have these nodes. A node is some sort of functionality, and it’s a well-defined piece of functionality. You might have, as I mentioned, a node that represents reading from a serial port, or a node that lets you set properties on a message; a node that lets you talk to Twitter and send tweets… Whatever it might be. So each node is self-contained, it’s well-defined, and the key thing is nodes don’t know what they’re wired to. They are just given some data, or they listen for an event, they do some work, and then they send out a message in response. That message, just be, is a plain JavaScript object, which can have whatever key-value pairs you want on it. But by convention - and this kind of reveals some of MQTT heritage - messages have a payload property… And we like to say the payload property is where the interesting information goes.

So in the case of a serial port, a data we read from the serial port will be in the payload property. But there might be other properties, depending on what the node is doing. For example, the Twitter node that listens to tweets - we put the text of the tweet in the payload, but then we set a property called “tweet” to the full however many kilobytes of metadata that come along with every tweet these days. And it’s that flexibility, again, which speaks to how you can easily extend what you do with Node-RED. You’re not constrained to only being able to set certain properties. This convention of using the payload does mean nodes can be written to by-and-large just work when you start wiring them together, because they know to use the payload.

I find that really interesting, that you started out with MQTT, pretty low to the metal, actually implementing the spec. I’ve actually used your Arduino pubsub library a lot, and thank you, you saved my butt in a lot of customer meetings, just because they didn’t wanna use a full SDK for the actual IoT infrastructure we were using… So your library was much smaller and was much more lightweight. It’s interesting when you talk to a lot of people who have been developing on the metal like that. Usually, what ends up emerging every time is you get tired of that and you just wanna create something that is this really lovely, easy-to-use interface that you wish you had. Is that why you chose to do it in the browser and chose to use JavaScript to do it? What took you from the low metal all the way up, from that perspective?

I think, going back to those days of doing the MQTT stuff, the day job was predominantly Java and doing Java implementations, and at that time the browser wasn’t really a space for creating the UIs for products, like it is much more predominantly now. It was Java Swing, and all those sorts of toolkits, and all of that was just a horrible experience for developing; it was just slow and cumbersome. So yeah, it was recognizing that the browser is – and you know, at the time, JavaScript was just becoming more ubiquitous for creating rich UIs, and why constrain yourself to a particular language toolkit? Do it in the browser, and then if you want it to run on a Raspberry Pi that you haven’t got a display on, or things like that… It’s not a desktop app, it’s something you just point your web browser at remotely.

Again, one of the things we found very early on was about two months into the project we ended up doing a project in an ice-cream factory, where we had to retro-fit a whole bunch of sensors to this particular line, so that they could gather data and do some interesting data on Linux… But we were there just to help them gather the data from the sensors. And that was three Raspberry Pi’s, Arduinos, things wired in, and Node-RED on each Raspberry Pi to coordinate and synchronize gathering data.

[12:08] And this place was like 200 miles away from the office, so it’s not somewhere we can just pop in… And they phoned up saying they’d had to replace one of the sensors on the Arduinos, so the 0-to-5 volts reading now mapped to a different pressure range; when could we come back to rebuild the code to get sensible data.

Whilst I’m on the phone with them I logged into the VPN, fired up the web browser, and in Node-RED I could just change the one number to represent the new mapping. I hit deploy before he finished asking when would we be able to come back on-site to fix the code.

Again, that ability to use the browser so you can access it remotely, securely, all that good stuff does make for a nice developer experience.

I really like that magical story. You also mentioned event-driven before too, and I think that anyone who’s used hardware with JavaScript knows that that event-driven model actually works extremely well with things like payload sending, and reacting to events, and interrupts, and things like that.

Yeah, absolutely. That’s the bit I’ve not mentioned - when I sat down to tackle this problem for how to do it, how to create this sort of UI tooling, this was then the first serious thing I ever wrote in Node.js. Part of the day job was keeping up with new technologies, so that when a client came in and asked for something rapidly, we had the experience. At that point in time I’d been meaning to do something with Node for a while, but I hadn’t really, so this was my excuse to start kicking the tires with Node.js and learning about Node.js. But again, it was a fortuitous choice, because as you say, the event-driven nature – in hindsight, of course, it was the sensible choice… But it was just good luck at the time that that’s the direction we went and we’ve not looked back.

I didn’t wanna take you out on too much of a tangent, but you brought up this thing where the ice-cream factory has all these Raspberry Pi’s, and on all these Raspberry Pi’s there’s an instance of Node-RED running. I use Node-RED, I’ve been using it for years, and I would always have a Node-RED server, and all of my things would talk to that… So I don’t quite understand, why would you want to have all these different instances of Node-RED running all over the place and talking to each other?

Well, this was two months into the life of Node-RED, so I wouldn’t claim necessarily we followed every best practice we’ve learned along the way… But this was at the time the approach that made sense, because we had an Arduino attached to each Pi, doing more real-time data capture… So we had to capture peaks and troughs of pressure sensors that would last less than a second. The Arduino was busy spotting those peaks and troughs, and when it spots them, it just writes over serial a couple of numbers.

We had Node-RED on each Pi, just because it made it easy then for each of those Arduinos to just write this data over serial, and not us having to worry keeping those Arduino on a network, and all that overhead. And then one of those Pi’s was in charge, and at that point in time we needed to capture data – it did a UDP broadcast, which we had flows on all the other Pi’s in Node-RED listening for… And when it received that UDP broadcast, that triggered each of the Pi’s, for the Node-RED flows to capture their data at that point in time.

So I think we didn’t exactly know how many different sensors we were strapping onto this line, we didn’t know how many Pi’s would – you know, we went with a bag-full; it was the expedient way to do it at the time.

Right, yeah. I think it makes sense then. Every Pi has an Arduino, and the Arduino is not on the network itself; well, something’s gotta be on the network. Node-RED already has this serial node, so you just pull the data out and send it with Node-RED. That makes sense to me.

[16:09] I guess that’s probably because now I tend not to build things that don’t have Wi-Fi in them, or some sort of network connectivity… So it’s like “Oh, well…” I guess before we had an ESP8266 you would need to plug your Arduino into something else to get that over the network.

Yeah. And in this case, when we turned up, they showed us the big steel box that all of our kit would be living inside… Because at the end of every day they get the pressure hose and the whole thing gets washed, because it’s food production. This big steel box was not conducive to Wi-Fi, so we had literally one Ethernet cable coming into this box, so… Interesting constraints of real-world environments.

Yeah. I really like this pessimistic viewpoint - if you think about it, you have almost pretty much halved the amount of devices that have to remain on the network, which effectively halves your headaches, because most of the issues with IoT is actually the I part of IoT. [laughs]

Yeah, absolutely. For us, the key thing here was the data capture. And in fact, if memory serves, each of those Node-REDs just wrote to a file, and those files then got uploaded by a cron job regularly. It was a nice example of using Node-RED as just one small part of a bigger solution. Those files then went up to the bigger analytics platform where they could do all the heavy-lifting using the more appropriate tools.

And one thing I think gets overlooked with Node-RED is - yes, you built it for IoT, but you don’t have to have an IoT application to wanna use it. If you know services like IFTTT - it’s just like “We’ll set up a thing that listens for this, and then it goes and triggers something else.” You can run your own service that does that with Node-RED, because you can set up Node-RED “Okay, let’s ping this RSS feed and see what changed, and if that changes, then go and fire off an email, or something.” There’s a lot you can do, and it’s not just IoT.

Absolutely. And we are quite consciously moving away from talking overtly in the context of IoT. That’s where we’ve come from and that’s always a strong use case for it. I said at the start Node-RED is “low-code” programming for event-driven applications. We used to call it a visual tool for wiring the internet of things. Anyway, you’re right - it’s anything event-driven, whether it’s REST APIs, integrations…

I’ve got a whole ton of Node-RED flows running on the internet, handling a whole bunch of GitHub webhooks, for example. It just makes it really quick and easy. I’ve got a whole bunch of Alexa Skills at home that are backed by Node-RED running in the cloud, just because it makes it so quick and easy just to spin something up to handle the request, and again, not have to worry about all that boilerplate code you need to do it.

Alright, so what company is there out there using Node-RED right now, and what are they using it for?

Great question, Chris. We’ve got quite a healthy collection - both for an open source project - of other companies who are making use of Node-RED. To slightly avoid that question initially, this was one of the big things we decided with Node-RED, to develop as an open source project, and we got there really quickly. It was about six months into the life of it we decided to open source, because we saw the real value was building a bigger community around it. And that has certainly borne out.

Today, other than ourselves, Hitachi, probably the main other contributor - and they are very heavily using Node-RED as part of their own IoT platform… Great examples out there; you can go find about the sorts of things they use it for. It is quite central. So they’ve become a great contributor to the project. And then there’s a long list of – well, it’s interesting to see how it breaks down. There’s a load of companies who create hardware - industrial controllers, or industrial gateway-type machines; you’ve got people like MultiTech, Opto22, there’s a company in the U.K. called iaconnects… They all create hardware for interfacing, whether it’s with building management systems, or industrial automation… They create gateway devices, and they have built Node-RED as part of the software stack on those devices.

For them, it’s all about – their users are not necessarily developers, but they understand the environment they’re running this hardware in. They’re system engineers, or they own a building. So Node-RED and its higher level of abstraction makes it easier for them to define the behavior of this hardware.

Nick, real quick - are those particular usages a replacement for a PLC, or is it something in addition to that?

Can you just explain what a PLC is for those back home?

I cannot. [laughter] Maybe Nick can.

A PLC (Programmable Logic Controller) is one of those abbreviations that you take for granted, and when you someone puts you on the spot. Programmable Logic Controller - it’s the bit of hardware that will talk to the actual machines, and it’s where you can program their behavior… And those are often fairly proprietary devices from the different manufacturers. These different industrial gateways - some are complementary to PLCs, some will sit alongside and they will have on them the APIs to talk to PLCs, but then try and open them up. Your Node-RED flows on those devices, will be able to talk to the PLC to get the raw data, but then give you much more freedom as to what you then do with the data because of the palette of nodes.

And then the building management systems, those sorts of things… Again, they’re all sort of designed to sit alongside whatever the standard hardware is out there. I think all these companies are trying to use these open tools to try and create a more open ecosystem for these sorts of environments, which have traditionally been very stovepipe. If you buy one manufacturer, you have to buy that manufacturer for everything, because that’s the only option.

So you’ve got device people, and then you’ve got people using it in the cloud. I mentioned Hitachi, I know Siemens have got an offering around Node-RED, NetApp have got a modified version of Node-RED for one of their products… Particle.io, a great IoT company, creating really cool devices; they’ve got their current beta of their Particle Rules Engine, so you can actually define rules for your IoT data using Node-RED… And then there’s a long list of companies, big and small, who are playing around with Node-RED. Some already have commercial offerings based around it. Lots of people are using it internally, and rather than creating products on the back of it, just using it for what it’s meant to be used for internally, that type of stuff.

[24:11] So it’s quite rich and varied, and I think it does reflect the fact – traditionally, it has come from an IoT background, so lots of those examples are IoT-based, but we know there’s some interesting companies doing… I mean, there’s one in particular who does chat services for massively multiplayer online games, and they have built a system using Node-RED that takes a real-time stream of all of the chat messages going on in a game, and the service they provide is it allows linguists to define rules to identify abuse, and people griefing other players… And it allows the game publisher to choose how do we respond in-game to when you see one player being abusive to another.

A great example where you need language experts, because it has to handle – it’s not just English, of course; it has to handle every human language. So you have the linguists who are experts in understanding those rules and patterns of behavior - Node-RED allows them to define that logical flow of how they can detect the behavior where they don’t have to write code to do it. That’s a great example, completely away from the IoT domain. And yeah, a whole host – some people use it for integrations of systems… Lots of different things.

One of the interesting challenges is – and this is one I found here, this was my little side project… And every now and then we just stumble across someone new who’s using it. And it’s not that they’ve come to us for help to use it, or maybe they have on the forum, but using a personal email address, so we have no idea… And we just stumble across someone else doing something really cool with Node-RED that we had no idea about. I think that’s one of our interesting challenges as an open source project - we every now and then just put this appeal out to the world, saying “Look, if you’re doing something cool with Node-RED, let us know”, because we would love to be able to talk about it; we have to point to these great examples of how it’s being used.

Yeah, that’s a common complaint with a lot of popular open source projects. You don’t know who’s actually using your tool, unless you do something like build telemetry into it, and then people have a hard time with that… So you might not know who’s actually using Node-RED. There’s probably many more companies that are using it that you don’t know about, because they haven’t reached out to you or made it public.

Yeah. And you know, I like to take that as a sign of personal success, that we’ve managed to actually create something that these people have been able to do what they’ve done without having to come knock on my door after five minutes and say “How do we do anything?” The downside is that it’s hard work to find out about these guys… But on the flipside, I think it sort of reflects what we’ve managed to achieve within the project, that these companies have been able to do that with confidence on their own.

One way that I get around this with open source hardware and the libraries that I maintain - if anyone opens an issue or a request, I will immediately ask them “By the way, would you mind sharing what you’re doing with it?” Because that’s really the only way that you can find out. But in your case, Nick, you actually have something that might give you a few clues, which is when you go on npm right now, if you just put in the term Node-RED, you get a ton of results. I’m talking nearly 3,000 packages that at least referenced Node-RED. At a guesstimate, how many of those are community-contributed, compared to ones that you’ve actually authored yourself? And does that give you clues about how it’s being used?

[27:43] I can tell you as of right this moment there are 2,216 npm modules that contribute extra nodes into the Node-RED palette. So 2,216 - I forget how many of those are ours, but I think about 100 of those might be ones that we in the Node-RED have published at one time or another. So there’s well over 2,000 truly third-party modules out there for Node-RED. And I think, again, in terms of how you go about building a community on a tool like Node-RED, the real strength was always about not gatekeeping who could extend its functionality.

Each of these nodes – it is an npm module, and as long as the module has got the Node-RED keyword, then our flow library will pick it up; it regularly scans npm, looking for modules with the Node-RED keyword. We do a bit of work to examine it and say “Does this actually contain a node, or is someone just using the keyword for the fun of it?” And if it contains a node, it automatically gets listed in our flow library.

Aside from a couple quality checks - that it’s got a readme, a couple of things like that - there is no gatekeeping. So literally anyone - as long as it meets a basic set of criteria - will get listed in the flow library, and then within Node-RED itself we have the Palette Manager where you can go and install extra nodes from within Node-RED itself. So that shows that there’s this huge appetite for extending it.

The other fact - we’ve got just the raw npm download stats, and in fact we hit two million installs of Node-RED a couple weeks ago, which we’re sort of saving making a big noise about, because we’ve got some stuff coming up and we’re gonna roll that as one of the highlights of some news we’ve got coming up.

I do keep a track of the npm stats, just out of interest. We know that each npm install is not a brand new user, of course. People reinstalling, or restaging, or whatever it might be… But over time, it’s shown there’s a nice, steady growth of people coming to Node-RED, of installing it. I wish with the npm stats we could just find out a bit more, but you know, that’s the fun of npm.

That’s an incredibly large ecosystem. It sounds like it’s probably pretty straightforward in order to create a node or an add-on for Node-RED… So what is the process there? It seems like you look for quality checks, but do you just create a main JavaScript file in a root directory? What other integration features do you have for wanting to get started with that?

So a node in the Node-RED palette consists of two things - it has a JavaScript file, which is a Node module, as I’m sure everyone will be familiar with… With a very simple, boilerplate structure of creating an object that is in Node and registering it with Node-RED runtime; you have to export a function with the right signature… So that’s the JavaScript file, and that determines the runtime behavior of the node.

Then there’s an HTML file which defines the edit dialogue of the node in the editor, help text, and its editable properties, and a bunch of metadata that the editor needs. But they get packaged up in an npm module, and again, a bit of metadata goes in the package.json file to help Node-RED. When Node-RED loads the module, that metadata tells it which JavaScript files it needs to go and load dynamically, because they contain nodes.

So Node-RED will load the modules; as long as they load okay, when you open up the editor, the HTML file gets sent up to the editor to register it in the editor side. So it’s really only those two things - the JavaScript file, the runtime behavior, the HTML for the editor, and the package.json metadata to pull all those together. And a single npm module can contain multiple nodes, can contain multiple of those files, as long as they’re all listed in the package.json file.

[31:54] So the node object that you have to implement is quite a simple thing. The node registers a listener on the input event, which will get triggered whenever the node is handed a message. It can do whatever it wants, and at some point it will call the Send function on its prototype, to send the message on to whoever it might be wired to. That’s it at its most simple. It can get more complicated, but at its heart, that’s all it is - you just register an event to handle messages coming in, and you either send a message on or you don’t.

There’s two sides then to every node - there’s the runtime, and this is what the node’s actual behavior is. So if that is a node that wants to tweet, or something like that, that is a node module, and it runs in Node.js, and it reaches out to Twitter.com, and listens for events, and sends events on. And then there’s this portion that runs in the browser; that’s the front-end. So when you load up Node-RED, if you install it and then you go to the server in your browser and you see this big layout – and I think it might not be obvious to people who haven’t used it before… Could you explain the difference between the runtime and staging? When you make changes in the browser and draw things out, what happens when you click that button that says “Okay, stage this. Publish this”?

Sure. When you’re drawing your flows in the editor, the editor is building up a model of what you’re creating. And essentially, it is a JSON array. When you hit that Deploy button, the editor exports your whole configuration as JSON, and it is just a long array of all of the nodes you have, with their key-value properties, with some other metadata, including the list of who’s wired to what. That JSON is sent back to the runtime, and then the runtime just basically breaks that down, iterates through all the nodes in the array, and starts creating the corresponding objects in the runtime, and then passing each one the configuration. So when you create a node, when you write your code for a node, the constructor function you create takes one argument, which is your properties that have been defined in the editor. So the runtime creates all those objects, and handles all of the wiring and all the state management that goes on with that.

A Node-RED flow is essentially JSON code, or a JSON string. So it’s not doing code generation. This is one of the questions we do get sometimes - “Once I’ve created a flow, can I then export that as just a Node.js app and then customize it?”, and that isn’t the Node-RED model. Node-RED isn’t generating JavaScript code that you could then go edit. It generates a JSON flow definition that you can then import in another Node-RED runtime, or you can share with others, whatever it might be… But that’s sort of the artifact of what the Node-RED flow is.

It actually sounds super-approachable to get started with that; thank you for explaining that. It kind of makes me wanna now figure out if I can maybe contribute a module back; that would be really cool.

Yeah. We’ve tried to make it as simple as possible. And again, the real value in Node-RED is the richness of that Palette. Now, I get all sorts of notifications every time the flow library finds something new. We’ve got a Slack channel, and we tweet about new stuff… And just seeing that steady stream of updates and new stuff coming through - it’s always amazing what people are creating just from really obscure APIs that someone’s had a need to.

There’s a brilliant node - and I’ve never really dug into who’s created it - that can give you a list of German public holidays. It’s been around for a long time, and I keep meaning just to ping its owner and find out, with no sense of criticism, why; it’s awesome that you’ve created that, and I’d love to know your use case for needing a node that can tell you the German public holidays.

[36:02] But it’s also great seeing – I always get a buzz when you see a node that’s obviously being made and maintained by a company for their own API, rather than someone in the open source community creating a node for someone else’s API… Because that’s also a nice sense of validation that “Here’s a company who are paying attention to what can be done, and are embracing the open source community by getting their thing (whatever it may be) enabled within the Node-RED ecosystem.”

That’s very cool. Could you also just quickly tell us why you’re so popular in Japan with Node-RED, and what happened there? What’s the story?

Yeah. I don’t know what sparked it, but one of our most active communities is in Japan. They have a Node-RED user group, they have monthly well-attended meetups… I have been invited and dialed into a couple of their meetups, and sat through an hour of Node representations in Japanese… Not understanding a word, but seeing some really nice demonstrations and slides.

I don’t know what exactly sparked it, but yeah, there does seem to have been a really awesome community that sprung up in Japan, to the extent – I mean, clearly, Hitachi is based in Japan, and they have a large investment, but… The Node-RED user group spans multiple companies, and individuals and companies in Japan. People often ask me “Why haven’t you written a book about Node-RED?” and it’s on the to-do list… But I slightly tongue-in-cheek point them at the three Japanese books about Node-RED on Amazon. I have copies of two of them on the shelf behind me, and I just think it’s brilliant.

As Node-RED is an editor, it’s very user-centric, early in its life I was strongly encouraged to build in internationalization in the UI… And I sort of begrudgingly did it, because it was like three months of not adding any new features, of just building an infrastructure to support multiple languages, finding every message and putting it out into a catalog… It was a slog, but we did it. But by virtue of doing that, then along comes Hitachi, who now maintains the Japanese translations. We have Chinese translations, we have Korean translations, we’ve recently gained German translations…

Again, that has certainly helped the fact that, you know, here’s a tool that if it was English-only - sure, it would get adoption around everyone resigns themselves to having to use English around the world; but the fact that, particularly for the Japanese market, they are able to use it in Japanese, clearly that has helped there. And as ever, interesting challenges around an open source project and maintaining translations, because that is hard work, particularly when it is a language that you are neither a native speaker, nor – I can get by in a number of European languages, and I don’t mean conversant; I can look at the translation and with Google Translate figure it out… But Japanese, Chinese - I would not have the confidence at all to… You know, when we get a pull request fixing a spelling, it’s –

Yeah, absolutely.

We wanted to finish up by talking about what sort of exciting things are happening with Node-RED in the future. I heard that there’s an upcoming release happening, which is version 1.0, which is a big deal… Can you tell us a little bit about that?

Yeah, so about two years we realized there are enough people using Node-RED in a very stable, in-production, commercial setting that we really ought to have at least a 1.0 version number… But we knew there were some technical changes we wanted to make, or enhancements that kind of rounded out the first chapter, if you like, of Node-RED. So we set out a technical roadmap to get to 1.0, and in that blog post two years ago I said “We’re not gonna put dates on it, but I would hope to be there in about six months’ time.” Well, here we are, two years later, and that six months has clearly not quite worked out… But we are on the cusp of releasing Node-RED 1.0. And when I say that, chances are by the time people are listening to this, we may have already got there. It is that perilously close.

So yeah, we’re doing a 1.0 release, which is kind of a big deal. Funnily enough, the code is all there, we’ve done beta releases, they’re there today… I mean, the hard part is just getting all the non-code stuff in place - updating documentation, doing some video tutorials… All this good stuff to really make sure this 1.0 release – we want to get across this message really, as I say, that Node-RED is good for production. There are people who say “Well, that’s just for prototyping” or “You wouldn’t build a real application in it…” We’ve got all this evidence that suggests otherwise, and I think we want to use this 1.0 release to really amplify that message, that Node-RED is stable, it is production-ready.

Now, of course, alongside being a major version change, we are taking this opportunity to tidy up some of the rougher or some of the bits that we have chosen not to tidy up, because they could have a bit more of an impact. Things like in the editor we’ve completely overhauled the CSS naming, because it was a bag of random CSS classes, to the extent there were still some CSS classes dating back to the original D3 example, like cribbed from six years ago.

Yeah, so we’ve properly namespaced everything, but one of the features we have is people can create custom themes. Now, of course those custom themes are going to be broken in 1.0 until they update the CSS classes, so… It’s things like that, but for most end-users, that isn’t gonna touch them at all.

There are some other more significant changes in the runtime. This is a bit deep in the weeds, but the message passing between nodes would typically be asynchronous, but there were cases where it could be synchronous, just by virtue of event emitter itself in Node.js being a synchronous API. It all depended on what a node did as to whether things would be handled asynchronously or not.

We’re going fully asynchronous with 1.0, so every event of a message going between nodes will become asynchronous, because that unlocks a whole bunch of cool stuff in the future, a whole bunch of new features we’ve got in the roadmap. A flow debugger, create breakpoints in your flow and actually pause the runtime and see the state of messages in the system - that’s on the roadmap. Being able to build in better timeout handling across your flows… A whole host of stuff.

That’s awesome. No more Zalgo in Node-RED, where you’re not sure if it’s gonna be synchronous or asynchronous. I know I’ve been bit by that before, and that’s great to hear that that’s getting done, especially.

[43:54] Yeah, it is an interesting challenge. We’ve all taken this opportunity to make some what would be breaking changes, but we’re trying to absolutely minimize the breakage to a smaller subset of users, and as containable as possible. Because I could take a flow that I created on the second week of the project’s existence and import it today, and it will still work. That’s being part of our mantra around stability of Node-RED. Getting to 1.0 will improve that stability moving forward, and like I said, it unlocks a whole bunch of interesting stuff that we can then start getting to in the future.

When you say “we”, I know that you’re still responsible for the bulk of this work, and you work on this a lot. Given that you have a day job and you work at IBM, do you get an opportunity to work on this as part of being on the clock, or is this completely off-the-clock, or can we talk about this? Because I wanna make sure that there’s credit where credit’s due.

Sure. I am now in a position where I am paid to work on Node-RED, and I am paid to run the Node-RED project. I’m in this position where what started out as a side project, and was evenings and weekends, found its way through to being something I could justify working on because it enabled my day job, through to the last couple of years where I’ve been a developer advocate for IoT… I could justify spending time on Node-RED because it helps the developer advocacy mission, to the point where now, in the last few months, I am now overtly an open source developer focused on Node-RED.

I have managed to turn what was my little side project into my day job, which is a great place to be; it has its ups and downs - now actually Node-RED is the job, as well as the side project, so often those evenings and weekends still get consumed… But I do that in full knowledge of also there are times when I will just turn things off and I will walk away from the project. I think I’ve found the right balance between the day job and the side project aspect of it.

Yes, I’m paid to maintain and run the Node-RED project, which is great, and as a growth project of the OpenJS Foundation, one of our goals is to improve the breadth of contributors to the project. As I mentioned, Hitachi are a big contributor, and they’ve got some really exciting features that they’re interested in and they will be contributing to as we move on.

We know there are others with interest, and it’s that perpetual challenge with open source of just trying to encourage more people to get involved and more people to help maintain and make it a sustainable project. Once we get to 1.0, personally taking a bit of a foot off the gas in terms of just writing code, and looking at “Well, now how can we expand the sustainability of the project, get more people involved, encourage more people and see if we can identify if anything is inhibiting people getting involved as contributors?”, or are they just happy that we’re doing such a bang-up job of it that they don’t need to get involved. But this is not unique to us; it’s a perpetual open source challenge, so something to look at in the next couple of months.

Are you the only active maintainer right now?

No. My colleague here, who still has a day job that’s not overtly Node-RED, the co-creator of it, Dave C.J. He and I continue to maintain it together. And as I said, there’s a really good community of – I was gonna say two tiers of contributors, and I don’t mean that in any way as a judgment of quality; it’s more – there are the core people who know the code backwards, which it tends to be me and a couple of others, who can dive in and implement the meatier stuff… But then there’s a great community of people in the forums, and on the issues list, who are helping field just the dozens of questions we get in the forum every day.

That’s been one of the great things about the community - we’ve got a really strong community of people willing just to take time to help others. That’s a workload that I keep an eye on, but I don’t have to step in very often at all, because our great community is there, helping each other.

[48:13] Even though there’s a Subreddit, and it gets new messages (I would say) every day…

Yeah, yeah. And I had to admit, the Subreddit is one I don’t spend any time in. Likewise, there’s a Facebook group I stumbled across, and I’ve joined it, and I’ve now been made an admin, but I do tend just to keep an eye on it, and when someone asks a really meaty question, I give them a hand, but then I do point them to our Discourse forum, just because that’s where the real activity is, and that’s where the real heart of the community is. We try to help people where they ask the question, but also trying to steer them to where the bulk of people are, just to help not spread things too thin.

So if people want to get started in contributing right now, even before you’ve sort of made improvements to that onboarding process, where is the place they should look? Should they start with the website? Can they easily find the GitHub? Is there a GitHub organization, and things like that?

Yeah, we are Node-RED on GitHub, as the organization. NodeRED.org is the site. We’ve got links from the site to the Discourse forum, to our Slack team… Everything you would expect. And really, come along, get involved, come say hi in Slack. Being self-critical, I think that there’s more I can do on the GitHub side, have more issues that are labeled to help people see where they could get started, that type of stuff. It’s that sort of stuff; for the people interested in contributing at that level, I think there’s more to be done… But there’s certainly a healthy backlog of ideas for the project. So yeah, NodeRED.org is the place to start.

And if people wanna be on the other side of it and just consume that library and get started with it, you’d recommend the same resources as well, right?

Yeah, absolutely. Get started there. There’s lots of ways you can install it. We have pretty much a one-click install on the Raspberry Pi to get it installed on the Pi, which makes it a great experience to just start playing with it really quickly. There’s a whole ton of really cool tutorials that people have created. I’m busy working on a whole series of videos that we’ll have out in a couple of weeks to help with that getting started experience.

I think one of the things is just get stuck in… But I think often when we do workshops and we show people it, that initial apprehension turns to a realization that you can do something quite powerful with it quite quickly, once you just understand what’s what and what you need to do.

Yeah, I heard it’s used in a lot of workshops and things like that, and I think that’s a testament to how easy it is actually to get started with it.

Absolutely. And certainly from a developer advocacy point of view, we use it not because we want to show them Node-RED, but because we want them to use some service in our cloud. But when you’ve only got an hour with them, you don’t want to have to deal with lines of code for an SDK… You can only go so far when you’re waiting for the whole room to type in bits of code to drive an SDK… So the fact that Node-RED abstracts out all of that, people can understand the value of a service, and what the service is capable of much quicker, and see it without having to get weighed down by the code.

I think that’s a really awesome description overall, so I wanted to thank you, Nick, for joining us today. I really loved digging into the technical details of Node-RED, and things like that… That has renewed my enthusiasm to also just give it another visit, too.

Again, thank you so much, and I wish you all the best with the release upcoming, and we’ll all be excited to check it out.