1. Introducing Node-RED and flow-Based Programming
- What is FBP?
- What is Node-RED?
- Node-RED benefits
- Node-RED and IoT
What is FBP?
It's the workflows you use in your work
Workflows
In a normal workflow,
boxes and wires indicate the process flow.
It may be just one business design.
Boxes represent processes.
Box processing is defined by who, when, where, what, and how much.
Sometimes, it's like explicitly writing out the flow of processing,
such as by using swim lanes or placing writing definitions inside boxes.
In any case, looking at the box should reveal what will be done.
On the other hand,
let's try to summarize this business process as a document.
Don't you think it will be complicated?
Who will do what as they read it,
even if they use some paragraphs well to put it together?
When will you do it?
It could be confusing:
Now, let's get back to software programming.
FBP is a kind of concept for softwareprogramming
that defines an application with a data flow
Each part of the process is there as a black box.
They communicate data between connected black boxes
that have been predefined.
FBP is said to be component-oriented
because these black-box processes can be connected repeatedly
to form several applications without needing to be modified internally.
Let's explore FBP in more detail.
Flow-based programming (FBP)
I think FBP is a good blend of workflow and dataflow.
FBP uses a data factory metaphor to define an application.
It sees an application as a network of asynchronous processes
that start at some point and do a single sequential process
that does one operation at a time until it ends,
rather than communicating by using a stream of structured chunks of data.
This is called an information packet (IP).
This view focuses on the data and its transformation process
to produce the output that is needed.
Networks are usually defined outside a process as a list of connections
that is interpreted by a piece of software called a scheduler.
Processes communicate via fixed capacity connections.
Connections are connected to processes using ports.
The port has a specific name
that is agreed on by the network definition and the process code.
At this point,
it is possible to execute the same code by using multiple processes.
A particular IP is usually only owned
by a single process or transferred between two processes.
The port can be either a normal type or an array type.
FBP applications typically run faster than traditional programs,
since FBP processes can continue to run
as long as there is room to put in data and output to process.
It does not require any special programming
and makes optimal use of all the processors on the machine.
FBP has a high-level, functional style
so that the behavior of the system can be easily defined;
for example,
in a distributed multi-party protocol
such as a distributed data flow model,
for accurately analyzing the criteria for determining
whether a variable or statement behaves correctly:
Now that you have a solid understanding of FBP,
let's learn how Node-RED can be implemented in this way.
What is Node-RED?
Node-RED is one of the FBP tools that we have described so far.
Developed by IBM's Emerging Technology Services team,
Node-RED is now under the OpenJS Foundation.
Overview
FBP was invented by J. Paul Morrison in the 1970s.
As we mentioned earlier,
FBP describes the behavior of the application as a black box network,
which in Node-RED is described as a "node."
Processing is defined in each node; data is given to it,
processing is performed using that data,
and that data is passed to the next node.
The network plays the role of allowing data to flow between the nodes.
This kind of programming method is very easy to use
to make a model visually and makes it easy to access for several layer users.
Anybody can understand what the flow is doing
if a problem is broken down into each step.
That's why you don't need to the code inside the nodes:
Flow editor and runtime
Node-RED is not only a programming tool
but also an execution platform that wraps up
the Node.js runtime for applications that are built using Node-RED.
We need to use the flow editor
to make Node-RED applications for IoT, web services, and more.
The flow editor is also a Node.js web application.
We will tell you how to use flow editor clearly
in Chapter 3,
Understanding Node-RED Characteristics by Creating Basic Flows.
The flow editor,
which is the core function of Node-RED,
is actually a web application made with Node.js.
It works with the Node.js runtime.
This flow editor operates within the browser.
You must select the node you want
to use from the various nodes in the palette and drag it to the workspace.
Wiring is the process of connecting the nodes to each other,
which creates an application.
The user (developer) can deploy the application to the target runtime with just one click.
The palette that contains various nodes can easily be expanded
as you can install new nodes created by developers,
meaning you can easily share the flow you created as a JSON file to the world.
Before we explore the benefits of Node-RED,
let's look at the brief history behind its creation.
History and origin of Node-RED
In early 2013, Nick-O'Leary and Dave Conway-Jones from IBM UK's Emerging
Technology Services Team created Node-RED.
Originally, it was a just proof of concept (PoC)
to help visualize and understand
the mapping between Message Queue Telemetry Transport (MQTT) topics,
but soon, it became a very popular tool
that could be easily extended to various uses.
Node-RED became open source in September 2013
and remains to be developed as open source now.
It became one of the founding projects of the JS Foundation in October 2016,
which has since merged with the Node.js Foundation
to create the OpenJS Foundation, doing so in March 2019.
The OpenJS Foundation supports
the growth of JavaScript and web technologies as a neutral organization
to lead and keep any projects and fund activities jointly,
which is beneficial to the whole of the ecosystem.
The OpenJS Foundation currently hosts over 30 open source JavaScript projects,
including Appium, Dojo, jQuery, Node.js, and webpack.
Node-RED has been made available under the Apache 2 license,
which makes it favorable to use in a wide range of settings,
both personal and commercial:
Why is it Called Node-RED?
The official documentation (https://nodered.org/about/ states
that the name was an easy play on words that sounded like "Code Red."
This was a dead end, and Node-RED was a big improvement
on what it was called in its first few days of conception.
The "Node" part reflects both the flow/node programming model,
as well as the underlying Node.js runtime.
Nick and Dave never did come to a conclusion
on what the "RED" part stands for.
"Rapid Event Developer" was one suggestion,
but it's never been compelled to formalize anything.
And so, the name "Node-RED" came to life.
Node-RED benefits
Let's think a little here. Why do you use cars?
I think the answer is very simple and clear.
First of all, we can come up with the answer
that they are used as a means of transportation in a broad sense.
There are other options for transportation, such as walking, bicycle, train, and bus.
Then, we have the reasons for choosing a car from among these other options,
as follows:
• You do not get exhausted.
• You can reach your destination quickly.
• You can move at your own pace.
• You can keep your personal space.
Of course, there are some disadvantages,
but I think these are the main reasons for using a car.
Although other means of transportation can also serve the same purpose,
the important thing is to consider the advantages and disadvantages of each,
and use the car as a transportation tool
for the reason that you feel is the most suitable to you.
We can see the same situation in software.
As an example, why do you use Word, Excel, and PowerPoint?
You'll probably use Word
because it's the most efficient way to write a document.
However, you could use a word processor separately or handwrite anything.
Similarly, instead of Excel,
you can use any other means to make spreadsheets.
There are also other means
if you want to make presentation materials and make them look effective,
besides PowerPoint.
However, you are likely to choose the optimum tool for your situation.
Let's recall what Node-RED is for.
It is a FBP tool, suitable for making data control applications
for web applications and IoT.
Its development environment and execution environment are
browser-based applications made with Node.js,
which makes their development as easy as possible.
So, what is the reason for using Node-RED, which provides these kinds of features? Do
you want to avoid heavy coding? Do you not have coding skills? Yes, of course, these are
also reasons to use the program.
Let's recall the example of a car. In a broad sense, our dilemma (transportation) is
replaced here by developing (creating) a Node.js application for describing software tools.
The transport options, such as cars, bicycles, trains, buses, ships, planes, and so on, are
options, and with software development, we also have numerous options, such as using
Node.js scratch, or using various frameworks of Node.js and using Node-RED. As for
reasons to choose Node-RED, let's take a look at some essential points.
Simplification
When programming with Node-RED,
you'll notice its simplicity.
As the name no-code/ low-code indicates,
coding is eliminated and programming is intuitively
completed with a minimal number of operations needing to be used.
Efficiency
The FBP typified by Node-RED can be completed with almost only GUI operations.
Node-RED flow editor takes care of building the application execution environment,
library synchronization, the integrated development environment (IDE), and editor
preparation so that you can concentrate on development.
Common
As represented by object-oriented development,
making the source code a common component is
one of the most important ideas in development.
In normal coding-based development,
each common component exists in functions and classes,
but in Node-RED,
they exist as an easy-to-understand node (just a box).
If you don't have a node as a common component you want to use,
anyone can create one immediately and publish it to the world.
High quality
High quality is the true value of flow-based and visual programming.
Each node provided as a component is a complete module
that has been unit tested.
As a result,
app authors can focus on checking the operation
at the join level without worrying about the contents of node.
This is a big factor that eliminates human error
at the single level and ensures high quality.
Open source
Node-RED is an open source piece of software.
Therefore, it can be used flexibly under the Apache2 license.
Some are developing their own services based on Node-RED,
while others are changing to their own UI and deploying it as built-in.
As we mentioned previously,
we have also established a platform
where we can publish our own developed
node so that anyone can use it.
Node-RED library
The library indexes all Node-RED modules published to the public npm repository
(https://www.npmjs.com/),assuming they follow the proper packaging guidelines.
Various platforms
Node-RED can be used on various platforms.
That's because Node-RED itself is a Node.js application,
as we mentioned previously.
If you have a runtime environment for Node.js,
you can run it.
It is mostly used on Edge devices, cloud services,
and in embedded formats.
You can get a sense of this
by understanding the relationship between Node-RED and IoT
and the architecture of IoT, which will be explained in the next section
Node-RED and IoT
Again, Node-RED is a virtual environment
that combines hardware devices, APIs, and
online services in a revolutionary way on a browser.
It provides the following features:
• Browser-based UI.
• Works with Node.js and is lightweight.
• Encapsulates function and can be used as a node
(meaning functions are locked in an abstract capsule) .
• You can create and add your own nodes.
• Easy access to IBM Cloud services.
In other words,
it can be said that this tool is suitable for building IoT-related services,
such as data control on devices, and linking edge devices and cloud services.
Originally, the development concept of Node-RED was for IoT,
so this makes sense.
Now, let's look at the basic structure of IoT
so that those who are only vaguely aware of IoT can understand it.
It can be said
that IoT is basically composed of six layers,
as shown in the following diagram:
Device
The device is a so-called edge device.
IoT has various sensors and handles the data that's acquired from them.
Since it doesn't make sense to have the data only on the edge device,
we need to send that data through the gateway to the network.
Network
This is the network that's required to send the data
that's been obtained from the device to a server on the internet.
It usually refers to the internet.
In addition to the internet,
there is also a P2P connection via Bluetooth or serial.
Platform
The party that receives and uses the data is the platform.
We may also have a database
for activating and authenticating things, managing communications,
and persisting receiveddata.
Analytics
This is a layer that analyzes the received data.
Broadly speaking, it may be classified as an application.
This is the part that prepares the data
so that it can be processed into a meaningful form.
Application
An application provides a specific service based on data analysis results.
It can be a web or mobile application, or
it can be a hardware-specific embedded application.
It can be said to be the layer
that's used by the end user of the IoT solution.
Now that we have an understanding of IoT,
we will examine why Node-RED should be used for it.
While explaining IoT so far,
we've made it clear why Node-RED is suitable for IoT.
For example,
you can understand why FBP tools that have been developed for IoT
survive when used with Node-RED.
In particular, the following three points should be taken into account:
• Since it can be run on edge devices
(pre-installed on specific versions of Raspberry Pi OS),
it is ideal for data handling at the device layer.
• Since it can be run on the cloud
(provided as a default service in IBM Cloud),
it is easy to link with storage and analysis middleware.
• Since MQTT and HTTP protocols can be covered,
it is very easy to exchange data
between the edge device and the server processing cloud.
In this way,
Node-RED, which largely covers the elements required for IoT,
is now used for a wide range of applications,
such as web services and chart display, as well as programming for IoT.
Also, as of June 2020,
if you look at Google Trends for Node-RED,
you can see that the number of users is gradually increasing.
As such, Node-RED is a very attractive FBP tool:
A typical edge device that can use Node-RED is Raspberry Pi.
Of course, it is possible to use Node-RED on other platforms,
but it goes well with Raspberry Pi,
which also has a pre-installed version of the OS.
IBM Cloud is a typical cloud platform that can use Node-RED.
Of course, you can use Node-RED on other clouds,
but IBM Cloud provides a service that anyone can easily start.
Summary
In this chapter, you learned what FBP and Node-RED are.
Due to this, you now understand why Node-RED is currently loved
and used by lots of people as an FBP tool.
At this point,
you may want to build an application using Node-RED.
In the next chapter,
we'll install Node-RED in our environment
and take a look at it in more depth.
2. Setting Up the Development Environment
In this chapter, you will install the tools that you'll need to use Node-RED.
This extends not only to Node-RED itself, but to its runtime, Node.js,
and how to update both Node- RED and Node.js.
Node-RED released its 1.0 milestone version in September 2019.
This reflects the maturity of the project,
as it is already being widely used in production environments.
It continues to be developed and keeps up to date
by making changes to the underlying
Node.js runtime.
You can check the latest status of Node-RED's installation
at https:// nodered.org/docs/getting-started/.
There are a number of installation guides
on the Node-RED official website,
such as local, Raspberry Pi, Docker, and major cloud platforms.
In this chapter,
you will learn how to install Node-RED on your local computer,
whether you are running it on Windows, Mac, or on a Raspberry Pi.
We will cover the following topics:
• Installing npm and Node.js for Windows
• Installing npm and Node.js for Mac
• Installing npm and Node.js for Raspberry Pi
• Installing Node-RED for Windows
• Installing Node-RED for Mac
• Installing Node-RED for Raspberry Pi
By the end of this chapter,
we'll have all the necessary tools installed and
be ready to move on to building some basic flows with Node-RED.
For reference,
the author's test operation environment is
Windows 10 2004 18363.476, macOS Mojave 10.14.6 (18G5033),
and Raspberry Pi OS 9.4 stretch.
Technical requirements
You will need to install the following for this chapter:
• Node.js (v12.18.1)*
• npm (v6.14.5)*
*LTS version at the time of writing for both.
Installing npm and Node.js for Windows
If you want to use Node-RED on Windows,
you must install npm and Node.js via the following website:
https://nodejs.org/en/#home-downloadhead.
You can get the Windows Installer of Node.js directly there.
After that, follow these steps:
1. Access the original Node.js website and download the installer.
You can choose both versions – Recommended or Latest Features –
but in this book, you should use the Recommended version:
2. Click the msi file you downloaded to start installing Node.js.
It includes the current version of npm.
Node-RED is running on the Node.js runtime, so it is needed.
3. Simply click the buttons of the dialog windows
according to the installation wizard, though there are some points
to bear in mind during the install.
4. Next, you need to accept the End-User License Agreement:
You can also change the install destination folder.
In this book, the default folder (C:/Program Files/nodejs/) will be used:
5. No custom setup is needed on the next screen.
You can select Next with only the default features selected:
6. On the following screen, you can click Next without checking anything.
However, it's OK to install the tools that can be selected here.
This includes the installations and settings the path of these environments
(Visual C++, windows-build-tools, and Python):
7. Check the versions of your tools with the following commands
when the installation for Node.js has finished:
When the installations of Node.js and npm are complete,
you can check their version numbers.
With this, you are prepared to install Node-RED.
Installing npm and Node.js for Raspberry Pi
If you want to use Node-RED on Raspberry Pi,
congratulations – you are already prepared to install Node-RED.
This is because Node.js and npm are installed by default.
You can use the existing installation script to install Node-RED,
including Node.js and npm.
This script will be described later in this chapter,
in the Installing Node-RED for Raspberry Pi section,
so you can skip this operation for now.
However,
you should check your Node.js and npm versions on your Raspberry Pi.
Please type in the following commands:
If it is not the LTS version or stable version, you can update it via the CLI.
Please type in and run the following commands to do this.
In this command, on the last line,
lts has been used, but you can also put stable instead of lts
if you want to install the stable version:
Now that we have successfully checked the versions of Node.js
and npm on our Raspberry Pi and
updated them (if applicable),
we will move on to installing Node-RED for Windows.
Installing Node-RED for Windows
In this section,
we will explain how to set up Node-RED in a Windows environment.
This procedure is for Windows 10, but it will work for Windows 7
and Windows Server 2008 R2 and above as well.
Windows 7 or earlier versions of Windows Server 2008 R2
are not currently supported and are not recommended.
For Windows, installing Node-RED as a global module
adds the node-red command to your system path.
Run the following command in Command Prompt:
Once you have finished installing Node-RED,
you can use Node-RED straight away.
Please run the following command.
After running this command,
you will recognize the URL being used to access the Node-RED flow editor.
Usually, localhost (127.0.0.1) with the default port 1880 will be allocated:
Let's access Node-RED on a browser. For this, type in the URL you received from Command Prompt. I strongly recommend using Chrome or Firefox for running Node- RED:
Now, you are ready to program in Node-RED.
From Chapter 3, Understanding Node-RED Characteristics by Creating Basic Flows,
onward, we will learn how to actually build an application using Node-RED.
Installing Node-RED for Raspberry Pi
we will explain how to set up Node-RED in a Raspberry environment.
This procedure is for Raspberry Pi OS Buster (Debian 10.x),
but it will work for Raspberry Pi OS Jessie (Debian 8.x) and above.
You can check your version of Raspberry Pi OS easily.
Just run the following command on your Terminal:
$ lsb_release -a
If you want to also check the version of Debian you have,
please run the following command:
$ cat /etc/debian_version
You have now prepared to install Node-RED.
The following script installs Node-RED, including Node.js and npm.
This script can also be used for upgrading your application,
which you have already installed.
This script works on Debian-based operating systems,
including Ubuntu and Diet-Pi:
You may need to run sudo apt install build-essential git
to ensure that npm can build the binary components that need to be installed.
Node-RED is already packaged as a Raspberry Pi OS repository
and is included in the Recommended Software list.
It can be installed with the apt-get install Node-RED command,
and it also contains a Raspberry Pi OS packaged version of Node.js,
but npm is not included.
While using these packages may seem convenient at first glance,
it is highly recommended to use the installation script instead.
After the installation, you can start Node-RED and
access the Node-RED flow editor.
We have two ways to start it, as follows:
1. Run with the CLI:
If you want to run Node-RED locally,
you can start Node-RED by using the node-red command in your Terminal.
Then, you can stop it by pressing Ctrl + C or closing the Terminal window:
$ node-red
2. Run via Programming menu:
Once Node-RED has been installed, you can start it from the Raspberry Pi menu.
Click Menu | Programming | Node-RED to open the Terminal and launch Node-RED.
Once Node-RED has been launched,
you can access the Node-RED flow editor from your browser,
just as you would in the CLI:
After launching Node-RED from the menu,
you should check the Node-RED running process on your Terminal
and find the URL of the Node-RED flow editor.
It is usually the same URL as the one that can be launched via the CLI directly:
Let's access Node-RED on a browser.
You can type in the URL you received from the Command Prompt to do this.
If your Raspberry Pi default web browser is Chromium,
then there should be no problems with using Node-RED.
However, if you wish to use another browser,
I strongly recommend installing Chromium for running Node-RED:
And that's it! We have now covered all the installation options
for each tool we'll need in order to start using Node-RED.
Summary
In this chapter, you've gotten your environment ready
so that you can use the Node-RED flow editor.
At this point,
I believe that you can already access the Node-RED flow editor,
so you'll want to learn how to use it.
In the next chapter,
we'll make a sample flow on it and learn
about the major features of the Node-RED flow editor.
3. Understanding Node-RED Characteristics
by Creating Basic Flows
In this chapter, we'll actually create a flow using Node-RED Flow Editor.
By creating a simple flow,
you will understand how to use the tool and its characteristics.
For a better understanding, we will create some sample flows.
From now on, you will create applications called flows using Node-RED.
In this chapter, you will learn how to use Node-RED and
how to create an application as a flow.
To do this, we will cover the following topics:
• Node-RED Flow Editor mechanisms
• Using the Flow Editor
• Making a flow for a data handling application
• Making a flow for a web application
• Importing and exporting a flow definition
By the end of this chapter,
you will have mastered how to use Node-RED Flow Editor and
know how to build a simple application with it.
Technical requirements
To complete this chapter, you will need the following:
• Node-RED (v1.1.0 or above).
• The code for this chapter can be found in Chapter03 folder at
https://github.com/PacktPublishing/-Practical-Node-RED-Programming.
Node-RED Flow Editor mechanisms
As you learned in the previous chapters,
Node-RED has two logical parts:
a development environment called the Flow Editor
and an execution environment for executing
the application that's been created there.
These are called the runtime and the editor, respectively.
Let's take a look at them in more detail:
• Runtime: This includes a Node.js application runtime.
It is responsible for running the deployed flows.
• Editor: This is a web application where the user can edit their flows.
The main installable package includes both components,
with a web server to provide Flow Editor
as well as a REST Admin API for administering the runtime.
Internally, these components can be installed separately
and embedded into existing Node.js applications,
as shown in the following diagram:
Now that you understand the mechanisms of Node-RED,
let's immediately learn how to use the Flow Editor.
Using the Flow Editor
Let's take a look at the main functions of the Flow Editor.
The main features of the Flow Editor are as follows:
• Node: The main building block of Node-RED applications,
they represent well- defined pieces of functionality.
• Flow: A series of nodes wired together that represent the series of
steps messages pass through within an application.
• The panel on the left is the palette: A collection of nodes
that are available within the editor that
you can use to build your application.
• Deploy button: Press this button to deploy your apps once you've edited them.
• Sidebar: A panel for displaying various functions,
such as processing parameter settings, specifications,
and debugger display.
• Sidebar tabs: Settings for each node, standard output,
change management, and so on.
• Main menu: Flow deletion, definition import/export,
project management, and so on.
These functions are arranged on the screen of the Flow Editor like so:
You need to understand what is contained in the Flow menu
before you start using Node-RED.
Its contents may differ,
depending on the version of Node-RED you're using,
but it has some setting items such as
Project management of flow, Arrange view,
Import / export of flow, Installation of node published in library,
and so on that are universal.
For more information on how to use Node-RED,
it's a good idea to refer to the official documentation as needed.
The following diagram shows all these Flow Editor menu options inside Node-RED:
With that, you are ready to use Node-RED to build an application.
So, let's get started!
First of all, you need to run Node-RED in your environment.
Please refer to Chapter 2, Setting Up the Development Environment,
to learn how to set it up with your environment,
such as Windows, Mac, or Raspberry Pi,
if you haven't done so already.
With Node-RED running,
let's move on to the next section, where we'll be making our first flow.
Making a flow for a data handling application
In this section, you will create a working application (called a flow in Node-RED).
Whether it is the internet of things (IoT) or server processing as a web application,
the basic operation that Node-RED performs is sequentially transferring data.
Here, we'll create a flow where JSON data is generated in a pseudo manner,
and the data is finally output to standard output via some nodes on Node-RED.
There are many nodes on the left-hand side of the palette.
Please pay attention to the common categories here.
You should be able to easily find the inject node,
as shown in the following screenshot:
This node can inject a message into the next node.
Let's get started:
1. Drag and drop it onto the palette of Flow 1 (the default flow tab).
You will see that the node is labeled with the word timestamp.
This is because its default message payload is a timestamp value.
We can change the data type, so let's change it to a JSON type.
2. Double-click the node and change its settings
when the Properties panel of the node is opened:
3. Click the drop-down menu of the first parameter and select {}JSON.
You can edit the JSON data by clicking the […] button on the right-hand side.
4. Click the […] button, and the JSON editor will open.
You can make JSON data with a text-based editor or a visual editor.
5. This time, let's make JSON data with an item called {"name" : "Taiji"}.
You should replace my name with your name:
Great – you have successfully made some sample JSON data!
6. Click the Done button and close this panel.
7. Similarly, place a Debug node on the palette.
8. After placing it, wire the Inject and Debug nodes to it.
Once you execute this flow,
the JSON data that was passed from the Inject node
will be output to the debug console (standard output) by the Debug node.
You don't need to configure anything on the Debug node:
9. Finally, you need to deploy the flow you created.
In Node-RED Flow Editor, we can deploy all our flows on the workspace
to the Node-RED runtime by clicking the Deploy button in the top-right corner.
10. Before running the flow, you should select the Debug tab
from the node menu's side panel to enable the debug console,
as shown in the following screenshot:
