In this class, you will become familiar with dataflows and the replication of dataflows, which allows users to create logic (i.e., flow) between devices and datapoints and apply contexts in the system to that flow for ease of replication.

This section consists of the following:

Class Contents

Understanding Flows

A flow is a combination of one or more Node-RED nodes connected together to form a sequence or business logic; additionally, flows can support a webpage or a dashboard and provide a simple way to create custom user interfaces.

Depending on how the workflow is designed and where it ultimately runs, the following types are supported:

Dataflows – consist only of Node-RED flows/sub-flows without a supporting webpage dashboard. Dataflows are primarily useful for defining cross-system sequence of operations (if-then-else and other logical operations) and can be used to extending SmartServer IoT capabilities by exposing these data-points to a wide range of standard nodes available in Node-RED and third party integration nodes.
Webflows – consist of both the Node-RED flows/sub-flows and a supporting webpage dashboard. Webflows are primarily used to quickly create custom web experiences for facility management and operations, exposing datapoint driven dashboards and control panels.

Exercise 3.1 - Creating a Dataflow will allow you to put this concept of dataflows into practice.

Understanding Replication

Replication provides the ability to build a flow once and apply it multiple times. It allows for creation of standard, template flows that can be replicated across multiple contexts or exported from one SmartServer to the next, therefore eliminating the need to build flows from scratch for every duplicate context or project. In this way, replication simplifies programming and integration using the SmartServer platform.

Replication acts on a flow that consists of at least one relative datapoint (see Class 2 - Contexts and IAP Nodes for information about fixed and relative datapoints). As the name suggests, a relative datapoint is a datapoint on a device type and not an actual device. Once a flow with a relative datapoint is deployed, you can apply one or more contexts to the flow. When you apply a context to a flow, a matching algorithm runs in the background matching datapoints that exists on that context to that relative datapoint automatically. If the flow is setup correctly, you have now replicated a flow with one action saving you lots of time.

Since contexts are hierarchical, every context has at most one parent and can have any number of children (zero or more). And, it is recursive in that the parent may have its own parent, and the children may have their own children.

For example: a room or area context belongs to a single floor, that belongs to a building, that belongs to a campus. Resolution of relative references is strictly limited to the context’s hierarchy, which is done in search phases. There are two separate search phases. The first phase is the descent and the second is the ascent.

During the descent, the context being applied and all its children are visited recursively until all the leaf nodes (no more children) are hit. If there is a flow target, the descent only operates on contexts defined by the flow target.

During the ascent, the traversal goes to the parent, then its parent, and so on, until the root node is hit. During the descent, each visited context is conditionally applied to the flow. The application to the flow is conditional because a flow may not qualify for a particular context.

In order for a flow to qualify, the following conditions must be true:

At least one relative IAP Input node or IAP Output node must be resolved at that level
All the inputs feeding into the flow must be resolvable using datapoints at that level, or by those found during the ascent

It is worth pointing out that you can select a desired behavior of a device by its device type when replicating a flow. For example, you can have two physically identical devices defined by one interface file, but you can change which relative datapoints match to a datapoint in that device by creating a different device types. Because the algorithm matches by device type, you can tailor the match with the outcome you desire.

Exercise 3.2 - Replicating a Dataflow will help you to better understand the replication of a Node-RED flow in the SmartServer by applying a flow to multiple contexts and viewing the flow running with data from datapoints over multiple contexts.

Exercise 3.1 – Creating a Dataflow

In this exercise, you will use Node-RED to create a simple control loop that feeds data from the two different room sensors into the VAV controller. For this type of use-case, where the HVAC system in a building is being managed, consider the following scenario:

A large room, such as the one being designed in this exercise, can have multiple temperature sensors and occupancy sensors. Data gathered from these sensors is used to maintain the optimal room temperature for comfort and energy efficiency. Typically the readings from all the temperature sensors are averaged to get the relevant space temperature and fed into the VAV controller along with the occupancy state of the room. Occupancy state is binary – it is either occupied or unoccupied. Therefore, an “or” logic is applied to outputs from all of the occupancy sensors in a room.

The VAV controller plays a key role in determining what happens in this scenario. The VAV controller receives the temperature and occupancy state of the room and determines how much hot or cool air to let in by opening or closing air inlets and by heating or cooling the incoming air. It tries to maintain a setpoint for an occupied, unoccupied, or standby state using sophisticated PID loop algorithms. All of these datapoints are variables in the equation.

The objective of this exercise is not to create the most complex HVAC algorithm, but to demonstrate how to use Node-RED to feed data into and out of the system to maintain a balanced temperature. To this end, a simple model is used where an average space temperature is fed from the two sensors in the room to the VAV, as well as occupancy state. In addition, only the cooling mode of the VAV controller will be implemented.

The block diagrams below represent the model for this exercise.

The flow in Exercise 3.1 can be created in one of two ways:

Using fixed datapoints
Using relative datapoints

Since this flow will be used as a basis for Exercise 3.2 that demonstrates replication, relative nodes will be used in the steps below. Replication requires the use of relative nodes and the application of flows to contexts.

To create this flow, perform the following steps:

Download Exercise 3.1 from the SmartServer GitHub Repository → Node-RED Tutorial folder and import it into the Sequencing widget.

This downloaded flow will serve as a template for the flow that you will be creating in this exercise, and will look similar to the following in your workspace:
Click Deploy to save the flow.
Click the Add Flow button () to begin creating a new flow.

A new flow tab is added to the workspace.
Double-click the new tab on the workspace.

The Edit flow view appears.
Enter the flow name as shown above and click Done.
Create and provision the BACnet room sensor:
1. Import the VT7200f5031B-V20.dtp from the Starter Kit archive → BACnet Channel → Viconics folder for the Viconics Thermostat device using the Device Types widget.
2. Create a new device using the Devices widget (see Discovering, Defining, or Importing Devices).
  1. Name: Tstat-01
  2. UID: 72008
  3. Integration method: Manual assignment
  4. Driver: BACnet
  5. Device type: VT7200-1H1C-3 (with VT7200f5031B-V20.dtp)
3. Provision the Viconics Thermostat device on your Starter Kit.
4. Assign the device to the Mango HQ/Building 1/Floor 1/Office 1 context using the Devices widget (see Discovering, Defining, or Importing Devices).
  
  See the SmartServer IoT's Starter Kit User's Guide → Setting up the BACnet Channel section for more information about creating and setting up this device.
Copy and paste the IAP Output nodes (VAV_sim nviSpaceTemp and VAV_sim nviOccSensor) from Exercise 2.1 to this Exercise 3.1 flow, and arrange as shown below. For all datapoints that are being used, be sure to enable monitoring and polling in the Datapoints Properties widget → MONITORING AND LOGGING CONFIGURATION settings, and set Poll Interval to 10 seconds and Publish Minimum Delta Value to Always.
Drag the IAP Input node onto the workspace.
Double-click the IAP Input node.

The Edit iap-input node view appears.
Enter VT7200 AV 7 Room Temp in the Name field.
Verify that the View all datapoints checkbox is enabled to list all datapoints.
Apply filters to select the datapoint Room Temperature (the reading from the temperature sensor in the Viconics device [AV 7]) for the Tstat-01 device.
Click Done.
Click Deploy to save your changes.
Drag the IAP Input node onto the workspace, placing it below the VT7200 AV 7 Room Temp node.
Double-click the IAP Input node.

The Edit iap-input node view appears.
Enter VT7200 BI 1 Status (Occ) in the Name field.
Apply filters to select the BI 1 Status datapoint (the reading from the occupancy sensor in the Viconics device [BI 29]) for the Tstat-01 device.
Click Done.
Click Deploy to save your changes.
Drag the IAP Input node onto the workspace, placing it below the VT7200 BI 1 Status (Occ) node.
Double-click the IAP Input node.

The Edit iap-input node view appears.
Enter SC100 nvoTempSensor in the Name field.
Apply filters to select the datapoint nvoTempSensor (the reading from the temperature sensor in the SC 100-MP device) for the DIO-01 device.
Click Done.
Click Deploy to save your changes.
Drag the IAP Input node onto the workspace, placing it below the SC100 nvoTempSensor node.
Double-click the IAP Input node.

The Edit iap-input node view appears.
Enter SC100 0/nvoDI (Occ) in the Name field.
Apply filters to select the datapoint nvoDI (the reading for occupancy in the SC 100-MP device) for the DIO-01 device. With multiple nvoDI datapoints, be sure to select [SC100-MP/Digital Input/0/nvoDI].
Click Done.
Click Deploy to save your changes.

Now that the IAP Input and IAP Output nodes are created, six function nodes will be created in the steps that follow.
Drag the function node and place it to the right of the VT7200 AV 7 Room Temp node, connecting the nodes together.
Double-click the function node.

The Edit function node view appears.
Enter data → celsius in the Name field.
In the Function tab, enter the following code to convert temperature from Fahrenheit to Celsius, round the value to 1/10 of a degree Celsius, and display the result as a status below the node:

The VT7200f5031B-V20.dtp file provides additional datapoints that appear in the Datapoints widget. With the VT7200f5031B-V20.dtp file, this function node is required if the BV:51 value is set to 1 (which reports temperature readings in Fahrenheit). This function node is not required if the BV:51 value is set to 0 (which reports temperature readings in Celsius).
```
var msg1 = {};
msg1.payload = {};
msg1.topic = "Room2";
var temp = msg.payload.data;
temp = (temp -32)*5/9;
msg1.payload.data = Math.floor(temp*10)/10;
msg1.payload.topic = msg.topic;
node.status({fill:"blue",shape:"ring",text:msg1.payload.data});
return msg1;
```
Click Done.
Click Deploy to save your changes.
Drag the function node and place it in between the newly created data → celsius function node and the VAV_sim nviSpaceTemp node, connecting the nodes together.
Double-click the function node.

The Edit function node view appears.
Enter Avg. Temp. in the Name field.

In the Function tab, enter the following code to store the incoming data for future use and calculate the average of all temperatures from all of the different nodes that are connected to it:

This function node initializes the data storage, stores the data in that storage, and retrieves the data from other nodes. Once all of the temperature nodes are available, it calculates an average and sends it downstream.

// this is loading all most recent values, then updating the new value in one of
// entries based on the topic which tells us what is the actual device sending it
values = context.get("mostRecentValues");
values.set(msg.topic, msg.payload.data);
// update the node context with latest values
context.set("mostRecentValues",mostRecentValues);

// now iterate through all most revent values and calculate average
var sumTemp = 0;
var count = 0;
values.forEach(function(value, key) {
  //node.warn(key + ' = ' + value)
  sumTemp += value;
  count++;
})
var avgTemp = sumTemp/count;

msg.payload.data = Math.floor(avgTemp*10)/10;
node.status({fill:"blue",shape:"ring",text:msg.payload.data});
return msg;

In the Setup tab, enter the following code to set the most recent values when the node is deployed:
```
mostRecentValues = new Map();
context.set("mostRecentValues", mostRecentValues);
```
Click Done.
Click Deploy to save your changes.
Drag the function node and place it to the right of the VT7200 BI 1 Status (Occ) node, connecting the nodes together.
Double-click the function node.

The Edit function node view appears.
Enter data → occupancy in the Name field.
In the Function tab, enter the following code to convert occupancy information from BACnet objects to a simple true or false value, send it downstream, and display the result as a status below the node:
```
var msg1 = {};
msg1.payload = {};
msg1.topic = "Room2";
msg1.payload.data = msg.payload.data;
node.status({fill:"blue",shape:"ring",text:msg1.payload.data});
return msg1;
```
Click Done.
Click Deploy to save your changes.
Drag the function node and place it to the left of the VAV_sim nviOccSensor node, connecting them, as well as to the data → occupancy function node.
Double-click this new function node that has not yet been configured.

The Edit function node view appears.
Enter Occupancy in the Name field.

In the Function tab, enter the following codes to store the incoming data for future use and test to see if any of the occupancy values are true or not:

This function node initializes the data storage, the function code stores the data in that storage, and retrieves the data from other nodes. Once all of the occupancy values are available, it tests to see if any of them are true, and then sends the result downstream.

// this is loading all most recent values, then updating the new value in one of
// entries based on the topic which tells us what is the actual device sending it
values = context.get("mostRecentValues");
values.set(msg.topic, msg.payload.data);
//node.warn("value("+msg.topic+")="+msg.payload.data);
    
// update the node context with latest values
context.set("mostRecentValues",mostRecentValues);

var occupancy = "OC_UNOCCUPIED";
values.forEach(function(value, key) {

    if (value) occupancy = "OC_OCCUPIED  ";
    //node.warn(key + ' = ' + value)
})

msg.payload.data = occupancy;
node.status({fill:"blue",shape:"ring",text:msg.payload.data});
return msg;

In the Setup tab, enter the following code to set the most recent values when the node is deployed:
```
mostRecentValues = new Map();
context.set("mostRecentValues", mostRecentValues);
```
Click Done.
Click Deploy to save your flow.
Drag the function node and place it to the right of the SC100 nvoTempSensor node, connecting them, as well as to the Avg. Temp. function node.
Double-click the function node.

The Edit function node view appears.
Enter data → celsius in the Name field.

In the Function tab, enter the following code to push the data downstream and display the result as a status below the node:

The LON temperature sensor readings are in Celsius and therefore do not need to be converted.

var msg1 = {};
msg1.payload = {};
msg1.topic = "Room1";
msg1.payload.data = msg.payload.data;
//var temp = msg.payload.data.value;
//msg.payload.data = temp;//(temp -32)*5/9;
node.status({fill:"blue",shape:"ring",text:msg1.payload.data});
return msg1;

Click Done.
Click Deploy to save your changes.
Drag the function node and place it to the right of the SC100 0/nvoDI (Occ) node, connecting them, as well as to the Occupancy node.
Double-click the function node.

The Edit function node view appears.
Enter data → occupancy in the Name field.

In the Function tab, enter the following code to convert occupancy information from a standard switch type to a true or false value, send it downstream and display the result as a status below the node:

var msg1 = {};
msg1.payload = {};
if (msg.payload.data.value === 0) msg1.payload.data = false;
else msg1.payload.data = true;
node.status({fill:"blue",shape:"ring",text:msg1.payload.data});
msg1.topic = "Room1";
return msg1;

Click Done.
Click Deploy to save your changes.

The last two nodes, an IAP Output node and an inject node, will be created to configure the VAV to operate in cooling mode.
Drag the IAP Output node onto the workspace, placing it in between the VAV_sim nviSpaceTemp and VAV_sim nviOccSensor nodes.
Double-click the IAP Output node.

The Edit iap-output node view appears.
Enter VAV_sim nviApplicMode in the Name field.
Apply filters to select the datapoint nviApplicMode for the VAV sim device.
Click Done.
Click Deploy to save your changes.
Drag the inject node onto the workspace and connect it to the VAV_sim nviApplicMode node.
Double-click the inject node.

The Edit inject node view appears.
Select {}JSON from the msg.payload dropdown list.
Enter {"data":"HVAC_COOL"} in the msg.payload field.
Enter ev/data in the msg.topic field.
Click Done.
Click Deploy to save your changes.

Great work creating your first automated application! Now that you have created your first flow template, you can now apply multiple contexts to your flow to replicate your flow to the physical devices in the contexts you select. Continue with Exercise 3.2 to take these next steps.

Exercise 3.2 – Replicating a Dataflow

Now, with replication, automated applications rise to a whole new level of possibility and productivity. In the previous exercise, if you had created a control loop for a specific room using fixed datapoints, the datapoint value inputs to the flow would have been tied to the physical devices you selected. In this case, the IAP Input and Output nodes (and therefore also the associated datapoints) cannot be associated with multiple contexts. Therefore, with fixed datapoints, the datapoint is never modified when you apply a new context to the flow.

But what if every room had the same set of devices and the same algorithm was used for every room, and you want to apply this flow for every room in the building? This exercise will teach you how to do so by replicating the flow using relative datapoints, which are generated from the device type definitions and are not tied to specific devices. In this case, the IAP node (and therefore also the datapoint) can be associated with multiple contexts. Therefore, if you are using a relative datapoint and apply a new context to the flow, the datapoint is modified for the new context.

Before you begin the exercise, it is important to understand the concept of replication. Replication is done with device types rather than with actual devices. A device type can be thought of as a model of a device that follows a certain behavior. This means that if you have two physically identical devices, but want them to behave differently in a Node-RED flow, you will need to create two separate device types. See Device Type Definition for more information.

Consider this example with a lighting controller: you have the same lighting controller from the same manufacturer bulb, with the same interface (XIF) file controlling output to a light. However, if you want one lighting controller to be in a hallway listening to multiple occupancy sensors, and another in a closed office only following the output of a switch, you will need to create two different device types.

With Exercise 3.2, you will take the flow that you created in Exercise 3.1 for room/office 1 and apply it to rooms/offices 2, 3, and 4. Follow the steps in the exercise below and see how replication simplifies and makes more efficient the creation large systems that share commonality and refer to the figure below, which represents your floor model.

The flow you created in Exercise 3.1 will be used for this replication exercise.

To get started, perform the following steps:

Download the context file from the SmartServer GitHub Repository → Node-RED Tutorial folder if you have not already done so.
Go to the SmartServer CMS and import the context file. For SmartServer 4.0 and higher, use the Import / Export button () on the SmartServer CMS. The Import button () is no longer available on the Planning widget and the top level geozone for all contexts types is the World geozone. For SmartServer releases prior to SmartServer 4.0, import contexts using the Import button () on the Planning widget.
The context hierarchy should appear similar to the following.

Planning widget example prior to SmartServer 4.0

In the next series of steps, you will be creating devices using the Devices widget that are not physically part of the Starter Kit.

In terms of system integration, this is known as an engineered system, where an integrator may want to set up contexts and flows before getting to the site and using real devices. In the steps that follows, these devices that do not exist yet will be referred to as logical devices and installed in different rooms of the MangoHQ floor plan.
Go to the SmartServer CMS Devices widget and create three new logical LON room sensors (DIO-01) for each room/office in the floor plan. (The DIO-01 device was created in Exercise 2.1.)
1. Create all of the LON room sensors using the same device type, that is SC100-MP (because the same behavior is desired from them in each room).
2. Name the new devices, for example: DIO-01-2, DIO-01-3, DIO-01-4.
  
  For replication to occur the devices to be replicated must have the same device type. While the device names are not required to follow a specific convention, for this exercise, the naming pattern shown above will be used for the logical devices.
3. Enter a UID for each logical device using 12 digits, such as 123456789012, 123456789013, and 123456789014.
4. Assign all of the devices to their appropriate contexts using the Devices widget (see Discovering, Defining, or Importing Devices), that is DIO-01 to Office 1, DIO-01-2 to Office 2, DIO-01-3 to Office 3, and DIO-01-4 to Office 4.
Create three new logical LON VAV controller devices (VAV sim) for each room/office in the floor plan. (The VAV sim device was created in Exercise 2.1.)
1. Create all of the VAV controllers using the same device type, that is VAV_sim (because the same behavior is desired from them in each room).
2. Name the new devices, for example: VAV sim 2, VAV sim 3, and VAV sim 4.
3. Enter a UID for each logical device using 12 digits, such as 123456789015, 123456789016, and 123456789017.
4. Assign all of the devices to their appropriate contexts using the Devices widget (see Discovering, Defining, or Importing Devices), that is VAV sim to Office 1, VAV sim 2 to Office 2, VAV sim 3 to Office 3, and VAV sim 4 to Office 4.
Create three new logical BACnet room sensors (Tsat-01) for each room/office in the floor plan. (The Tsat-01 device was created in Exercise 3.1.)
1. Create all of the BACnet room sensors using the same device type, that is VT7200-1H1C-3 (because the same behavior is desired from them in each room).
2. Name the new devices, for example: Tstat-01-2, Tstat-01-3, Tstat-01-4.
3. Enter a UID for each logical device such as 72007, 72006, and 72005.
4. Provision all the new logical Tsat-01 devices using the Device widget (see Provisioning, Deprovisioning, and Replacing Devices). Unlike LON devices, the provision action is necessary to register the BACnet devices to the driver.
5. Assign all of the devices to their appropriate contexts using the Devices widget (see Discovering, Defining, or Importing Devices), that is Tstat-01 to Office 1, Tstat-01-2 to Office 2, Tstat-01-3 to Office 3, and Tstat-01-4 to Office 4.
  
  In the steps that follow, you will be using the Sequencing widget to replicate the Exercise 3.1 flow.
Go to the SmartServer CMS Sequencing widget.

For SmartServer 4.0 Update 1 and prior, the Sequencing widget buttons appear as shown below.
For SmartServer 4.1 (Beta) and higher, click the Flows Info button () to display the flow menu. For SmartServer 4.1 (Beta) and higher, the Flows Info button () and the Refresh button () appear at the top of the Sequencing widget and the flow menu.

For SmartServer 4.0 Update 1 and prior, click the Open Sidebar button (). For SmartServer 4.0 Update 1 and prior, the Open Sidebar button () appears at the top of the Sequencing widget. The flow menu displays a Close Sidebar button () and the Refresh button () as shown below.

A list of flows appears similar to the following:

For more information about the flow menu, see Creating Sequence of Operations.
Click the Apply to Flow button () for the Exercise 3.1 flow to apply one or more contexts to the flow.
The Apply Contexts to Flow view appears for the selected flow (Exercise 3.1 in this class).
On the Apply Contexts to Flow view, select the four room contexts you created and assign them to this flow using the Search field. In doing so, the same flow is applied automatically without the need to create and assign it four times. The SmartServer matches the device types in the flow to the actual devices in the context you chose to apply.
Click Apply.

With the Exercise 3.1 deployed and applied to contexts, the data should start flowing into the VAV controller. You can verify that the flow is working by adding debug nodes or by checking the LCD display on the FT 6050 EVB board. For example: toggle the switches that are used to simulate the occupancy sensor to the OFF position for both sensors. The occupancy status should read OFF. Toggle one of the sensors to the ON position. The occupancy status should read ON. The LCD display should also show the average of the two temperature outputs from the room sensors.
Click the List Flows () button.

Scroll as needed to find service flow for Exercise 3.1.
Select [s]exercise 3.1[1].

To verify that the sub-flows (service flows) were created as you expected, you can view all hidden tabs in the Node-RED workspace and click the service flow that was replicated. The workspace will show you as many flows as there were replications up to the available space on the page. Flows that do not fit on the page are still created, you just cannot see them. You can click the IAP Input and Output nodes to see if your expected datapoints were selected. Or, you can click the flow menu in the Sequencing widget to see context and device associations with flows.
The replicated service flow is added to the workspace with all replicated contexts appearing in the service flow. You will need to use the scroll bar to view all of the flow in the workspace.

SmartServer 4.0 Update 1 and prior example

At this point, if you had real devices connected to the SmartServer IoT in each of the rooms, the data from the sensors in each room will start to flow to the VAV controllers in that room. Replication can be applied at any level in the context hierarchy. For example, a flow created at the the floor level could be replicated to all of the floors in the building in Mango HQ. Terrific job creating a dataflow and replicating it!

Next Step

Once you have completed the exercises in this section, continue with Class 4 - Webflows.

EnOcean Edge Portal

Class 3 - Dataflows