node-red-contrib-mysensors release thread

[tbowmo]

An almost complete rewrite of the node-red-contrib-mysensors has been released.

What have been done this time is:

• Rewritten in typescript
• a controller node is added, allowing node-red to hand out node id's to a mysensors network.

The move to typescript should enable more test of the software, as I now have a compile time check, and also implemented the first unit tests of the code.

Please note that the mysencdec node has now been removed.. If you still use it, then you should remove it before upgrading the package, or node-red might stop the flows, as it is missing nodes.

Please use the issue tracker on github, if you find some undocumented "features"

[tbowmo]

Btw. the only really breaking change from the 1.x series from a user perspective, is that the mysdecenc is gone.

All other nodes should perform as the old version.

[tbowmo]

Bugger.. I found an issue myself that sneaked past my typescript compile checks..

So expect an update later today..

[tbowmo]

Release 2.0.1 is out, fixing the bugs that I discovered earlier.. Thought that I had everything covered, and even did some regression testing in a dummy node-red install.. But missed some important details, that unfortunately can't be covered by the testframework at the moment (perhaps because my lack of understanding of mocha / chai)

[tbowmo]

Released 2.1.0 today,

Improved debug node, where I ported the parser from https://mysensors.org/build/parser.

The controller node now also logs the last repeater a given node has used, to reach the gateway.

Some minor details here and ther in the ui have been updated as well.

Linting of the typescript files have been added using tslint.

[tbowmo]

Released 2.2.0 today

the mysdecode node now also decodes messageType and sensorType to string values, in the node message it emits.

The emitted message now contains the following propperties:

topicRoot: string; // Only used if it is decoding from a mqtt topic
nodeId: number;
childSensorId: number;
messageType: mysensor_command;
messageTypeStr: string;
ack: 0|1;
subType: mysensor_data| mysensor_internal| mysensor_sensor| mysensor_stream;
subTypeStr: string;
origin: MsgOrigin; // This is an indicator for where the message was received from (mqtt, serial or "undefined")

Also did a lot of cleanup, set stricter rules in the typescript compiler etc. to avoid common mistakes.

[tbowmo]

Released 3.0.0

Breaking change is in the controller node, which now use a migration feature in the sqlite lilbrary that I use. It makes it easier to update database schema later on, as I can now create "patches" for the schema, that is included in the library. The database needs to be re-created from scratch, as the migration part has some extra tables to keep track of the schema version.

Also documentation has been received an overhaul, correcting a few spelling errors, and a lot of formatting.

[Thucar]

Awesome work! I was trying to create a bridge between a MQTT gateway and a Serial gateway (over TCP) so I could use firmware upload function of the MySController. However for some reason it seems to only work one way. I can send commands from the MySController to MQTTGateway but nothing is being received on the MySController side when data is sent from MQTT

This is my flow, would be awesome if someone would spot the issue...

[{"id":"e6a66581.c16ec8","type":"mysdecode","z":"229ed636.c3ff2a","name":"","mqtt":false,"x":450,"y":460,"wires":[["58ac0291.04974c","a3ba801.a63be8"]]},{"id":"eb5b52a1.29566","type":"tcp in","z":"229ed636.c3ff2a","name":"","server":"server","host":"","port":"5003","datamode":"stream","datatype":"utf8","newline":"\\n","topic":"","base64":false,"x":270,"y":460,"wires":[["e6a66581.c16ec8","ab0f8982.8a9588","78954899.688c28"]]},{"id":"58ac0291.04974c","type":"debug","z":"229ed636.c3ff2a","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":510,"y":560,"wires":[]},{"id":"a3ba801.a63be8","type":"mysencode","z":"229ed636.c3ff2a","name":"","mqtt":true,"mqtttopic":"myGreenhouse-in","x":1040,"y":200,"wires":[["8d64662e.fbada8","595543a3.f537cc"]]},{"id":"ab0f8982.8a9588","type":"globalGetSet","z":"229ed636.c3ff2a","name":"Save Session","topic":"","context":"msg","variable":"_session","outContext":"global","outVar":"_session","x":440,"y":420,"wires":[[]]},{"id":"78954899.688c28","type":"debug","z":"229ed636.c3ff2a","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":500,"y":720,"wires":[]},{"id":"cdf94190.9fb8e","type":"myscontroler","z":"229ed636.c3ff2a","database":"5fdbb924.88fb48","name":"","handleid":true,"x":770,"y":200,"wires":[["a3ba801.a63be8"]]},{"id":"8d64662e.fbada8","type":"mqtt out","z":"229ed636.c3ff2a","name":"","topic":"","qos":"","retain":"","broker":"1920bc44.5c3304","x":1270,"y":200,"wires":[]},{"id":"595543a3.f537cc","type":"debug","z":"229ed636.c3ff2a","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":1270,"y":260,"wires":[]},{"id":"c800320f.2a2b","type":"mysdecode","z":"229ed636.c3ff2a","name":"","mqtt":true,"x":560,"y":200,"wires":[["cdf94190.9fb8e","aa22f275.cc8ec"]]},{"id":"2222f84e.56c4e8","type":"mqtt in","z":"229ed636.c3ff2a","name":"","topic":"myGreenhouse-out/#","qos":"2","broker":"1920bc44.5c3304","x":320,"y":200,"wires":[["41588b7e.b61114","c800320f.2a2b"]]},{"id":"aa22f275.cc8ec","type":"switch","z":"229ed636.c3ff2a","name":"","property":"nodeId","propertyType":"msg","rules":[{"t":"eq","v":"203","vt":"num"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":570,"y":360,"wires":[[],["ea8a0d53.8140b"]]},{"id":"41588b7e.b61114","type":"debug","z":"229ed636.c3ff2a","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":530,"y":260,"wires":[]},{"id":"ea8a0d53.8140b","type":"mysencode","z":"229ed636.c3ff2a","name":"","mqtt":false,"mqtttopic":"","x":870,"y":420,"wires":[["2036eda1.dd8bf2","ad5ca8e4.1c4f68"]]},{"id":"2036eda1.dd8bf2","type":"debug","z":"229ed636.c3ff2a","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":1030,"y":460,"wires":[]},{"id":"ad5ca8e4.1c4f68","type":"globalGetSet","z":"229ed636.c3ff2a","name":"Load Session","topic":"","context":"global","variable":"_session","outContext":"msg","outVar":"_session","x":1060,"y":420,"wires":[["cb6d084b.432238","f99fd97d.5f73c8"]]},{"id":"cb6d084b.432238","type":"tcp out","z":"229ed636.c3ff2a","host":"localhost","port":"5003","beserver":"reply","base64":true,"end":false,"name":"","x":1250,"y":420,"wires":[]},{"id":"f99fd97d.5f73c8","type":"debug","z":"229ed636.c3ff2a","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":1240,"y":480,"wires":[]},{"id":"5fdbb924.88fb48","type":"mysensorsdb","z":"","name":"Device DB","file":"sqlite"},{"id":"1920bc44.5c3304","type":"mqtt-broker","z":"","name":"LocalNew","broker":"localhost","port":"8883","tls":"","clientid":"Node-RedHome","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"willTopic":"","willQos":"0","willRetain":"false","willPayload":"","birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":""}]```

[tbowmo]

@Thucar

The MysController node doesn't pass any messages through it, it only emits messages if it receives a node ID request.

Also, (depending on your version) then it is not needed to pass messages through mysdecode / mysencode before / after the myscontroller node. It understands mqtt / serial protocol directly. (this has been a feature since 2.2.0)

/ Thomas

[tbowmo]

New release : 3.1.0

Added features:

Controller now supports the following messages:

• I_TIME is replied with a unix timestamp corrected with local timezone offset
• I_CONFIG, responds with Metric / Imperial setting

Minor documentation updates to other nodes as well.

[tbowmo]

Bugfix release is out: 3.1.1

@mirodin found an issue with assigning node ID's with the controller node, which should be addressed with this release.

Basically I tried to assign nodeid 0 to nodes, and it never went out to the nodes. At the same time, the controller never got node ID 0 marked as used, as it filtered away all packages where the nodeId was 0, null or undefined, due to a missing nullcheck.

[tbowmo]

New release: 3.2.0

I added the possibility to do database lookups in the decode node, which is used to add info about sensor type to the mysensors messages in node-red. The info is gathered during node presentations

An example, from a node that report door state from the chicken coop:

topic: "mys-in/21/5/1/0/16"
payload: "1"
qos: 2
retain: false
_msgid: "ddf9aef9.9a21a"
topicRoot: "mys-in"
nodeId: 21
childSensorId: 5
messageType: 1
ack: 0
subType: 16
origin: 2
messageTypeStr: "C_SET"
subTypeStr: "V_TRIPPED"
sensorTypeStr: "S_DOOR"

As seen above, the sensorTypeStr is added.

[Maschler]

Hi,
great work. I tried to install it, without success. First I installed node-red und npm. As operation system I use openHABian v1.4.1.

This is the output:

npm install node-red-contrib-mysensors
(node:4359) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
npm WARN package.json node-red-project@0.0.1 No repository field.
npm WARN package.json node-red-project@0.0.1 No README data
npm WARN package.json node-red-project@0.0.1 No license field.
npm WARN package.json copy-descriptor@0.1.1 No README data
npm WARN package.json log-ok@0.1.1 No README data
npm WARN package.json log-utils@0.2.1 No README data
npm WARN package.json object-copy@0.1.0 No README data
npm WARN package.json static-extend@0.1.2 No README data
npm WARN package.json ansi-wrap@0.1.0 license should be a valid SPDX license expression
npm WARN package.json events@1.1.1 events is also the name of a node core module.
npm WARN package.json format@0.2.2 No license field.
npm WARN package.json jsonify@0.0.0 license should be a valid SPDX license expression
npm WARN package.json string_decoder@1.1.1 string_decoder is also the name of a node core module.
npm ERR! Error: Method Not Allowed
npm ERR!     at errorResponse (/usr/share/npm/lib/cache/add-named.js:260:10)
npm ERR!     at /usr/share/npm/lib/cache/add-named.js:203:12
npm ERR!     at saved (/usr/share/npm/node_modules/npm-registry-client/lib/get.js:167:7)
npm ERR!     at FSReqWrap.oncomplete (fs.js:135:15)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR!     <http://github.com/npm/npm/issues>

npm ERR! System Linux 4.14.52+
npm ERR! command "/usr/bin/node" "/usr/bin/npm" "install" "node-red-contrib-mysensors"
npm ERR! cwd /home/openhabian/.node-red
npm ERR! node -v v8.11.1
npm ERR! npm -v 1.4.21
npm ERR! code E405
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /home/openhabian/.node-red/npm-debug.log
npm ERR! not ok code 0

[tbowmo]

@Maschler

Which version of node-red, nodejs and npm are you using?

And what does /home/openhabian/.node-red/npm-debug.log tell you?

[Maschler]

hey @tbowmo

I found my mistake in /home/openhabian/.npm/_logs
I had to use sudo ....

But I have another Question. Do I need an Arduino in-between or can I connect the modul directly to the Raspberry pi like it is done here.

Thank you very much for your help!!

[tbowmo]

@Maschler

It's not necessary to have an arduino to make the gateway. You can configure the linux variant as a MQTT gateway (which I recommend), and then have mosquitto as a broker and nodered listening for mqtt messages.

https://www.mysensors.org/build/raspberry#configure

[tbowmo]

New release: 3.2.1

It's a minor update, as @pansen discovered that node IDs wasn't shown correctly in the controller view.

Also behind the scenes a couple of module dependencies have been updated to mitigate a couple of (low-risk) security issues that has been reported against some of the npm packages that was used.

[tbowmo]

Today I have released version 3.3.0

New in this release is that battery level messages is now stored in the database by myscontroller, and are also shown in the table of mysensors nodes, when you open the myscontroller node.

[tbowmo]

Version 3.3.1 is now released

A bug with the Encapsulate node was reported, where child sensor types didn't have the correct IDs. Also updated dependencies as a few where reported to have security vulnerabilities.

[magpern]

@tbowmo I feel a little confused on how to use this node-red plugin. Is this something like "a node-red plug-in framework", but you have to invent the wheel yourself kind of solution?

I have just opted for the Home assistant (hass.io) system as the hub for my home automation.
Home assistant, according to documentation, is mysensors enabled, but compared to other add-ons, like zigbee2mqtt, zwave2mqtt, the mysensors implementaton is like a child's tricycle, while the others are sports cars.
The zigbee2mqtt, you just turn on and off the IKEA light buld six times and it is added, whereas the mysensors you have to type configuration files and have limited graphical administration possibilities.

So I thought, what if the mysensors could talk to mqtt and get into home assistant that way (my gateway is serial).

I understand the node-red implementation enables you to decode the messages as you like, but do I really need to invent how it should decode it and send it to mqtt?

Are there any ready to use implementations of the plug.in?

Could it support "auto discovery"? So you just have to turn the device on, and voila, it appears in home assistant (if mqtt auto discovery is turned on)?

[tbowmo]

@magpern this module is more if you want to only use nodered for your home automation, without Hass.io or the likes.

Most of the standard controllers (domoticz, openhab, home assistant etc) are all able to handle the mysensors network directly (more or less), and handle the device registration for you.

My module for nodered is implementing a crude controller, that only handles out node id's to the mysensors network, when asked for, and then leaves everything else up to your imagination, for how to wire things together behind the scenes.

That being said, if you only need to convert your serial mysensors gateway into mqtt, I do have a small python script that I use for that particular case, as my nodered instance can't access the serial Ports from my docker swarm setup. So this script is running besides everything else. I can post it tomorrow if interested, as my pc is closed for the night now.

/ Thomas

[magpern]

@tbowmo said in node-red-contrib-mysensors release thread:

That being said, if you only need to convert your serial mysensors gateway into mqtt, I do have a small python script that I use for that particular case, as my nodered instance can't access the serial Ports from my docker swarm setup. So this script is running besides everything else. I can post it tomorrow if interested, as my pc is closed for the night now.

/ Thomas

I'd love to try the script! /Magnus

[tbowmo]

@magpern

It's more or less the script below, I removed some stuff from my utility meter, that I also handles in my script (but isn't needed for just mysensors).

#!/usr/bin/python3

import serial
import paho.mqtt.client as mqtt
import time

mysensors = serial.serial_for_url('/dev/ttyNRF24', 115200, rtscts=0, timeout=1000)
mysensors.isOpen()
mqtt_connect = False

def mqtt_reception(client, userdata, msg):
    payload = msg.payload.decode("ascii")
    topic = msg.topic.split('/')
    topic.remove('mys-out')
    mymsg = ';'.join(topic) + ';' + payload + '\n'
    print("out <- ", mymsg)
    mysensors.write(str.encode(mymsg))

def mqtt_connect(client, userdata, flags, rc):
    print('Connected')
    mqtt_connect = True
    client.subscribe('mys-out/#')

def mqtt_disconnect(client, userdata, rc):
    print(rc)
    
client = mqtt.Client('mys')
client.on_connect = mqtt_connect
client.on_message = mqtt_reception
client.on_disconnect = mqtt_disconnect

client.connect('192.168.1.64')

def handle_data(data):
    print("in -> ", data)
    s = data.split(';')
    payload = s[-1]
    s.remove(payload)
    topic = 'mys-in/' + '/'.join(s)
    if mqtt_connect: 
        client.publish(topic, payload.rstrip())

if __name__ == "__main__":
    client.loop_start()
    mys_data_str = '';
    util_data_str = '';
    while (True):
        if (mysensors.inWaiting()>0):
            char = mysensors.read(1).decode('ascii')
            mys_data_str += char
            if (char == '\n'):
                handle_data(mys_data_str)
                mys_data_str = ''
        time.sleep(0.01)