node-red-contrib-mysensors release thread

[tbowmo]

Bugfix release is out: 3.1.1

@mirodin found an issue with assigning node ID's with the controller node, which should be addressed with this release.

Basically I tried to assign nodeid 0 to nodes, and it never went out to the nodes. At the same time, the controller never got node ID 0 marked as used, as it filtered away all packages where the nodeId was 0, null or undefined, due to a missing nullcheck.

[tbowmo]

New release: 3.2.0

I added the possibility to do database lookups in the decode node, which is used to add info about sensor type to the mysensors messages in node-red. The info is gathered during node presentations

An example, from a node that report door state from the chicken coop:

topic: "mys-in/21/5/1/0/16"
payload: "1"
qos: 2
retain: false
_msgid: "ddf9aef9.9a21a"
topicRoot: "mys-in"
nodeId: 21
childSensorId: 5
messageType: 1
ack: 0
subType: 16
origin: 2
messageTypeStr: "C_SET"
subTypeStr: "V_TRIPPED"
sensorTypeStr: "S_DOOR"

As seen above, the sensorTypeStr is added.

[Maschler]

Hi,
great work. I tried to install it, without success. First I installed node-red und npm. As operation system I use openHABian v1.4.1.

This is the output:

npm install node-red-contrib-mysensors
(node:4359) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
npm WARN package.json node-red-project@0.0.1 No repository field.
npm WARN package.json node-red-project@0.0.1 No README data
npm WARN package.json node-red-project@0.0.1 No license field.
npm WARN package.json copy-descriptor@0.1.1 No README data
npm WARN package.json log-ok@0.1.1 No README data
npm WARN package.json log-utils@0.2.1 No README data
npm WARN package.json object-copy@0.1.0 No README data
npm WARN package.json static-extend@0.1.2 No README data
npm WARN package.json ansi-wrap@0.1.0 license should be a valid SPDX license expression
npm WARN package.json events@1.1.1 events is also the name of a node core module.
npm WARN package.json format@0.2.2 No license field.
npm WARN package.json jsonify@0.0.0 license should be a valid SPDX license expression
npm WARN package.json string_decoder@1.1.1 string_decoder is also the name of a node core module.
npm ERR! Error: Method Not Allowed
npm ERR!     at errorResponse (/usr/share/npm/lib/cache/add-named.js:260:10)
npm ERR!     at /usr/share/npm/lib/cache/add-named.js:203:12
npm ERR!     at saved (/usr/share/npm/node_modules/npm-registry-client/lib/get.js:167:7)
npm ERR!     at FSReqWrap.oncomplete (fs.js:135:15)
npm ERR! If you need help, you may report this *entire* log,
npm ERR! including the npm and node versions, at:
npm ERR!     <http://github.com/npm/npm/issues>

npm ERR! System Linux 4.14.52+
npm ERR! command "/usr/bin/node" "/usr/bin/npm" "install" "node-red-contrib-mysensors"
npm ERR! cwd /home/openhabian/.node-red
npm ERR! node -v v8.11.1
npm ERR! npm -v 1.4.21
npm ERR! code E405
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /home/openhabian/.node-red/npm-debug.log
npm ERR! not ok code 0

[tbowmo]

@Maschler

Which version of node-red, nodejs and npm are you using?

And what does /home/openhabian/.node-red/npm-debug.log tell you?

[Maschler]

hey @tbowmo

I found my mistake in /home/openhabian/.npm/_logs
I had to use sudo ....

But I have another Question. Do I need an Arduino in-between or can I connect the modul directly to the Raspberry pi like it is done here.

Thank you very much for your help!! :)

[tbowmo]

@Maschler

It's not necessary to have an arduino to make the gateway. You can configure the linux variant as a MQTT gateway (which I recommend), and then have mosquitto as a broker and nodered listening for mqtt messages.

https://www.mysensors.org/build/raspberry#configure

[tbowmo]

New release: 3.2.1

It's a minor update, as @pansen discovered that node IDs wasn't shown correctly in the controller view.

Also behind the scenes a couple of module dependencies have been updated to mitigate a couple of (low-risk) security issues that has been reported against some of the npm packages that was used.

[tbowmo]

Today I have released version 3.3.0

New in this release is that battery level messages is now stored in the database by myscontroller, and are also shown in the table of mysensors nodes, when you open the myscontroller node.

[tbowmo]

Version 3.3.1 is now released

A bug with the Encapsulate node was reported, where child sensor types didn't have the correct IDs. Also updated dependencies as a few where reported to have security vulnerabilities.

[magpern]

@tbowmo I feel a little confused on how to use this node-red plugin. Is this something like "a node-red plug-in framework", but you have to invent the wheel yourself kind of solution?

I have just opted for the Home assistant (hass.io) system as the hub for my home automation.
Home assistant, according to documentation, is mysensors enabled, but compared to other add-ons, like zigbee2mqtt, zwave2mqtt, the mysensors implementaton is like a child's tricycle, while the others are sports cars.
The zigbee2mqtt, you just turn on and off the IKEA light buld six times and it is added, whereas the mysensors you have to type configuration files and have limited graphical administration possibilities.

So I thought, what if the mysensors could talk to mqtt and get into home assistant that way (my gateway is serial).

I understand the node-red implementation enables you to decode the messages as you like, but do I really need to invent how it should decode it and send it to mqtt?

Are there any ready to use implementations of the plug.in?

Could it support "auto discovery"? So you just have to turn the device on, and voila, it appears in home assistant (if mqtt auto discovery is turned on)?

[tbowmo]

@magpern this module is more if you want to only use nodered for your home automation, without Hass.io or the likes.

Most of the standard controllers (domoticz, openhab, home assistant etc) are all able to handle the mysensors network directly (more or less), and handle the device registration for you.

My module for nodered is implementing a crude controller, that only handles out node id's to the mysensors network, when asked for, and then leaves everything else up to your imagination, for how to wire things together behind the scenes.

That being said, if you only need to convert your serial mysensors gateway into mqtt, I do have a small python script that I use for that particular case, as my nodered instance can't access the serial Ports from my docker swarm setup. So this script is running besides everything else. I can post it tomorrow if interested, as my pc is closed for the night now.

/ Thomas

[magpern]

@tbowmo said in node-red-contrib-mysensors release thread:

That being said, if you only need to convert your serial mysensors gateway into mqtt, I do have a small python script that I use for that particular case, as my nodered instance can't access the serial Ports from my docker swarm setup. So this script is running besides everything else. I can post it tomorrow if interested, as my pc is closed for the night now.

/ Thomas

I'd love to try the script! /Magnus

[tbowmo]

@magpern

It's more or less the script below, I removed some stuff from my utility meter, that I also handles in my script (but isn't needed for just mysensors).

#!/usr/bin/python3

import serial
import paho.mqtt.client as mqtt
import time

mysensors = serial.serial_for_url('/dev/ttyNRF24', 115200, rtscts=0, timeout=1000)
mysensors.isOpen()
mqtt_connect = False

def mqtt_reception(client, userdata, msg):
    payload = msg.payload.decode("ascii")
    topic = msg.topic.split('/')
    topic.remove('mys-out')
    mymsg = ';'.join(topic) + ';' + payload + '\n'
    print("out <- ", mymsg)
    mysensors.write(str.encode(mymsg))

def mqtt_connect(client, userdata, flags, rc):
    print('Connected')
    mqtt_connect = True
    client.subscribe('mys-out/#')

def mqtt_disconnect(client, userdata, rc):
    print(rc)
    
client = mqtt.Client('mys')
client.on_connect = mqtt_connect
client.on_message = mqtt_reception
client.on_disconnect = mqtt_disconnect

client.connect('192.168.1.64')

def handle_data(data):
    print("in -> ", data)
    s = data.split(';')
    payload = s[-1]
    s.remove(payload)
    topic = 'mys-in/' + '/'.join(s)
    if mqtt_connect: 
        client.publish(topic, payload.rstrip())

if __name__ == "__main__":
    client.loop_start()
    mys_data_str = '';
    util_data_str = '';
    while (True):
        if (mysensors.inWaiting()>0):
            char = mysensors.read(1).decode('ascii')
            mys_data_str += char
            if (char == '\n'):
                handle_data(mys_data_str)
                mys_data_str = ''
        time.sleep(0.01)

[DenisJ]

Hi there all,
I just try to install the node in my Node-Red (Home Assistant)
and I got error:

-----------------------------------------------------------
2021-04-30T07:18:09.293Z Install : node-red-contrib-mysensors 3.3.1

2021-04-30T07:18:07.624Z npm install --no-audit --no-update-notifier --no-fund --save --save-prefix=~ --production node-red-contrib-mysensors@3.3.1
2021-04-30T07:18:11.638Z [err] npm WARN deprecated @types/moment-timezone@0.5.30: This is a stub types definition. moment-timezone provides its own type definitions, so you do not need this installed.
2021-04-30T07:18:14.813Z [err] npm WARN
2021-04-30T07:18:14.813Z [err]  deprecated node-pre-gyp@0.11.0: Please upgrade to @mapbox/node-pre-gyp: the non-scoped node-pre-gyp package is deprecated and only the @mapbox scoped package will recieve updates in the future
2021-04-30T07:18:19.176Z [out] 
2021-04-30T07:18:19.176Z [out] > sqlite3@4.2.0 install /config/node-red/node_modules/sqlite3
2021-04-30T07:18:19.176Z [out] > node-pre-gyp install --fallback-to-build
2021-04-30T07:18:19.176Z [out] 
2021-04-30T07:18:19.551Z [err] node-pre-gyp
2021-04-30T07:18:19.552Z [err]  WARN Using request for node-pre-gyp https download 
2021-04-30T07:18:20.511Z [err] node-pre-gyp
2021-04-30T07:18:20.512Z [err]  WARN Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.2.0/node-v83-linux-x64.tar.gz 
2021-04-30T07:18:20.512Z [err] node-pre-gyp WARN Pre-built binaries not found for sqlite3@4.2.0 and node@14.16.1 (node-v83 ABI, musl) (falling back to source compile with node-gyp) 
2021-04-30T07:18:21.714Z [err] gyp
2021-04-30T07:18:21.715Z [err]  
2021-04-30T07:18:21.715Z [err] ERR!
2021-04-30T07:18:21.715Z [err]  
2021-04-30T07:18:21.716Z [err] build error
2021-04-30T07:18:21.716Z [err]  
2021-04-30T07:18:21.716Z [err] gyp
2021-04-30T07:18:21.717Z [err]  
2021-04-30T07:18:21.717Z [err] ERR!
2021-04-30T07:18:21.717Z [err]  
2021-04-30T07:18:21.717Z [err] stack
2021-04-30T07:18:21.717Z [err]  Error: not found: make
2021-04-30T07:18:21.717Z [err] gyp
2021-04-30T07:18:21.717Z [err]  
2021-04-30T07:18:21.717Z [err] ERR!
2021-04-30T07:18:21.717Z [err]  
2021-04-30T07:18:21.717Z [err] stack
2021-04-30T07:18:21.717Z [err]      at getNotFoundError (/usr/lib/node_modules/npm/node_modules/which/which.js:13:12)
2021-04-30T07:18:21.718Z [err] gyp
2021-04-30T07:18:21.718Z [err]  
2021-04-30T07:18:21.718Z [err] ERR!
2021-04-30T07:18:21.718Z [err]  
2021-04-30T07:18:21.718Z [err] stack
2021-04-30T07:18:21.718Z [err]      at F (/usr/lib/node_modules/npm/node_modules/which/which.js:68:19)
2021-04-30T07:18:21.718Z [err] gyp
2021-04-30T07:18:21.718Z [err]  
2021-04-30T07:18:21.718Z [err] ERR!
2021-04-30T07:18:21.718Z [err]  
2021-04-30T07:18:21.718Z [err] stack
2021-04-30T07:18:21.719Z [err]      at E (/usr/lib/node_modules/npm/node_modules/which/which.js:80:29)
2021-04-30T07:18:21.719Z [err] gyp
2021-04-30T07:18:21.719Z [err]  
2021-04-30T07:18:21.719Z [err] ERR!
2021-04-30T07:18:21.719Z [err]  
2021-04-30T07:18:21.719Z [err] stack
2021-04-30T07:18:21.719Z [err]      at /usr/lib/node_modules/npm/node_modules/which/which.js:89:16
2021-04-30T07:18:21.719Z [err] gyp
2021-04-30T07:18:21.719Z [err]  
2021-04-30T07:18:21.719Z [err] ERR!
2021-04-30T07:18:21.720Z [err]  
2021-04-30T07:18:21.720Z [err] stack
2021-04-30T07:18:21.720Z [err]      at /usr/lib/node_modules/npm/node_modules/isexe/index.js:42:5
2021-04-30T07:18:21.720Z [err] gyp
2021-04-30T07:18:21.720Z [err]  
2021-04-30T07:18:21.720Z [err] ERR!
2021-04-30T07:18:21.720Z [err]  
2021-04-30T07:18:21.720Z [err] stack
2021-04-30T07:18:21.720Z [err]      at /usr/lib/node_modules/npm/node_modules/isexe/mode.js:8:5
2021-04-30T07:18:21.720Z [err] gyp
2021-04-30T07:18:21.720Z [err]  
2021-04-30T07:18:21.721Z [err] ERR!
2021-04-30T07:18:21.721Z [err]  
2021-04-30T07:18:21.721Z [err] stack
2021-04-30T07:18:21.721Z [err]      at FSReqCallback.oncomplete (fs.js:183:21)
2021-04-30T07:18:21.721Z [err] gyp
2021-04-30T07:18:21.721Z [err]  
2021-04-30T07:18:21.721Z [err] ERR!
2021-04-30T07:18:21.721Z [err]  
2021-04-30T07:18:21.721Z [err] System
2021-04-30T07:18:21.722Z [err]  Linux 5.0.0-27-generic
2021-04-30T07:18:21.722Z [err] gyp
2021-04-30T07:18:21.722Z [err]  
2021-04-30T07:18:21.722Z [err] ERR!
2021-04-30T07:18:21.722Z [err]  
2021-04-30T07:18:21.722Z [err] command
2021-04-30T07:18:21.722Z [err]  "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "build" "--fallback-to-build" "--module=/config/node-red/node_modules/sqlite3/lib/binding/node-v83-linux-x64/node_sqlite3.node" "--module_name=node_sqlite3" "--module_path=/config/node-red/node_modules/sqlite3/lib/binding/node-v83-linux-x64" "--napi_version=7" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v83"
2021-04-30T07:18:21.722Z [err] gyp
2021-04-30T07:18:21.722Z [err]  
2021-04-30T07:18:21.722Z [err] ERR!
2021-04-30T07:18:21.723Z [err]  
2021-04-30T07:18:21.723Z [err] cwd
2021-04-30T07:18:21.723Z [err]  /config/node-red/node_modules/sqlite3
2021-04-30T07:18:21.723Z [err] gyp
2021-04-30T07:18:21.723Z [err]  
2021-04-30T07:18:21.723Z [err] ERR!
2021-04-30T07:18:21.723Z [err]  
2021-04-30T07:18:21.723Z [err] node -v
2021-04-30T07:18:21.724Z [err]  v14.16.1
2021-04-30T07:18:21.724Z [err] gyp
2021-04-30T07:18:21.724Z [err]  
2021-04-30T07:18:21.724Z [err] ERR!
2021-04-30T07:18:21.724Z [err]  
2021-04-30T07:18:21.724Z [err] node-gyp -v
2021-04-30T07:18:21.725Z [err]  v5.1.0
2021-04-30T07:18:21.725Z [err] gyp
2021-04-30T07:18:21.725Z [err]  
2021-04-30T07:18:21.725Z [err] ERR!
2021-04-30T07:18:21.725Z [err]  
2021-04-30T07:18:21.725Z [err] not ok
2021-04-30T07:18:21.725Z [err]  
2021-04-30T07:18:21.730Z [err] node-pre-gyp
2021-04-30T07:18:21.730Z [err]  ERR! build error 
2021-04-30T07:18:21.731Z [err] node-pre-gyp ERR! 
2021-04-30T07:18:21.731Z [err] stack Error: Failed to execute '/usr/bin/node /usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js build --fallback-to-build --module=/config/node-red/node_modules/sqlite3/lib/binding/node-v83-linux-x64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/config/node-red/node_modules/sqlite3/lib/binding/node-v83-linux-x64 --napi_version=7 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v83' (1)
2021-04-30T07:18:21.731Z [err] node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/config/node-red/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
2021-04-30T07:18:21.731Z [err] node-pre-gyp 
2021-04-30T07:18:21.731Z [err] ERR! stack     at ChildProcess.emit (events.js:315:20)
2021-04-30T07:18:21.731Z [err] node-pre-gyp ERR! 
2021-04-30T07:18:21.731Z [err] stack     at maybeClose (internal/child_process.js:1048:16)
2021-04-30T07:18:21.731Z [err] node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:288:5)
2021-04-30T07:18:21.731Z [err] node-pre-gyp ERR! 
2021-04-30T07:18:21.731Z [err] System Linux 5.0.0-27-generic
2021-04-30T07:18:21.731Z [err] node-pre-gyp
2021-04-30T07:18:21.731Z [err]  ERR! command "/usr/bin/node" "/config/node-red/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
2021-04-30T07:18:21.731Z [err] node-pre-gyp
2021-04-30T07:18:21.731Z [err]  ERR! cwd /config/node-red/node_modules/sqlite3
2021-04-30T07:18:21.731Z [err] node-pre-gyp
2021-04-30T07:18:21.731Z [err]  ERR! node -v v14.16.1
2021-04-30T07:18:21.732Z [err] node-pre-gyp ERR! node-pre-gyp -v v0.11.0
2021-04-30T07:18:21.732Z [err] node-pre-gyp ERR! not ok 
2021-04-30T07:18:21.732Z [out] Failed to execute '/usr/bin/node /usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js build --fallback-to-build --module=/config/node-red/node_modules/sqlite3/lib/binding/node-v83-linux-x64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/config/node-red/node_modules/sqlite3/lib/binding/node-v83-linux-x64 --napi_version=7 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v83' (1)
2021-04-30T07:18:22.158Z [err] npm ERR!
2021-04-30T07:18:22.158Z [err]  code ELIFECYCLE
2021-04-30T07:18:22.158Z [err] npm ERR! errno 1
2021-04-30T07:18:22.164Z [err] npm ERR!
2021-04-30T07:18:22.164Z [err]  sqlite3@4.2.0 install: `node-pre-gyp install --fallback-to-build`
2021-04-30T07:18:22.164Z [err] npm ERR! Exit status 1
2021-04-30T07:18:22.164Z [err] npm ERR! 
2021-04-30T07:18:22.164Z [err] npm 
2021-04-30T07:18:22.164Z [err] ERR! Failed at the sqlite3@4.2.0 install script.
2021-04-30T07:18:22.164Z [err] npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
2021-04-30T07:18:22.178Z [err] 
2021-04-30T07:18:22.178Z [err] npm ERR! A complete log of this run can be found in:
2021-04-30T07:18:22.178Z [err] npm ERR!     /root/.npm/_logs/2021-04-30T07_18_22_169Z-debug.log
2021-04-30T07:18:22.201Z rc=1

Is there still possibility to install it please ?
Thanks a lot
Denis

[tbowmo]

NPM is trying to build sqlite, because it cannot find a suitable pre-built one for your architecture, it fails because you do not have the needed tools installed for this on your machine.

Might help if you install build-essentials (sudo apt-get install build-essentials, assuming you are on a debian like install)

I currently use it on node-red 1.1.2, without problems. But I'm using plain node-red in a docker container..

[DenisJ]

@tbowmo said in node-red-contrib-mysensors release thread:

ed 1.1.2, without problem

Thanks for the answer.
I have now Node-Red 1.3.3 under Home Assistant installed.
But I must search on net how can I install the build-essentials from the docker console.

Do you know if is there some other node that I can install from the palette that maybe incluse the build-essentials please ?

Thanks again
Denis

[tbowmo]

What I did way back when I got it up and running in docker, was to jump into the container shell, and install the package manually (So you are using the same node.js version that node-red is using).

You could also (in that docker shell) do a apt install (given that it builds upon a debian based image).

something in the line of:

$ docker exec -it <id of container running node-red> /bin/bash
# now you are in docker..
$ apt install build-essentials
$ cd <to your node-red data directory>
$ npm install node-red-contrib-mysensors
$ exit

Above is taken purely from what I think is the right commands, as I have not tried working with home-assistant for a while.

[DenisJ]

@tbowmo said in node-red-contrib-mysensors release thread:

apt install build-essentials

I think I have not the apt inside of Node_red docker.

Can I install it from outside in this case.
Sorry but I'm very beginner so I need some basic info in this case.

Thanks a lot again
Denis

[electrik]

In Home Assistant you have the Node Red configuration (under supervisor). There you can add packages to be installed under system_packages: []

[DenisJ]

@electrik said in node-red-contrib-mysensors release thread:

system_packages
I have try to insert in the system_packages like this:

But it give me errore when start:

I think I'll make some translate node by my self
Thanks a lot again

Denis