💬 Building a Raspberry Pi Gateway

marceloaqno · 13 Mar 2018, 23:19

@otto001 The second patch can be applied without the first one.

otto001 · 13 Mar 2018, 23:28

@gohan :
dietpi looks interesting! did not know about this yet.
what hardware are you using? and what version of mysgw?

otto001 · 13 Mar 2018, 23:39

@marceloaqno :
THANKS! Just applied your patch.
I will report in a few days when I can see if the problem is solved...
Cheers,
Otto

14 Mar 2018, 06:06

Can someone tell me how to apply this patch?

gohan · 14 Mar 2018, 06:30

@otto001 normal rpi3 and mysgw 2.2

MasMat · 14 Mar 2018, 14:41

I need help with my Rpi mqtt-gw.
I tried to get dev-branch working but changed now to stable. All the install routines run without problems. The service just doesn't seem to work and I can't figure out why.

Rpi W, Raspbian wheezy, nRF24-PA, MQTT-gw, Mosquitto and Openhab2 on same board
Mosquitto-log:

1521010232: New connection from 127.0.0.1 on port 1883.
1521010232: New client connected from 127.0.0.1 as mygateway1 (c1, k15, u'xxxx').
1521010232: Socket error on client mygateway1, disconnecting.

syslog:

Mar 14 08:51:02 GwMqOH2 systemd[1]: Starting MySensors Gateway daemon...
Mar 14 08:51:02 GwMqOH2 systemd[1]: Started MySensors Gateway daemon.
Mar 14 08:51:03 GwMqOH2 mysgw: Starting gateway...
Mar 14 08:51:03 GwMqOH2 mysgw: Protocol version - 2.2.0
Mar 14 08:51:03 GwMqOH2 mysgw: MCO:BGN:INIT GW,CP=RNNGLSQX,VER=2.2.0
Mar 14 08:51:03 GwMqOH2 mysgw: TSF:LRT:OK
Mar 14 08:51:03 GwMqOH2 mysgw: TSM:INIT
Mar 14 08:51:03 GwMqOH2 mysgw: TSF:WUR:MS=0
Mar 14 08:51:03 GwMqOH2 mysgw: TSM:INIT:TSP OK
Mar 14 08:51:03 GwMqOH2 mysgw: TSM:INIT:GW MODE
Mar 14 08:51:03 GwMqOH2 mysgw: TSM:READY:ID=0,PAR=0,DIS=0
Mar 14 08:51:03 GwMqOH2 mysgw: MCO:REG:NOT NEEDED
Mar 14 08:51:03 GwMqOH2 mysgw: MCO:BGN:STP
Mar 14 08:51:03 GwMqOH2 mysgw: MCO:BGN:INIT OK,TSP=1
Mar 14 08:51:03 GwMqOH2 mysgw: GWT:RMQ:MQTT RECONNECT
Mar 14 08:51:03 GwMqOH2 mysgw: connected to 127.0.0.1
Mar 14 08:51:03 GwMqOH2 mysgw: GWT:RMQ:MQTT CONNECTED
Mar 14 08:51:03 GwMqOH2 mysgw: GWT:TPS:TOPIC=mysensors-out/0/255/0/0/18,MSG SENT
Mar 14 08:51:03 GwMqOH2 systemd[1]: mysgw.service: main process exited, code=killed, status=11/SEGV
Mar 14 08:51:03 GwMqOH2 systemd[1]: Unit mysgw.service entered failed state```



GW configure-line:

./configure --my-transport=nrf24 --my-gateway=mqtt --my-controller-ip-address=127.0.0.1 --my-mqtt-publish-topic-prefix=mysensors-out --my-mqtt-subscribe-topic-prefix=mysensors-in --my-mqtt-client-id=mygateway1 --my-rf24-irq-pin=15 --my-leds-err-pin=12 --my-leds-rx-pin=16 --my-leds-tx-pin=18 --my-mqtt-user=xxxx --my-mqtt-password=yyyyy --my-signing=password --my-signing-password=zzzzzz```

I have checked mosquitto has correct user-pw combination, but what should I check next? Help!

gohan · 14 Mar 2018, 15:12

@masmat said in Building a Raspberry Pi Gateway:

--my-mqtt-client-id=mygateway1

Do you have another client named mygateway1 connecting to mqtt? Try changing it to a different name just to play safe

MasMat · 14 Mar 2018, 15:33

@gohan Can I just edit mysgw.cpp and do "daemon-reload"?
I have just this one gw. And openHab2 but that has a different id.

gohan · 14 Mar 2018, 16:07

No, you need to recompile it again in case you make that change. Have you ever tried dietpi for RPI?

MasMat · 14 Mar 2018, 16:10

@gohan OK. I did that, the same socket error remains in Mosquitto just under new id.
Is my syntax correct or am I somehow having problems with mosquitto user-pw...?

Haven't tried DietPi. Would Mys+Mosquitto+Openhab2 have trouble running on it? Or why should I change to it?

gohan · 14 Mar 2018, 17:29

have you tried connecting with another mqtt client like mqttfx to see if everything is working?

marceloaqno · 14 Mar 2018, 22:15

@masmat What is the mqtt broker / version you are using?

marceloaqno · 14 Mar 2018, 22:41

@gieljnssns One way is to replace the changed files, in this case EthernetServer.cpp for the first patch and EthernetClient.cpp for the second patch.

You can download from:
https://raw.githubusercontent.com/marceloaqno/MySensors/linux-fix-stability/drivers/Linux/EthernetServer.cpp
https://raw.githubusercontent.com/marceloaqno/MySensors/linux-fix-stability/drivers/Linux/EthernetClient.cpp

Here is a step-by-step:

git clone https://github.com/mysensors/MySensors.git
cd MySensors
curl https://raw.githubusercontent.com/marceloaqno/MySensors/linux-fix-stability/drivers/Linux/EthernetServer.cpp -o drivers/Linux/EthernetServer.cpp
curl https://raw.githubusercontent.com/marceloaqno/MySensors/linux-fix-stability/drivers/Linux/EthernetClient.cpp -o drivers/Linux/EthernetClient.cpp

from here just run ./configure with your options and then make

MasMat · 15 Mar 2018, 05:07

@marceloaqno Mosquitto v1.4.14.
The broker is working, at least sending & receiving command line (_sub & _pub) messages.

15 Mar 2018, 09:54

@gohan

Can you provide some installation instructions for dietpi?
I will try this...

gohan · 15 Mar 2018, 11:23

just download the image and do the same as a Raspian image. I haven't tried mqtt gateway yet, so I don't know if it may be a gateway bug or else.

otto001 · 15 Mar 2018, 17:22

@marceloaqno :
Hi, unfortunately the latest patch did not work for me.
Still the gw stops receiving, but without high cpu usage
Fortunately I can restart the gw now using a button and this udp-scripts

Of cource I could give dietpi a try, but I would think, that a general solution would be better (for other users as well)

cheers,
Otto

MasMat · 15 Mar 2018, 18:36

@gohan I had another sd card and tried a ready image of DietPi with Mosquitto pre-installed.
I tried it without user/password and mosquitto accepting anonymous but:

/var/log/mosquitto/mosquitto.log:
1521141160: Socket error on client <unknown>, disconnecting.
1521141160: New connection from 127.0.0.1 on port 1883.
..keeps repeating...

...and sudo ./bin/mysgw -d:

mysgw: Starting gateway...
mysgw: Protocol version - 2.2.0
mysgw: MCO:BGN:INIT GW,CP=RNNGLSQX,VER=2.2.0
mysgw: TSF:LRT:OK
mysgw: TSM:INIT
mysgw: TSF:WUR:MS=0
mysgw: TSM:INIT:TSP OK
mysgw: TSM:INIT:GW MODE
mysgw: TSM:READY:ID=0,PAR=0,DIS=0
mysgw: MCO:REG:NOT NEEDED
mysgw: MCO:BGN:STP
mysgw: MCO:BGN:INIT OK,TSP=1
mysgw: GWT:RMQ:MQTT RECONNECT
mysgw: connected to 127.0.0.1
mysgw: GWT:RMQ:MQTT RECONNECT
mysgw: connected to 127.0.0.1
mysgw: GWT:RMQ:MQTT RECONNECT```

mvader · 15 Mar 2018, 19:59

1st patch is still working good for me. have had no issues since applying it.

question. is it normal/correct for the gateway to ping or spam? every 10 seconds i see the gateway announce it's version.
its not doing the full restart or anything.. just giving the 2.2.0 every 10 seconds.

gohan · 15 Mar 2018, 20:43

@masmat have you tried with ethernet gateway? If I have time I'll try the mqtt this weekend

@mvader I never noticed that.... is it the mqtt GW?

MasMat · 15 Mar 2018, 20:45

@gohan How would I interface the rest of my HA without MQTT...that's at the center of my plan really

gohan · 15 Mar 2018, 20:47

I think OH should support it, but I was just asking to make a test.

MasMat · 15 Mar 2018, 21:53

@gohan I configured:
./configure --my-transport=nrf24 --my-gateway=ethernet --my-controller-ip-address=127.0.0.1 --my-rf24-irq-pin=15 --my-leds-err-pin=12 --my-leds-rx-pin=16 --my-leds-tx-pin=18 --my-signing=password --my-signing-password=ZZZZZZ

./bin/mysgw -d:

mysgw: Starting gateway...
mysgw: Protocol version - 2.2.0
mysgw: MCO:BGN:INIT GW,CP=RNNGLSQX,VER=2.2.0
mysgw: TSF:LRT:OK
mysgw: TSM:INIT
mysgw: TSF:WUR:MS=0
mysgw: TSM:INIT:TSP OK
mysgw: TSM:INIT:GW MODE
mysgw: TSM:READY:ID=0,PAR=0,DIS=0
mysgw: MCO:REG:NOT NEEDED
mysgw: connect: Connection refused
mysgw: failed to connect
mysgw: GWT:TIN:ETH OK
mysgw: connect: Connection refused
mysgw: failed to connect
mysgw: GWT:TPS:ETH OK
mysgw: connect: Connection refused
mysgw: failed to connect
mysgw: GWT:TPS:ETH OK

The controller doesn't exist on the DietPi card yet but I'm not sure if that's causing the problem.

gohan · 15 Mar 2018, 23:10

@masmat said in Building a Raspberry Pi Gateway:

--my-controller-ip-address=127.0.0.1

remove that parameter as it is not needed for ethernet

marceloaqno · 16 Mar 2018, 00:17

@masmat I did a test with the following configuration:

Raspberry Pi 1 with Raspbian (2017-11-29-raspbian-stretch-lite)

pi@raspberrypi:~/MySensors $ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 9.1 (stretch)
Release:        9.1
Codename:       stretch

MySensors master branch - Protocol version - 2.2.0

./configure --my-transport=nrf24 --my-gateway=mqtt --my-controller-ip-address=127.0.0.1 --my-mqtt-publish-topic-prefix=mysensors-out --my-mqtt-subscribe-topic-prefix=mysensors-in --my-mqtt-client-id=m ygateway1 --my-mqtt-user=rpi --my-mqtt-password=password

Mosquitto version 1.4.10 (build date Fri, 22 Dec 2017 08:19:25 +0000) [installed from apt-get]

pi@raspberrypi:~/MySensors $ cat /etc/mosquitto/mosquitto.conf
# Place your local configuration in /etc/mosquitto/conf.d/
#
# A full description of the configuration file is at
# /usr/share/doc/mosquitto/examples/mosquitto.conf.example

pid_file /var/run/mosquitto.pid

persistence true
persistence_location /var/lib/mosquitto/

log_dest file /var/log/mosquitto/mosquitto.log

allow_anonymous false
password_file /etc/mosquitto/passwordfile

include_dir /etc/mosquitto/conf.d

Added the mosquitto user (user=rpi, password=password) with:

sudo mosquitto_passwd -c /etc/mosquitto/passwordfile rpi

It didn't show any error to connect to the MQTT broker:

pi@raspberrypi:~/MySensors $ sudo ./bin/mysgw -d
mysgw: Starting gateway...
mysgw: Protocol version - 2.2.0
mysgw: MCO:BGN:INIT GW,CP=RNNGL---,VER=2.2.0
mysgw: TSF:LRT:OK
mysgw: TSM:INIT
mysgw: TSF:WUR:MS=0
mysgw: TSM:INIT:TSP OK
mysgw: TSM:INIT:GW MODE
mysgw: TSM:READY:ID=0,PAR=0,DIS=0
mysgw: MCO:REG:NOT NEEDED
mysgw: MCO:BGN:STP
mysgw: MCO:BGN:INIT OK,TSP=1
mysgw: GWT:RMQ:MQTT RECONNECT
mysgw: connected to 127.0.0.1
mysgw: GWT:RMQ:MQTT CONNECTED
mysgw: GWT:TPS:TOPIC=mysensors-out/0/255/0/0/18,MSG SENT
mysgw: TSF:MSG:READ,5-5-0,s=255,c=3,t=24,pt=1,l=1,sg=0:1
mysgw: TSF:MSG:PINGED,ID=5,HP=1
mysgw: TSF:MSG:SEND,0-0-5-5,s=255,c=3,t=25,pt=1,l=1,sg=0,ft=0,st=OK:1
mysgw: TSF:MSG:READ,5-5-0,s=255,c=3,t=32,pt=5,l=4,sg=0:500
mysgw: GWT:TPS:TOPIC=mysensors-out/5/255/3/0/32,MSG SENT
mysgw: TSF:MSG:READ,5-5-0,s=255,c=3,t=33,pt=5,l=4,sg=0:4951
mysgw: GWT:TPS:TOPIC=mysensors-out/5/255/3/0/33,MSG SENT
mysgw: TSF:MSG:READ,5-5-0,s=1,c=1,t=37,pt=4,l=4,sg=0:36053
mysgw: GWT:TPS:TOPIC=mysensors-out/5/1/1/0/37,MSG SENT
mysgw: TSF:MSG:READ,5-5-0,s=255,c=3,t=32,pt=5,l=4,sg=0:500
mysgw: GWT:TPS:TOPIC=mysensors-out/5/255/3/0/32,MSG SENT
mysgw: TSF:MSG:READ,5-5-0,s=255,c=3,t=33,pt=5,l=4,sg=0:5000
mysgw: GWT:TPS:TOPIC=mysensors-out/5/255/3/0/33,MSG SENT
mysgw: TSF:MSG:READ,5-5-0,s=1,c=1,t=37,pt=4,l=4,sg=0:36055
mysgw: GWT:TPS:TOPIC=mysensors-out/5/1/1/0/37,MSG SENT
mysgw: TSF:MSG:READ,5-5-0,s=255,c=3,t=32,pt=5,l=4,sg=0:500
mysgw: GWT:TPS:TOPIC=mysensors-out/5/255/3/0/32,MSG SENT
mysgw: TSF:MSG:READ,5-5-0,s=255,c=3,t=33,pt=5,l=4,sg=0:5000
mysgw: GWT:TPS:TOPIC=mysensors-out/5/255/3/0/33,MSG SENT
mysgw: TSF:MSG:READ,5-5-0,s=1,c=1,t=37,pt=4,l=4,sg=0:36055
mysgw: GWT:TPS:TOPIC=mysensors-out/5/1/1/0/37,MSG SENT

mvader · 16 Mar 2018, 00:51

@gohan said in Building a Raspberry Pi Gateway:

@masmat have you tried with ethernet gateway? If I have time I'll try the mqtt this weekend

@mvader I never noticed that.... is it the mqtt GW?

I'm not using mqtt.
for debug purposes i use MYSController
every 10 seconds on the dot. it shows me the version of the gateway.

201	3/15/2018 20:47:15	RX	0 - Gateway	INTERNAL	C_INTERNAL	NO	I_VERSION	2.2.0
202	3/15/2018 20:47:25	RX	0 - Gateway	INTERNAL	C_INTERNAL	NO	I_VERSION	2.2.0
203	3/15/2018 20:47:35	RX	0 - Gateway	INTERNAL	C_INTERNAL	NO	I_VERSION	2.2.0
204	3/15/2018 20:47:45	RX	0 - Gateway	INTERNAL	C_INTERNAL	NO	I_VERSION	2.2.0
205	3/15/2018 20:47:55	RX	0 - Gateway	INTERNAL	C_INTERNAL	NO	I_VERSION	2.2.0
206	3/15/2018 20:48:05	RX	0 - Gateway	INTERNAL	C_INTERNAL	NO	I_VERSION	2.2.0

@marceloaqno any ideas about that?

gohan · 16 Mar 2018, 06:41

@mvader oh, that's normal. I though you meant something else

otto001 · 19 Mar 2018, 22:10

Yesterday I cloned the latest version on a recent raspbian (9.3) and I applied the first patch of @marceloaqno .
We will see. So long there were no problems. I will send an update in a few days...
As a last help I could switch to mqtt and do a restart after a defined amount of time (if no events are received)....

Cheers,
Otto

gohan · 20 Mar 2018, 00:21

@marceloaqno I got my gw rfm69 gateway suddenly stopped communicating with nodes. I tried everything to reinstall it but without luck. I was able to have it running on my test rpi with a complete OS reinstall. Is there anything else I need need to clean up from my install to reset every settings? (besides the mysensors folder and mysensors.dat file I already deleted)

romeo01 · 20 Mar 2018, 13:21

How can I monitor the traffic data between the raspberry and RF module (NRF24L01) on a Ethernet gateway ??

My gateway is built for Ethernet. I start the gateway with "service mysgw start"
Now, I would like to get the same data output than output debug (./bin/mysgw -d) but without stop mysgw service.
I need to use MYSController as Ethernet client at the same time.

Thanks

mfalkvidd · 20 Mar 2018, 15:24

@romeo01 see instructions at https://www.mysensors.org/build/raspberry#troubleshooting

MasMat · 20 Mar 2018, 22:35

@marceloaqno What does your mosquitto.log show?
I did a fresh install of DietPi (stretch), Mosquitto and MySensors --branch master. I left out everything and pretty much did the configure command as you did. I get the same result, but mosuiqtto log shows:

1521584708: New connection from 127.0.0.1 on port 1883.
1521584708: New client connected from 127.0.0.1 as mygateway1 (c1, k15, u'masi').
1521584733: Socket error on client mygateway1, disconnecting.
1521584778: New connection from 127.0.0.1 on port 1883.

So is this normal?
I have one sensors built that's not connecting (its log shows it just trying and trying..). I would like to user simple password and the LEDs, I will build one extra sensors just to help testing.

otto001 · 21 Mar 2018, 00:30

@MasMat : what does your mysgw.log show?

romeo01 · 21 Mar 2018, 08:53

@mfalkvidd Thanks fer info, now I can see the serial protocol as well in the syslog. Next step should be to have a logfile dedicated to mysgw.

MasMat · 21 Mar 2018, 18:11

@otto001
Syslog:

Mar 21 20:06:05 DietPi systemd[1]: Started MySensors Gateway daemon.
Mar 21 20:06:05 DietPi mysgw: Starting gateway...
Mar 21 20:06:05 DietPi mysgw: Protocol version - 2.2.0
Mar 21 20:06:05 DietPi mysgw: MCO:BGN:INIT GW,CP=RNNGL---,VER=2.2.0
Mar 21 20:06:05 DietPi mysgw: TSF:LRT:OK
Mar 21 20:06:05 DietPi mysgw: TSM:INIT
Mar 21 20:06:05 DietPi mysgw: TSF:WUR:MS=0
Mar 21 20:06:05 DietPi mysgw: TSM:INIT:TSP OK
Mar 21 20:06:05 DietPi mysgw: TSM:INIT:GW MODE
Mar 21 20:06:05 DietPi mysgw: TSM:READY:ID=0,PAR=0,DIS=0
Mar 21 20:06:05 DietPi mysgw: MCO:REG:NOT NEEDED
Mar 21 20:06:05 DietPi mysgw: MCO:BGN:STP
Mar 21 20:06:05 DietPi mysgw: MCO:BGN:INIT OK,TSP=1
Mar 21 20:06:05 DietPi mysgw: GWT:RMQ:MQTT RECONNECT
Mar 21 20:06:05 DietPi mysgw: connected to 127.0.0.1
Mar 21 20:06:05 DietPi mysgw: GWT:RMQ:MQTT CONNECTED
Mar 21 20:06:05 DietPi mysgw: GWT:TPS:TOPIC=mysensors-out/0/255/0/0/18,MSG SENT

Where is mysgw.log?

otto001 · 21 Mar 2018, 20:41

@MasMat : I just forgot, that I reconfigured rsyslogd to log mysgw to a different file. syslog is correct.

I do not use mqtt for mysensors yet, but the syslog seems good?

Richard van der Plas · 22 Mar 2018, 19:55

Was pointed here from another post, having serious issues with high CPU utilization op the gateway, is there any workaround and the moment?

top:

asks: 157 total,   2 running, 155 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.5 us, 23.0 sy,  0.0 ni, 74.3 id,  0.1 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem :   949580 total,   448732 free,   146912 used,   353936 buff/cache
KiB Swap:   102396 total,   102396 free,        0 used.   737656 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                                                         
  337 root      20   0   20492   1060    932 R 100.0  0.1  28:54.55 mysgw                                                                                                                                           
 3947 pi        20   0    8248   3324   2736 R   1.6  0.4   0:00.82 top                                                                                                                                             
  447 homeass+  20   0  340112  63684   9916 S   1.0  6.7   3:28.71 hass                                                                                                                                            
    7 root      20   0       0      0      0 S   0.3  0.0   0:04.18 rcu_sched                                                                                                                                       
  767 root      20   0  126572   8628   6544 S   0.3  0.9   0:00.95 piplight-daemon

PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"
NAME="Raspbian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
Linux raspberrypi 4.9.59-v7+ #1047 SMP Sun Oct 29 12:19:23 GMT 2017 armv7l GNU/Linux

Rebooting the pi sometimes helps, sometimes freezes the pi
seeing nothing strange in the mysensors log what so ever
running MySensors Version 2.2.0
and Home Assistant Version 0.64.0
Anyone can point me in the direction where to troubleshoot this further ?
its not a high traffic gateway (around 10 sensors connected of which 5 updating every 15 minutes)

mfalkvidd · 22 Mar 2018, 21:19

@richard-van-der-plas the best workaround so far is in the post I linked to.

marceloaqno · 22 Mar 2018, 23:53

@mvader The gateway will respond with an I_VERSION every time a controller connected to port 5003 sends an I_VERSION request.

marceloaqno · 23 Mar 2018, 00:16

@gohan Sorry to hear that, mysgw doesn't change any system files, a simple reboot and rebuild of the gateway should be enough to resume communication in a case of more serious error.

marceloaqno · 23 Mar 2018, 00:23

@masmat This is my mosquitto.log

1521680648: mosquitto version 1.4.10 (build date Fri, 22 Dec 2017 08:19:25 +0000) starting
1521680648: Config loaded from /etc/mosquitto/mosquitto.conf.
1521680648: Opening ipv4 listen socket on port 1883.
1521680648: Opening ipv6 listen socket on port 1883.
1521762617: Saving in-memory database to /var/lib/mosquitto/mosquitto.db.
1521763549: New connection from 127.0.0.1 on port 1883.
1521763549: New client connected from 127.0.0.1 as mygateway1 (c1, k15, u'rpi').

A socket error message:

1521764339: Socket error on client mygateway1, disconnecting.

shows only if I stop/exit mysgw.

MasMat · 23 Mar 2018, 18:38

@marceloaqno OK, I will look into that once I have an extra sensor feeding my gw (the one I built is a little complex and could be buggy).

Another question: I configured mysgw without "simple password" to sort out problems.
Now to reconfigure mysgw I have to run the "./configure --blah --blahblah..." command and "make", but do I also have to do "make install"? And do I have to re-start & enable the service? Should I stop it first also? Anybody got a step-by-step for this?

gohan · 23 Mar 2018, 20:17

You could also make a simple node running mock mysensors sketch that can simulate a working sensor, if you need some data running through the gateway. When I recompile the gateway I usually stop the service, uninstall the mysgw service and do the make and make install. I don't really know if it is actually required but I like to keep things clean

marceloaqno · 24 Mar 2018, 10:54

The two patches to fix ethernet stability have been merger into the development branch, and also some changes to the gateway logging. I updated the build instructions to reflect the changes.

MasMat · 24 Mar 2018, 17:43

@gohan I made a sensor that sends DHT11 temp+hum. This is the output (some of it..) from mosquitto_sub to topic mysensors-out. So I appear to have a something working. OpenHab2 is not finding any items still...

pi@DietPi:~ $ mosquitto_sub  -u XXXXXX -P YYYYYY -v -t 'mysensors-out/#'
mysensors-out/99/1/1/0/0 14.0
mysensors-out/99/0/1/0/1 46.0
mysensors-out/99/1/1/0/0 15.0
mysensors-out/99/0/1/0/1 45.0
mysensors-out/99/1/1/0/0 15.0
mysensors-out/99/0/1/0/1 45.0
mysensors-out/99/0/1/0/1 44.0
mysensors-out/99/0/1/0/1 45.0
mysensors-out/99/0/1/0/1 44.0
mysensors-out/99/1/1/0/0 15.0
mysensors-out/99/0/1/0/1 45.0
mysensors-out/99/1/1/0/0 15.0
mysensors-out/99/0/1/0/1 44.0
mysensors-out/99/0/1/0/1 45.0
mysensors-out/99/0/1/0/1 44.0
mysensors-out/99/1/1/0/0 15.0
mysensors-out/99/0/1/0/1 44.0
mysensors-out/99/1/1/0/0 15.0

Once I changed the config to include simple password, it's not showing anything anymore in mosquitto_sub
It would appear that something is missing from my setup or there's a bug in the configure-line. Can someone tell if there's a bug here somewhere:

sudo ./configure --my-transport=nrf24 --my-rf24-irq-pin=15 --my-signing=password --my-signing-password=ZZZZZZ --my-gateway=mqtt --my-controller-ip-address=127.0.0.1 --my-mqtt-user=XXXXXX --my-mqtt-password=YYYYYY --my-mqtt-publish-topic-prefix=mysensors-out --my-mqtt-subscribe-topic-prefix=mysensors-in --my-mqtt-client-id=mygateway1 --my-leds-err-pin=12 --my-leds-rx-pin=16 --my-leds-tx-pin=18

gohan · 24 Mar 2018, 20:42

Look also in node log, that's the first place to look at

MasMat · 25 Mar 2018, 07:16

@gohan I did, I will post it (but it appears to send normally). But the only things I changed was:

Rpi configure added: --my-signing=password --my-signing-password=ZZZZZZ
In the nodes code: #define MY_SIGNING_SIMPLE_PASSWD = "ZZZZZZ"

gohan · 25 Mar 2018, 12:18

I use also these parameters with the normal signing

--my-signing=software --my-signing-request-signatures --my-signing-weak_security --my-signing-debug

of course the debug parameter is there only for debug reasons

Anticimex · 25 Mar 2018, 12:36

@masmat check that the version you are using (release or beta) match your use of the simple flags, as they differ at the moment.

MasMat · 25 Mar 2018, 12:49

@anticimex Both Arduino IDE is 2.2 and Rpi is compiled with 2.2 stable.
I'm seeing !TSF:MSG:SIGN FAIL in the serial monitor so something is not compatible... I will try making password >8 characters next (currently 6).

I will also add those parameters to see if that makes a difference before reuploading arduino code

Anticimex · 25 Mar 2018, 12:52

@masmat if you add the signing debug flag and use the log parser on the homepage it should become clear what the problem is. You can also use the troubleshooting guide in the signing documentation.

MasMat · 25 Mar 2018, 13:34

@anticimex This is what's happening in the gw:

TSF:MSG:READ,36-192-15,s=69,c=0,t=81,pt=1,l=21,sg=0:221
!TSF:MSG:LEN,16!=28
TSF:MSG:READ,75-68-77,s=48,c=1,t=110,pt=1,l=25,sg=1:249
!TSF:MSG:LEN,16!=32

The parser didn't really help much: more baffled than before....and I came away from the troubleshooting guide feeling really stupid

Anticimex · 25 Mar 2018, 13:35

@masmat looks to me that your gateway is not set up for signing

gohan · 25 Mar 2018, 13:36

@anticimex I think he didn't use all the parameters for the configure, right?

Anticimex · 25 Mar 2018, 13:38

@gohan the simple flags should not need more args, but I don't use a rPi as gw so I honestly am not sure exactly the args to use. The beta documentation has clear instructions for rPi signing, but those are specifically for beta branch and probably don't map exactly to the release yet.

MasMat · 25 Mar 2018, 15:26

@gohan The whole configure-line:

sudo ./configure --my-transport=nrf24 --my-rf24-irq-pin=15 --my-signing=software --my-signing-request-signatures --my-signing-weak_security --my-signing-debug --my-signing=password --my-signing-password=ZZZZZZ --my-gateway=mqtt --my-controller-ip-address=127.0.0.1 --my-mqtt-user=XXXX --my-mqtt-password=YYYYY --my-mqtt-publish-topic-prefix=mysensors-out --my-mqtt-subscribe-topic-prefix=mysensors-in --my-mqtt-client-id=mygateway1 --my-leds-err-pin=12 --my-leds-rx-pin=16 --my-leds-tx-pin=18

If anyone can tell what's missing or if there's a typo?

Anticimex · 25 Mar 2018, 15:34

@masmat https://www.mysensors.org/apidocs/group__RaspberryPiGateway.html

Anticimex · 25 Mar 2018, 15:35

@masmat you have redundant flags. The password option require no other signing flags. Only if you select software as signing backend you need other options (and none of the simple flags)

And remember that if you use software signing and not password signing, you need to personalize the GW and/or node.

MasMat · 25 Mar 2018, 21:06

@anticimex I thought so too, but I'm grasping at straws to get the signing to work.
To clarify, is the following enough for simple signing: --my-signing-debug --my-signing=password --my-signing-password=ZZZZZZ

And the arduino code #define MY_SIGNING_SIMPLE_PASSWD = "ZZZZZZ"
Any difference where it's placed in the code? Anything else to check for?

Anticimex · 2 Apr 2018, 20:03

@masmat it needs to be defined prior to the inclusion of mysensors.h. That should be it. (on the arduino node that is).

alowhum · 31 Mar 2018, 10:16

While on the subject: what would be the flags needed for setting the gateway to only use the simple encryption but not the (simple) signing feature?

I looked in the documentation and the node commands aren't mirrored for the gateway. I was hoping for something like:

--MY_SIGNING_SIMPLE_PASSWD=flowerpot77
--MY_ENCRYPTION_SIMPLE_PASSWD=spiderman41

gohan · 31 Mar 2018, 10:19

It's actually not needed since you can still set signing as optional on gateway

Anticimex · 31 Mar 2018, 10:21

@alowhum that feature is still only available for beta and is documented here: https://www.mysensors.org/apidocs-beta/group__SecuritySettingGrpPub.html

EDIT: not yet for rPi

Pull requests are welcome. I don't have time for this at the moment.

alowhum · 31 Mar 2018, 10:47

Right. So is this correct?

NODES (arduino nano)
On all my nodes I will update them to have this code at the top:

#define MY_ENCRYPTION_SIMPLE_PASSWD spiderman41 // unfortunately Nano hardware doesn't really have enough memory for signing.
#define MY_RF24_CHANNEL 100 // in EU the default channel 76 overlaps with wifi.
#define MY_RF24_DATARATE RF24_1MBPS // slower datarate makes the network more stable?

GATEWAY (Raspberry Pi Zero W)
On my gateway I will use this configure code:

--my-security-password=spiderman41
--my-rf24-encryption-enabled
--my-signing-weak_security
--my-rf24-channel=100
--extra-cxxflags="-DMY_RF24_DATARATE=(RF24_1MBPS)"

Anticimex · 31 Mar 2018, 10:51

@alowhum you will need to enable weak security as well as that will enable both signing and encryption with signature requirements from all nodes on the gw

alowhum · 31 Mar 2018, 11:08

@Anticimex thanks!

as that will enable both signing and encryption with signature requirements from all nodes on the gw
But I don't want signing? Or do you mean that it will remove that requirement?

I only need to set that on the gateway, right?
I've also added a slower datarate, thinking that will also create a more stable connection. I am in a busy urban environment with lots of RF noise. Is that smart?

Anticimex · 31 Mar 2018, 11:12

@alowhum I thought you did not want signing: "but not the (simple) signing feature?"

alowhum · 31 Mar 2018, 11:14

Yes, I don't want signing. But what you wrote said that it wil ENABLE signing. Check your sentence. Probably a typo, but I wanted to make sure

Anticimex · 31 Mar 2018, 12:45

@alowhum the simple security flag enables signing yes.

alowhum · 31 Mar 2018, 13:14

@alowhum said in Building a Raspberry Pi Gateway:

--my-signing-weak_security

But does the "--my-signing-weak_security" enable signing?

I want to disable signing completely. What flags do I need to use when building a gateway that only uses encryption?

Anticimex · 31 Mar 2018, 13:18

@alowhum just don't use any flags mentioning signing, personalize the gw according to the documentation and pick the appropriate encryption flag.

MasMat · 2 Apr 2018, 20:03

@anticimex I've built a new node, DHT11 that sends temp&hum data. I reconfigured Rpi with this:

sudo ./configure --my-transport=nrf24 --my-rf24-irq-pin=15 --my-signing-debug --my-signing=password --my-signing-password=XXXXXX --my-gateway=mqtt --my-controller-ip-address=127.0.0.1 --my-mqtt-user=YYYY --my-mqtt-password=ZZZZZ --my-mqtt-publish-topic-prefix=mysensors-out --my-mqtt-subscribe-topic-prefix=mysensors-in --my-mqtt-client-id=mygateway1 --my-leds-err-pin=12 --my-leds-rx-pin=16 --my-leds-tx-pin=18

This is what I get in gw syslog:

Apr  2 22:53:11 DietPi mysgw: Starting gateway...
Apr  2 22:53:11 DietPi mysgw: Protocol version - 2.2.0
Apr  2 22:53:11 DietPi mysgw: MCO:BGN:INIT GW,CP=RNNGLSQX,VER=2.2.0
Apr  2 22:53:11 DietPi mysgw: !SGN:BND:PWD<8
Apr  2 22:53:11 DietPi mysgw: !SGN:INI:BND FAIL
Apr  2 22:53:11 DietPi mysgw: TSF:LRT:OK
Apr  2 22:53:11 DietPi mysgw: TSM:INIT
Apr  2 22:53:11 DietPi mysgw: TSF:WUR:MS=0
Apr  2 22:53:11 DietPi mysgw: TSM:INIT:TSP OK
Apr  2 22:53:11 DietPi mysgw: TSM:INIT:GW MODE
Apr  2 22:53:11 DietPi mysgw: TSM:READY:ID=0,PAR=0,DIS=0
Apr  2 22:53:11 DietPi mysgw: MCO:REG:NOT NEEDED
Apr  2 22:53:11 DietPi mysgw: MCO:BGN:STP
Apr  2 22:53:11 DietPi mysgw: MCO:BGN:INIT OK,TSP=1
Apr  2 22:53:11 DietPi mysgw: GWT:RMQ:MQTT RECONNECT
Apr  2 22:53:11 DietPi mysgw: connected to 127.0.0.1
Apr  2 22:53:11 DietPi mysgw: GWT:RMQ:MQTT CONNECTED
Apr  2 22:53:11 DietPi mysgw: GWT:TPS:TOPIC=mysensors-out/0/255/0/0/18,MSG SENT
Apr  2 22:53:23 DietPi mysgw: TSF:MSG:READ,99-99-255,s=255,c=3,t=7,pt=0,l=0,sg=0:
Apr  2 22:53:23 DietPi mysgw: TSF:MSG:BC
Apr  2 22:53:23 DietPi mysgw: TSF:MSG:FPAR REQ,ID=99
Apr  2 22:53:23 DietPi mysgw: TSF:PNG:SEND,TO=0
Apr  2 22:53:23 DietPi mysgw: TSF:CKU:OK
Apr  2 22:53:23 DietPi mysgw: TSF:MSG:GWL OK
Apr  2 22:53:23 DietPi mysgw: SGN:SKP:MSG CMD=3,TYPE=8
Apr  2 22:53:23 DietPi mysgw: TSF:MSG:SEND,0-0-99-99,s=255,c=3,t=8,pt=1,l=1,sg=0,ft=0,st=OK:0
Apr  2 22:53:25 DietPi mysgw: TSF:MSG:READ,99-99-0,s=255,c=3,t=24,pt=1,l=1,sg=0:1
Apr  2 22:53:25 DietPi mysgw: SGN:SKP:MSG CMD=3,TYPE=24
Apr  2 22:53:25 DietPi mysgw: TSF:MSG:PINGED,ID=99,HP=1
Apr  2 22:53:25 DietPi mysgw: SGN:SKP:MSG CMD=3,TYPE=25
Apr  2 22:53:25 DietPi mysgw: TSF:MSG:SEND,0-0-99-99,s=255,c=3,t=25,pt=1,l=1,sg=0,ft=0,st=OK:1
Apr  2 22:54:18 DietPi mysgw: TSF:MSG:READ,99-99-0,s=1,c=3,t=16,pt=0,l=0,sg=0:
Apr  2 22:54:18 DietPi mysgw: SGN:SKP:MSG CMD=3,TYPE=16
Apr  2 22:54:18 DietPi mysgw: !SGN:NCE:GEN
Apr  2 22:54:23 DietPi mysgw: TSF:MSG:READ,99-99-0,s=0,c=3,t=16,pt=0,l=0,sg=0:
Apr  2 22:54:23 DietPi mysgw: SGN:SKP:MSG CMD=3,TYPE=16
Apr  2 22:54:23 DietPi mysgw: !SGN:NCE:GEN

This is kicking my butt... I cant understand that last part about the nonce
I will add the code from the node as soon as possible.

mfalkvidd · 2 Apr 2018, 20:43

@masmat according to the log parser

mysgw: !SGN:BND:PWD<8

means the password is too short.

Anticimex · 2 Apr 2018, 20:55

@masmat and this

!SGN:INI:BND FAIL

Means the signing backend failed to initialize. So you need to make the password longer.

MasMat · 2 Apr 2018, 21:01

@mfalkvidd Made the password 10 characters. From looks of the logs, it's working now.
I cant believe I skipped the step of making the password longer... Just got too fixated on the password I came up with.

Anticimex · 2 Apr 2018, 21:03

@masmat the nonce failures is because the backend never initialized so basically any call to the backend that can fail will fail.

alowhum · 12 Apr 2018, 18:13

I just discovered these USB-to-NRF24 devices. Would it be possible to use that instead of connecting to the GPIO pins?

https://www.aliexpress.com/item/USB-wireless-serial-module-serial-to-nRF24L01-digital-communication-remote-control-acquisition-module-nRF2401/32722937957.html

mfalkvidd · 12 Apr 2018, 18:43

@alowhum not with the existing code. Is there a datasheet that describes how these modules work?

One of the reviews says "have no idea wath software to use with this" so documentation might be hard to come by.

sineverba · 26 Apr 2018, 22:16

Background: I use signing software on some nodes + whitelisting on a node (of course only PI serial is on the whitelist).

With current --development branch (I think 2.3.0alpha) I cannot anymore set my previous personalized flags on gateway RPI3.

In detail, with current master (2.2.0) I can do:

sudo mysgw --set-soft-hmac-key=F618D4[...]848992B
sudo mysgw --set-soft-serial-key=26[...]9
sudo mysgw --set-aes-key=EC7[...]CEB4

WIth development I did see only the --get[...] flags and cannot set anymore. Can you confirm?

How can I set my previous values for signing?

Anticimex · 27 Apr 2018, 00:24

@sineverba I believe this change alters how rPi port handles signing related personalisation: https://github.com/mysensors/MySensors/commit/3c0b2727a56907277d4d04c985fd72b14e4a483c

And, as usual, the documentation is a good place to start https://www.mysensors.org/apidocs-beta/group__MySigninggrpPub.html#MySigninggrphowuse

joaoabs · 30 Apr 2018, 15:37

I'm currently compiling the code for connecting the NRF24l01+ directly to the Raspberry PI. Since I'm running the controller (on this case OpenHab) on the same Raspberry PI, would it be recommended the Ethernet or the serial flavor of it?
In the case of ethernet, can I place the own machine IP (127.0.0.1)? This way, if I need to change the IP address of the PI it wouldn't affect the communication between the controller and the MySensors code, right?
--my-gateway=ethernet --my-controller-url-address=127.0.0.1

And by the way, for the signing configuration, it is mentioned in the documentation "Update the gateway config file with the generated keys/valeus", what file is that (name, path)?

I should be able to re-use the keys I have defined before, right? So I just go to step 2 (no need to generate new keys).
I'm refering to this link.

Thanks,

gohan · 30 Apr 2018, 16:13

--my-controller-url-address=127.0.0.1 is not needed for ethernet

joaoabs · 30 Apr 2018, 16:28

I found that the configuration file is /etc/mysensors.conf but " The first time you start the gateway the configuration file will be created if it does not already exist.", so that's why I wasn't finding it.

Anyway, the file states that:
"Note: The gateway must have been built with signing support to use the options below."

What flags should I include to have signing support?

OK, found it: just run ./configure -h and there is a list of options. Leaving it here just for future reference

gohan · 30 Apr 2018, 16:39

@joaoabs said in Building a Raspberry Pi Gateway:

ust run ./configure -h and there is a list of options. Leaving it here just for future reference

it is already written in the article

Anticimex · 30 Apr 2018, 16:50

@joaoabs
Release: https://www.mysensors.org/apidocs/group__RaspberryPiGateway.html
Beta/development: https://www.mysensors.org/apidocs-beta/group__RaspberryPiGateway.html

joaoabs · 30 Apr 2018, 22:27

Thanks for the replies.

I've changed the PI from a PI1 to a PI3 and was able to make it working with signing. Its running smoothly, I guess the GW is chosen :).

I'll now focus on the nodes, specially the SenseBender micros that are 2xAA battery powered and are consuming ~3% bat per day, but that will be for another post...

Thanks!
Joaoabs

Damian · 10 May 2018, 09:34

Hi, is there any way to check the current gateway operating parameters? My purpose is to check the radio channel and power level set before compilation but I would be glad if I could check other configurable parameters. I looked at Makefile.inc and found some flags set, but not the information I was looking for (does it mean that the default values are set)? Debug logs neither provide these informations.

gohan · 10 May 2018, 10:46

What radio are you using?

Damian · 10 May 2018, 20:24

@gohan nrf24, one deployment pa+lna version with external antenna and the other standard with build-in "zig-zag" antenna.

gohan · 10 May 2018, 22:26

I always set gw to max power if the radio is shielded

joaoabs · 10 May 2018, 23:46

Hi,

Is it normal to have frequently these errors in the RPI GW?

May 11 00:43:54 nettemp mysgw: RF24: Recovered from a bad interrupt trigger.
May 11 00:43:54 nettemp mysgw: RF24: Recovered from a bad interrupt trigger.
May 11 00:43:56 nettemp mysgw: RF24: Recovered from a bad interrupt trigger.
May 11 00:43:56 nettemp mysgw: RF24: Recovered from a bad interrupt trigger.
May 11 00:43:58 nettemp mysgw: RF24: Recovered from a bad interrupt trigger.
May 11 00:43:58 nettemp mysgw: RF24: Recovered from a bad interrupt trigger.
May 11 00:44:00 nettemp mysgw: RF24: Recovered from a bad interrupt trigger.
May 11 00:44:00 nettemp mysgw: RF24: Recovered from a bad interrupt trigger.

Thanks

Damian · 11 May 2018, 10:50

@gohan Well, that is not an answer to my question apparently I suppose that max power lever is the default setting according to the configure script's help.

gohan · 11 May 2018, 12:38

also look for a channel outside the WiFi range

kberck3 · 12 Jun 2018, 02:20

Openhab2 on pi3b+ using nrf24 radios and mqtt. I can follow the wiring diagram, but the commands are a bit confusing and seem to jump around randomly. Do I put all the --my-blah-blah together after the ./configure to build a file? Should I do development version? Do the nfr improvements go together with everything else?

gohan · 12 Jun 2018, 06:48

@kberck3 said in Building a Raspberry Pi Gateway:

Do I put all the --my-blah-blah together after the ./configure to build a file?

From the article "Note: All options must be added to the same line, after ./configure"

Leave the development version for now, the stable works fine.

mfalkvidd · 12 Jun 2018, 07:34

@gohan to be fair, I added that note this morning so kberck3 might not have seen it

kberck3 · 12 Jun 2018, 16:09

Appreciate the reply mfalkvidd and gohan! Service is running, will report back if I run into other issues.

Mark Swift · 16 Jun 2018, 08:44

Hi Guys,

Is anyone here using the Pi with OTA updates? I'm finding them painfully slow (1-2 stream messages every few seconds). It's taking 1 hour per OTA update compared with 2-3 minutes when using the Arduino gateway!

gohan · 17 Jun 2018, 07:32

I did some tests on pi3 and I found slow ota too. Tekka has an open issue on github about this.

mirodin · 18 Jun 2018, 11:24

Hey guys,

I tried to setup my Raspberry running Node-RED and serial gateway but I got stuck halfway. I followed the serial gateway compilation above and got it up and running, however using the serial nodes an Node-RED I could not manage to get them to connect tho the gateway.

First I tried to connect to /dev/ttyMySensorsGateway which did not work due to permissions (owned by root). When checking out /dev/ttyMySensorsGateway I found it links to /dev/pts/1
which belongs to root and the tty group. But I could not connect to this as the tty group only has write privileges. I think I miss some crucial point connecting those two together so I hope somebody can save the day by pointing me in the right direction.

Thanks in advance,
mirodin

💬 Building a Raspberry Pi Gateway

Suggested Topics

Missing Notification Settings
Bug Reports • 27 Mar 2014, 09:59 • edoardoo 28 Mar 2014, 18:20

💬 Ago Control
Announcements • 11 Sept 2016, 21:06 • hek 24 May 2023, 19:27

💬 Building a Orange Pi Gateway
Announcements • 10 Jan 2017, 06:51 • hek 30 Nov 2022, 20:23

💬 Building a WiFi Gateway using ESP8266
Announcements • 11 Sept 2016, 21:13 • hek 5 Mar 2023, 11:18

💬 Battery Powered Sensors
Announcements • battery • 11 Sept 2016, 21:13 • hek 15 Oct 2023, 12:55

💬 Building a wired RS485 sensor network
Announcements • 14 Sept 2016, 18:30 • hek 24 Jul 2024, 05:25

0
Online

11.5k
Users

11.1k
Topics

112.8k
Posts

💬 Building a Raspberry Pi Gateway

Suggested Topics

Missing Notification Settings Bug Reports • 27 Mar 2014, 09:59 • edoardoo 28 Mar 2014, 18:20

💬 Ago Control Announcements • 11 Sept 2016, 21:06 • hek 24 May 2023, 19:27

💬 Building a Orange Pi Gateway Announcements • 10 Jan 2017, 06:51 • hek 30 Nov 2022, 20:23

💬 Building a WiFi Gateway using ESP8266 Announcements • 11 Sept 2016, 21:13 • hek 5 Mar 2023, 11:18

💬 Battery Powered Sensors Announcements • battery • 11 Sept 2016, 21:13 • hek 15 Oct 2023, 12:55

💬 Building a wired RS485 sensor network Announcements • 14 Sept 2016, 18:30 • hek 24 Jul 2024, 05:25

0Online

11.5kUsers

11.1kTopics

112.8kPosts

Missing Notification Settings
Bug Reports • 27 Mar 2014, 09:59 • edoardoo 28 Mar 2014, 18:20

💬 Ago Control
Announcements • 11 Sept 2016, 21:06 • hek 24 May 2023, 19:27

💬 Building a Orange Pi Gateway
Announcements • 10 Jan 2017, 06:51 • hek 30 Nov 2022, 20:23

💬 Building a WiFi Gateway using ESP8266
Announcements • 11 Sept 2016, 21:13 • hek 5 Mar 2023, 11:18

💬 Battery Powered Sensors
Announcements • battery • 11 Sept 2016, 21:13 • hek 15 Oct 2023, 12:55

💬 Building a wired RS485 sensor network
Announcements • 14 Sept 2016, 18:30 • hek 24 Jul 2024, 05:25

0
Online

11.5k
Users

11.1k
Topics

112.8k
Posts