💬 Building a Raspberry Pi Gateway

[ej3-martin]

@MasMat I was purely wanting to find a way of testing them, to check my wiring, if they do come on during boot then this is a great way to test. It would also possibly be useful (for those of us less experienced) to have a script to run to check the wiring is correct. I think it much more useful to have them off, and blink on, and it was the wiring that confused me. I will have to see if I can investigate this further.

[mfalkvidd]

Since the pins are configurable before compilation, creating a script that uses the same settings as the compiled code would be pretty hard.

The easiest way with existing code is probably to add --my-leds-blinking-inverse to your configure command and start the gateway, as suggested by @MasMat

[Fear na Boinne]

Just trying to figure out whether there is a switch or config file option for the channel, or whether the only option to change the channel is truly to recompile?!...

Just ran the Poor Man's scanner and channels 1-98 a a garbage mess, so I need to go >> 100, but am surprised the only option there seems to be a recompile?...

[mfalkvidd]

@Fear-na-Boinne yes that is correct

[Fear na Boinne]

Sorry for the lack of response, life's been crazy... Thanks for the reply, I have since recompiled, but need to make some add'l changes to the sensor sketches, so I want to do all of that at once before I deploy to the new channel... ;)

[mfalkvidd]

@Fear-na-Boinne if you want to be able to switch your rpi gateway between the old and the new channel while testing, you could try my patch which allows configurable service name

https://github.com/mysensors/MySensors/pull/1421

Or just skip running make install so you don’t overwrite the old gateway

[Fear na Boinne]

@mfalkvidd Just a quick update...

Been extremely busy, so not a lot of time, but when I run the recompiled version (channel 109) gateway and use the existing - known good - sensor sketches with the channel 109 macro declaration included, it doesn't connect at all.
When I run the "old" version of the gateway and the same sensor sketches with channel 109 commented out, it just-works™...

More testing and such due I guess! :(

[hawtcher]

Is it possible to have the RPi4B, that is being used as the Webthings Gateway, also be the My Sensor controller once I add a NRF24L01+ module and follow steps above?

[hawtcher]

To clarify, I have the Mozilla Webthings Controller and I want to know if I can use the instructions above to make the same RPi4B to be the MySensors gateway. Any help would be greatly appreciated.

[iguaan]

In case someone has similar issue:
Running everything latest up to date.

Raspberry pi zero w with nrf24 connected directly on GPIO and domoticz. Had to have startup delay on mysgw service since domoticz wouldn't or couldn't start automatically on boot.
So i added to:
sudo nano /etc/systemd/system/mysgw.service

[Service]
ExecStartPre=/bin/sleep 10

Maybe 10 seconds is overkill, but at least it works now.
With raspberry pi 3b+ there wasn't that issue.

Also in domoticz had to use raspi ip address, when setting up mysensors LAN gateway address, instead of localhost 127.0.0.1

And myscontroller OTA updates are working too.🙂

[OldSurferDude]

I'm new to MySensors. I'm trying to build an MQTT gateway with an nRF24 radio on my RPi 3B+. I follow the instructions here,
My radio is connected with CE on physical pin 15 (GPIO22) and CSN on physical pin 24 (GPIO8) I am confident this works has I was able to run the tMRH20 examples

I follow the instructions here on this web page

I configured with:

./configure --my-gateway=mqtt --my-controller-ip-address=127.0.0.1 --my-mqtt-publish-topic-prefix=mysensors-out --my-mqtt-subscribe-topic-prefix=mysensors-in --my-mqtt-client-id=mygateway1 --my-transport=rf24

I run make but quickly error with "missing stropts.h" for which I found a fix "sudo touch /usr/include/stropts.h". I run make again after a many warnings it stops with Error 1. The last few lines:

re/Linux/drivers/core/compatibility.o build/hal/architecture/Linux/drivers/core/EthernetClient.o build/hal/architecture/Linux/drivers/core/EthernetServer.o build/hal/architecture/Linux/drivers/core/GPIO.o build/hal/architecture/Linux/drivers/core/interrupt.o build/hal/architecture/Linux/drivers/core/IPAddress.o build/hal/architecture/Linux/drivers/core/noniso.o build/hal/architecture/Linux/drivers/core/Print.o build/hal/architecture/Linux/drivers/core/SerialPort.o build/hal/architecture/Linux/drivers/core/SoftEeprom.o build/hal/architecture/Linux/drivers/core/SPIDEV.o build/hal/architecture/Linux/drivers/core/StdInOutStream.o build/hal/architecture/Linux/drivers/core/Stream.o build/examples_linux/mysgw.o build/hal/architecture/Linux/drivers/BCM/bcm2835.o build/hal/architecture/Linux/drivers/BCM/BCM.o build/hal/architecture/Linux/drivers/BCM/RPi.o build/hal/architecture/Linux/drivers/BCM/SPIBCM.o build/hal/architecture/Linux/drivers/BCM/Wire.o 
/usr/bin/ld: build/examples_linux/mysgw.o: in function `.LANCHOR1':
/home/appian3344/MySensors/./hal/architecture/Linux/drivers/core/config.h:39: multiple definition of `conf'; build/hal/architecture/Linux/drivers/core/config.o:/home/appian3344/MySensors/hal/architecture/Linux/drivers/core/config.h:39: first defined here
collect2: error: ld returned 1 exit status
make: *** [Makefile:80: bin/mysgw] Error 1

OK, what didn't I do right?

[mfalkvidd]

@OldSurferDude loos like it could be the same problem as https://github.com/mysensors/MySensors/pull/1454

In that case, applying the suggested fix manually, or using the development branch (where the fix is already included) should work.

[OldSurferDude]

@mfalkvidd I implemented your suggestion of running the development branch. This didn't work for a different reason. Below my questions is what I did.
Questions:
o Is the gateway trying to connect to a controller such as Home Assistant (HA)?
o The IP address of 127.0.0.1 implies the controller is on the same machine. Would the gateway connect to a docker containerized HA?
o What example programs will connect to the RF24 radio side of the gateway? We could start off with a simple ONoff switch and progress to that node controlling a relay on another node. (passing the request through an MQTT broker?)
o Note that I made the assumption that the gateway doesn't distinguish between wired ethernet or WiFi. Is that a good assumption?

Hardware RPi 3B+, nRF24 wired with CE to physical pin 15
OS: Raspbian GNU/Linux 11 (bullseye)

It is assumed that deleting the MySensors folder will uninstall the existing master branch. (probably a bad assumption)

1. sudo rm -r MySensors

2. git clone https://github.com/mysensors/MySensors.git --branch development

3. cd MySensors

4. ./configure --my-gateway=mqtt --my-controller-ip-address=127.0.0.1 --my-mqtt-publish-topic-prefix=mysensors-out --my-mqtt-subscribe-topic-prefix=mysensors-in --my-mqtt-client-id=mygateway1 --my-transport=rf24
   note: It is stated that the default configuration is ethernet port 5003 (./configure --my-gateway=ethernet --my-port=5003)
   It is assumed that the software doesn't distinguish between ethernet and WiFi

5. make
   many compiler warnings

6. Start gateway with
   sudo ./bin/mysgw

   which results in:

   Apr 04 12:12:37 INFO Config file /etc/mysensors.conf does not exist, creating new file.
   Apr 04 12:12:37 INFO Starting gateway...
   Apr 04 12:12:37 INFO Protocol version - 2.4.0-alpha
   Apr 04 12:12:37 INFO EEPROM file /etc/mysensors.eeprom does not exist, creating new file.
   Apr 04 12:12:37 DEBUG MCO:BGN:INIT GW,CP=RNNGL---,FQ=NA,REL=0,VER=2.4.0-alpha
   Apr 04 12:12:37 DEBUG TSF:LRT:OK
   Apr 04 12:12:37 DEBUG TSM:INIT
   Apr 04 12:12:37 DEBUG TSF:WUR:MS=0
   Apr 04 12:12:37 DEBUG TSM:INIT:TSP OK
   Apr 04 12:12:37 DEBUG TSM:INIT:GW MODE
   Apr 04 12:12:37 DEBUG TSM:READY:ID=0,PAR=0,DIS=0
   Apr 04 12:12:37 DEBUG MCO:REG:NOT NEEDED
   Apr 04 12:12:37 DEBUG MCO:BGN:STP
   Apr 04 12:12:37 DEBUG MCO:BGN:INIT OK,TSP=1

   followed by repeating:

   Apr 04 12:12:37 DEBUG GWT:RMQ:CONNECTING...
   Apr 04 12:12:37 ERROR connect: Connection refused
   Apr 04 12:12:37 ERROR failed to connect
   Apr 04 12:12:38 DEBUG !GWT:RMQ:FAIL
   Apr 04 12:12:38 DEBUG TSM:READY:NWD REQ
   Apr 04 12:12:40 DEBUG ?TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:

Assumed that it failed to connect to an MQTT broker and the my-controller-ip-address is a broker address

Changed the --my-controller-ip-address to that of my MQTT broker (It is assumed that the default MQTT port is used)
repeat step 4. with the change and step 5. and 6.
no joy same output
Restarted with step 1 with changes
no joy same output

[mfalkvidd]

@OldSurferDude said in 💬 Building a Raspberry Pi Gateway:

o Is the gateway trying to connect to a controller such as Home Assistant (HA)?

Yes, a controller or a mqtt broker.

o The IP address of 127.0.0.1 implies the controller is on the same machine. Would the gateway connect to a docker containerized HA?

I think it depends on how the docker is set up, but I don't have enough knowledge about docker to say for sure.

o What example programs will connect to the RF24 radio side of the gateway? We could start off with a simple ONoff switch and progress to that node controlling a relay on another node. (passing the request through an MQTT broker?)

You could build any of the nodes on the build page, or use https://github.com/mysensors/MySensors/tree/development/examples/MockMySensors or just create a local sensor on the gateway (if you don't need to test the nrf24 communication).

o Note that I made the assumption that the gateway doesn't distinguish between wired ethernet or WiFi. Is that a good assumption?

In the case of the Raspberry Pi gateway yes. It will use the Pi's networking, regardless of type.

[ejlane]

@OldSurferDude I don't see a password in your configure settings. Since it is saying that the connection is refused from your MQTT broker, that was the first thing that popped out to me. Do you have access control set up on the broker? I'm assuming that you have the IP address and port set up correctly.

I'm not sure if there would be a different message for other things, but when I had the access control on and messed up the username or password it would tell me connection refused or at least something very similar.

[OldSurferDude]

@ejlane @mfalkvidd

I installed an MQTT broker on another linux machine. I did not set up access control(username, password) If I install mosquitto-clients I can pub/sub to the broker without username and password. In addition, on my windows box, I installed MQTT Explorer and can monitor the broker, again without username and password. (should I specify --my-mqtt-user="" and --my-mqtt-password=""?)

I have HA on the RPi. I can access it on my windows box with http://192.168.sss.sss:8123. It requires a user password. but I don't see any parameter that refers to a password for an ethernet controller.

I'll try those programs, but first I think I need to get MySensors working.

Has anyone developed a cookbook? http://tmrh20.github.io/ has many examples of using the RF24 with combinations of RPi and Arduinos. These examples start with very basic connection to very complex. I built a for Arduino Nodes, 1 RPi master system that has been running well for two years. But I decided I need MQTT to exchange data with other devices. Alas, the MQTT client example works, but the computational load on the Arduino was too much and therefore unusable.

I'm looking to MySensors because it appears that the computational load for sensor nodes is tolerable. and it appears to have an MQTT gateway.

Cookbook? working examples?

[OldSurferDude]

@ejlane @mfalkvidd

A bit more thrashing and found that I would need an integration set up on HA.

I tried a TCP connection and HA could not connect. I then then set up a TCP configuration on MS (./configure --my-transport=rf24 --my-gateway=ethernet --my-controller-ip-address=127.0.0.1) and while it was streaming error messages, I tried to connect HA.

The I tried an HA/MS connecting using MQTT at address 127.0.0.1 but HA failed to connect (you have to install the HA MQTT integration before installing the HA MS integration.) With the same ip address on the MS configuration (./configure --my-transport=rf24 --my-gateway=mqtt --my-controller-ip-address=127.0.0.1 --my-mqtt-publish-topic-prefix=mysensors-out --my-mqtt-subscribe-topic-prefix=mysensors-in --my-mqtt-client-id=mygateway1) still failed to connect,

Some joy was found when I set the ip address to that of my MQTT broker. Neither HA nor MS reported any error messages.

OK, now onto the Arduino side of things.

OSD

[OldSurferDude]

@ejlane @mfalkvidd

On the RPi: MySensors and running HA in a docker container. Another linux box is running the MQTT broker. MQTT Explorer is running on a windows PC.

MySensors has the configuration: (./configure --my-transport=rf24 --my-gateway=mqtt --my-controller-ip-address=192,.168.sss.sss --my-mqtt-publish-topic-prefix=mysensors-out --my-mqtt-subscribe-topic-prefix=mysensors-in --my-mqtt-client-id=mygateway1) where 192,.168.sss.sss is the ip address of my MQTT broker

HA has the integration of MQTT: ip 192,.168.sss.sss; sub mysensors-out pub mysensors-in and MySensors is configure to MQTT.

HA can send to, and receive from, the broker. This was determined by the MQTT Explorer.

For my Arduino program I took the relay example with button. My Arduino is a RF-NANO so I had to add the pointers to the CE and CSN
#define MY_RF24_CS_PIN 9
#define MY_RF24_CE_PIN 10

The Arduino doesn't seem to send to the broker. See messages below messages, and MS doesn't seem to be receiving them (see below, below)

OK, why isn't the status of the switch being received by MS and passed onto the MQTT broker:

Repeating Arduino output
36190 ?TSF:MSG:SEND,11-11-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
38197 !TSM:FPAR:NO REPLY
38199 TSM:FPAR
38203 ?TSF:MSG:SEND,11-11-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
40211 !TSM:FPAR:NO REPLY
40213 TSM:FPAR
40217 ?TSF:MSG:SEND,11-11-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
42225 !TSM:FPAR:NO REPLY
42227 TSM:FPAR
42231 ?TSF:MSG:SEND,11-11-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
44239 !TSM:FPAR:FAIL
44240 TSM:FAIL:CNT=3
44242 TSM:FAIL:DIS
44244 TSF:TDI:TSL
54248 TSM:FAIL:RE-INIT
54250 TSM:INIT
54256 TSM:INIT:TSP OK
54258 TSF:SID:OK,ID=11
54260 TSM:FPAR

Output from mysgw:
sudo bin/mysgw
Apr 07 19:35:16 INFO Starting gateway...
Apr 07 19:35:16 INFO Protocol version - 2.4.0-alpha
Apr 07 19:35:16 DEBUG MCO:BGN:INIT GW,CP=RNNGL---,FQ=NA,REL=0,VER=2.4.0-alpha
Apr 07 19:35:16 DEBUG TSF:LRT:OK
Apr 07 19:35:16 DEBUG TSM:INIT
Apr 07 19:35:16 DEBUG TSF:WUR:MS=0
Apr 07 19:35:16 DEBUG TSM:INIT:TSP OK
Apr 07 19:35:16 DEBUG TSM:INIT:GW MODE
Apr 07 19:35:16 DEBUG TSM:READY:ID=0,PAR=0,DIS=0
Apr 07 19:35:16 DEBUG MCO:REG:NOT NEEDED
Apr 07 19:35:16 DEBUG MCO:BGN:STP
Apr 07 19:35:16 DEBUG MCO:BGN:INIT OK,TSP=1
Apr 07 19:35:16 DEBUG GWT:RMQ:CONNECTING...
Apr 07 19:35:16 DEBUG connected to 192.168.1.142
Apr 07 19:35:16 DEBUG GWT:RMQ:OK
Apr 07 19:35:16 DEBUG GWT:TPS:TOPIC=mysensors-out/0/255/0/0/18,MSG SENT
Apr 07 19:35:16 DEBUG TSM:READY:NWD REQ
Apr 07 19:35:17 DEBUG ?TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Apr 07 19:50:16 DEBUG TSF:SAN:OK
Apr 07 19:55:16 DEBUG TSM:READY:NWD REQ
Apr 07 19:55:17 DEBUG ?TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Apr 07 20:05:16 DEBUG TSF:SRT:OK
Apr 07 20:05:16 DEBUG TSF:SAN:OK

[ejlane]

@OldSurferDude It looks like your gateway is set to send to the wrong topic? In the debug log it looks like the gateway is sending to the 'out' topic, while you mentioned that that was output from mysensors. But in your text you said you had it set the other way, so ??

I don't have a system set up with MQTT now. I used it for a while, but got tired of having to remember to update the RPi that was doing the gateway work. I kept letting it get way out of date and that's a bit of a security issue. So I switched to a serial port connection that's powered by the HA server and plugged into a USB port from it.

But have you used the log parser? https://www.mysensors.org/build/parser

It can help a ton with understanding all the messages that you're getting.

But double check your topic settings is the only idea I have right now after seeing those log messages. It looks like the gateway sent to the 'out' topic, rather than just listening to it.

[OldSurferDude]

@ejlane

I saw the log parser. Like the last comment in log parser, my output doesn't work with it. :(

I discovered my problem, and this has always be a challenge. I had the RF24 CE connected to RPi(physical15/GPIO22). When I changed it over to (physical22/GPIO24) there were no error messages.

Thanks for noting the in-out issue. I'm dyslexic so that's always a tough one for me. I made the assumption that, like serial, TX goes to RX and RX to TX, but then, I could have screwed that up, too. Now I'm really confused! But then, it's only "two wires" to swap.

Thanks for your help.