Pi 4 ethernet gateway with NRF24L01 + PA and LNA & Domoticz

[bsmtdweller]

I'm brand new here, and hope someone will nudge me in the right direction to figure this out. Trying to set up a Raspberry Pi 4 B as an ethernet gateway for Domoticz using a NRF24L01 + PA & LNA. I have an arduino nano clone set up as the node also using the NRF24L01+, running the DallasTemperatureSensor sketch. I have followed every link and lead I can find on this forum as well as the web and either I'm missing something fundamental, or I have a legitimate technical issue. I have Domoticz installed, running and connected to the gateway at this point, but no sensors are detected. Any help is greatly appreciated!

./configure --my-rf24-ce-pin=22 --my-rf24-cs-pin=24
(I had attempted adding a lot more option details like my-controller-ip & my-port#, but it would never connect until I slimmed the options down to just specifying the ce & cs pins with everything else at default)

here are the test results: the command prompt did not show up without stopping the output - it was in some sort of ping loop.
pi@raspberrypi:~/MySensors $ sudo ./bin/mysgw
Jul 24 23:35:57 INFO Starting gateway...
Jul 24 23:35:57 INFO Protocol version - 2.4.0-alpha
Jul 24 23:35:57 DEBUG MCO:BGN:INIT GW,CP=RNNGL---,FQ=NA,REL=0,VER=2.4.0-alpha
Jul 24 23:35:57 DEBUG TSF:LRT:OK
Jul 24 23:35:57 DEBUG TSM:INIT
Jul 24 23:35:57 DEBUG TSF:WUR:MS=0
Jul 24 23:35:57 DEBUG TSM:INIT:TSP OK
Jul 24 23:35:57 DEBUG TSM:INIT:GW MODE
Jul 24 23:35:57 DEBUG TSM:READY:ID=0,PAR=0,DIS=0
Jul 24 23:35:57 DEBUG MCO:REG:NOT NEEDED
Jul 24 23:35:57 DEBUG Listening for connections on pо:5003
Jul 24 23:35:57 DEBUG MCO:BGN:STP
Jul 24 23:35:57 DEBUG MCO:BGN:INIT OK,TSP=1
Jul 24 23:35:57 DEBUG TSM:READY:NWD REQ
Jul 24 23:35:57 DEBUG ?TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Jul 24 23:36:06 DEBUG New connection from 192.168.1.147
Jul 24 23:36:06 DEBUG GWT:TSA:C=0,CONNECTED
Jul 24 23:36:06 DEBUG GWT:RFC:C=0,MSG=0;0;3;0;2;
Jul 24 23:36:06 DEBUG GWT:RFC:C=0,MSG=0;0;3;0;2;Get Version
Jul 24 23:36:06 DEBUG GWT:RFC:C=0,MSG=0;0;3;0;18;PING
Jul 24 23:36:16 DEBUG GWT:RFC:C=0,MSG=0;0;3;0;18;PING
Jul 24 23:36:26 DEBUG GWT:RFC:C=0,MSG=0;0;3;0;18;PING
Jul 24 23:36:36 DEBUG GWT:RFC:C=0,MSG=0;0;3;0;18;PING
Jul 24 23:36:46 DEBUG GWT:RFC:C=0,MSG=0;0;3;0;18;PING
Jul 24 23:36:56 DEBUG GWT:RFC:C=0,MSG=0;0;3;0;18;PING
Jul 24 23:37:06 DEBUG GWT:RFC:C=0,MSG=0;0;3;0;18;PING
^CJul 24 23:37:10 NOTICE Received SIGINT

[mfalkvidd]

Welcome to the MySensors forum @bsmtdweller

Did you start the node while mysgw was running? The log looks normal, if the node was not (re)started while the gateway was running.

The debug output for the node would be the next thing to check.

If you haven't already, see https://forum.mysensors.org/topic/666/read-this-first-it-could-save-you-a-lot-of-time/ for how to troubleshoot quickly and efficiently.

[bsmtdweller]

@mfalkvidd Thanks for your help! I wasn't utilizing the debug feature on the node (I had to uncomment it), but once I had that at my disposal, it was helpful to know the gateway radio was either not receiving, not acknowledging, or both. The radios I bought were the cheapest ones on amazon from a Chinese seller under the "brand" of "SongHe". The chip might be a valid (yet cloned) nrf24l01+ chip, but everything else about these boards screams fake-clone - not even any markings to identify the pins. Once I pulled one of my other radios out of another project and plugged it all in, voila - it works!! I was so happy! Thanks again!

[bsmtdweller]

@mfalkvidd My excitement was short lived. It did work, for a little while, but then stopped and went back into the loop of not being able to find the parent. I've gone through the troubleshooting flowchart a dozen times now and done everything I can given my limited debugging skill. I've replaced both radios, re-wired both radios, both radios have capacitors, moved them closer together/further apart, tried using separate power supplies for both radios, deleted an re-installed the MySensors dir and gateway, deleted and reinstalled Domoticz.

In the troubleshooting flowchart, one of the boxes is 'debugging the controller/gateway' - is there some nifty unix command that might shed some light on this? All I have to go on is the log of the Domoticz, and it connects up fine and doesn't have any errors.

Could this be a raspberry Pi 4 issue? Or am I simply fighting the low quality chinese cloned radios? The ones Im using work great for regular arduino (non-mysensor) sketches.

Any thoughts? I appreciate any help!

[bsmtdweller]

@bsmtdweller Update to this mystery:

1. I added a 1micro farad tantalum cap right at the RX radio, which initially did not start working, but after a few minutes, did start working. Observing the radios communicate for about an hour, the RX radio would work normally for a period, then fail - not ack sending radio, then go back to working. These observations lend real credibility that these radios are RIDICULOUSLY sensitive to their power supply - at least in this use case. (my RX radio is the nrf24l01+ PA + LNA with added 470 micro farad electrolytic cap after the regulator and now the tantalum cap at the board)

2. I dont know if it is factoring into these symptoms, but it may -- I'm not using the IRQ pin, which seems to play a vital role in the radio's sleep behavior and therefor power consumption (at least as it relates to the charging of the caps) which might explain the intermittent successes. Later I will add the IRQ wire into the mix.

3. Being honest - I have been banging my head against the wall with this and wanted to give up on it many times! But perseverance does pay off -- so any of you newcomers (like me) reading this -- dont give up!!

[bsmtdweller]

@bsmtdweller Another update:

Im beginning to think these radios suck! But alas, give them what they want and they do work! This configuration is: power comes from Raspberry Pi 4 (5v) fed into a small voltage regulator then on to the 'solution' -- premium low-ESR capacitors (1) 470 mic farad followed by a 230 mic farad right before the radio. Also, twisted wires between the reg and radio and the radio was rewired where the wires (not dupont jumper wires anymore, but 24ga silicone wires) soldered right to the board. I also crimped new ends on the communication wires connected to the Pi pins.

So far so good! Most successful concurrent transmissions so far! And no loss of communication yet! I'm thrilled! Now on to how to cram all those capacitors into the fancy enclosure I printed out - ugh.

[scalz]

yes, rf module directly connected to rpi is not ideal, as rpi doesn't provide a very clean RF power source.

[bsmtdweller]

@bsmtdweller Next challenge with this project:

I'm trying to connect another node now, the first one is humming along with no issues at all. The new node is a arduino nano running the DHT temp sketch. Smaller NRF24 is the radio and powered by an external wall wort --> into regulator --> into a 230 mic farad cap, then a .1 mic farad tantalum --> into radio. This is the second radio I've tried and the symptoms are the same - radio sends message ok, but FPAR fails - much like when I set up the gateway radio. But the 1st node is running fine on through the gw. I've cleared the nodes eeprom, and ran the sketch that @mfalkvidd posted back in 2016 to set a new node ID. Heres a little of the output from the node. Am I supposed to be posting the output this way? or is there a better/preferred way? Thanks for the help!

MCO:BGN:INIT NODE,CP=RNNNA---,FQ=16,REL=255,VER=2.3.2
26 TSM:INIT
28 TSF:WUR:MS=0
34 TSM:INIT:TSP OK
36 TSF:SID:OK,ID=42
38 TSM:FPAR
41 ?TSF:MSG:SEND,42-42-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
2050 !TSM:FPAR:NO REPLY
2052 TSM:FPAR
2056 ?TSF:MSG:SEND,42-42-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
4063 !TSM:FPAR:NO REPLY
4065 TSM:FPAR
4069 ?TSF:MSG:SEND,42-42-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
6076 !TSM:FPAR:NO REPLY
6078 TSM:FPAR
6082 ?TSF:MSG:SEND,42-42-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
8089 !TSM:FPAR:FAIL
8090 TSM:FAIL:CNT=1

[mfalkvidd]

@bsmtdweller the debug output will be rendered better if you select it and press the </> button.

Is the nrf24 antenna clear from metal on all sides? Does the gateway log show anything?

[bsmtdweller]

@mfalkvidd The radio is out in the open, I've tried holding it up and in different orientations as well as touching the rx antenna. The Domoticz log only shows the original node as connected, it doesn't see the new node. Is there a way to see more detailed output directly from the raspberry pi regarding what it happening with the gateway radio?

[mfalkvidd]

@bsmtdweller yes. See https://www.mysensors.org/build/raspberry#troubleshooting for the different options.

[bsmtdweller]

@mfalkvidd Please forgive my newness, but I'm not sure where to do what with those instructions. I've been playing with arduino for a couple years, but this is my first go round with raspberry pi. So the configuration file is accessible how? I see it is stated that the path is /etc/mysensors.conf but how do I manipulate it? Is it from the base prompt (pi@raspberrypi:~ $ )? Or from pi@raspberrypi:~/MySensors $ .

I entered all the commands for each type in the base prompt, then restarted the pi, but how can I view the log in the first option?

In the second option, this is the output:

pi@raspberrypi:~ $ tail -f /var/log/syslog
Aug  1 14:58:04 raspberrypi kernel: [  849.334318] w1_master_driver w1_bus_master1: Attaching one wire slave 00.080000000000 crc c2
Aug  1 14:58:04 raspberrypi kernel: [  849.345802] w1_master_driver w1_bus_master1: Family 0 for 00.080000000000.c2 is not registered.
Aug  1 14:58:54 raspberrypi kernel: [  899.290476] v3d fec00000.v3d: MMU error from client L2T (0) at 0x3981000, pte invalid
Aug  1 14:58:58 raspberrypi kernel: [  902.593309] w1_master_driver w1_bus_master1: Attaching one wire slave 00.880000000000 crc 4e
Aug  1 14:58:58 raspberrypi kernel: [  902.601096] w1_master_driver w1_bus_master1: Family 0 for 00.880000000000.4e is not registered.
Aug  1 14:58:58 raspberrypi systemd[1]: Starting Cleanup of Temporary Directories...
Aug  1 14:58:58 raspberrypi systemd[1]: systemd-tmpfiles-clean.service: Succeeded.
Aug  1 14:58:58 raspberrypi systemd[1]: Started Cleanup of Temporary Directories.
Aug  1 14:59:35 raspberrypi kernel: [  940.447725] w1_master_driver w1_bus_master1: Attaching one wire slave 00.480000000000 crc 84
Aug  1 14:59:35 raspberrypi kernel: [  940.455499] w1_master_driver w1_bus_master1: Family 0 for 00.480000000000.84 is not registered.
Aug  1 15:00:26 raspberrypi kernel: [  991.152408] w1_master_driver w1_bus_master1: Attaching one wire slave 00.c80000000000 crc 08
Aug  1 15:00:26 raspberrypi kernel: [  991.160465] w1_master_driver w1_bus_master1: Family 0 for 00.c80000000000.08 is not registered.

And finally, the 3rd method must not have been performed properly, because it returns:

pi@raspberrypi:~ $ cat /tmp/mysgw.pipe
cat: /tmp/mysgw.pipe: No such file or directory
pi@raspberrypi:~ $ 

Thank you for your patience!! & Willingness to help!

[mfalkvidd]

Sorry @bsmtdweller, if you're not familiar with linux, editing the file can be a bit cumbersome. Do this instead:

cd ~/MySensors
sudo systemctl stop mysgw.service
sudo ./bin/mysgw

This will let you see the gateway's log. When you're done, hold down ctrl and press C. This will stop the gateway. You can then start it again in the background with

sudo systemctl start mysgw.service

[bsmtdweller]

@mfalkvidd Thank you very much! So, this whole thing is a bit wonky! But when I stopped the gw and started it again, the new sensor was picked up! What the hell. They are both being recognized by the controller now, although there are still some strange behaviors happening. But now that I have the gateway log at my disposal, I have the ability to utilize the log parser and get a better understanding of what is going on.

Thanks again @mfalkvidd, you the man! I'm sure you will hear more from me soon!

[mfalkvidd]

Nice work @bsmtdweller
Ttyl

[bsmtdweller]

@mfalkvidd Now that I'm getting my feet wet a bit with MySensors, I would like to get your opinion on designing my first "real" project. This might be more appropriate as a new thread, but I will share my objective and follow your advice for either moving it or re-starting it somewhere else (general discussion?)

The idea is that I have a box, consider it an incubator, where I will need to maintain 80 degrees or so and 50-60% humidity. There will be 3 computer cooling fans to circulate and bring in cool air when too warm, a 110V mini space heater to add heat when its too cool, 2 servo arms to open ventilation gates based on a schedule and if it is too warm, and a solenoid valve to control a gravity drip system (supplied by a bucket of water outside the box) to maintain the humidity. There will be a 16x2 lcd keypad shield used with a simple menu to adjust setpoints and display temp humidity at the box.

If it were a standalone arduino project, I would be fine - or at least muddle through it to functionality, but I would like to be able to monitor and adjust temp/humidity setpoints through domoticz. The important thing is that the box not rely on the domoticz controller, but rather work with it and when the gateway is down, the program runs its course.

So from a design perspective, do I plan to code using MySensors and squeeze in the standalone functionality, or vice versa? Also, is it easier/better to have multiple nodes for this project, or can a single node handle what will equate to 2 sensors, and 7 relays.

Perhaps you know of some other threads that might be similar?

Thanks so much for your input!!

[mfalkvidd]

Cool project @bsmtdweller

New thread or stay in this doesn't matter much, but for people searching the forum (or their favorite search engine) in the future, I think a new thread with a subject containing "incubator" will be more clear.

I would build it with multiple nodes. Smaller chunks of code, easier wiring, easier to isolate individual parts which helps tremendously when troubleshooting, easer to reuse code for one node in case you (or someone else) want to expand the project later, easier to remove a part if you (or someone else) don't need it.

The non-mysensors code is more important, so I would start with that, but make sure all parameters that you want to be able to change are stored in variables and not hardcoded. Then add the MySensors code to update the values of the variables.

The autonomy will be hard. Normally I would prefer to have all logic in the controller, since a controller has much better ways to program and debug stuff. Not fun to write - and maintain - in C with very rudamentary debugging tools. But as you say, being reliant on having Domoticz working all the time might not be good. Having the system fail gracefully will be a challenge. A start could be to search the forum for MY_TRANSPORT_WAIT_READY_MS to see what others have done. And maybe - to make the system as reliable as possible - you should add everything to a single node instead of splitting it up into multiple nodes.