[solved] RFM69 based nodes unable to report Lib Version

[chrille]

@scalz said:

• List of changes, some notes, and current status : https://docs.google.com/spreadsheets/d/191NpTBogLPijYxS2V_oHZnVlcW4B65J1kXAqYrr4qeE/edit#gid=884074439
• https://github.com/scalz/Mysensors

Be a little bit more patient, PR should go soon now as you can see from the current status ;)

It's great to see work being put into the RFM69 driver. I have tried to install your code and updated my gateway and a sensor node

On the sensor node I added

sendSignalStrength(1);
sendRadioTxLevel(1);

and in the debug window I see

6899 TSF:MSG:SEND,1-1-0-0,s=255,c=3,t=29,pt=1,l=1,sg=0,ft=0,st=OK:51
7035 !TSF:MSG:SEND,1-1-0-0,s=255,c=3,t=30,pt=1,l=1,sg=0,ft=0,st=NACK:0

Message type 30 always fails (st=NACK) - is this expected?

Gateway is ESP8266 and node is Anarduino (328p+RFM69CW)

[jpaulin]

@korttoma

I get exactly the same bug. I found a way to solve it, but not sure if it's the root cause.
I use RFM69W and the latest release from the Development Branch.

From your message dump:

TSF:MSG:SEND,101-101-0-0,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
!TSF:MSG:SEND,101-101-0-0,s=255,c=0,t=17,pt=0,l=10,sg=0,ft=0,st=NACK:2.0.1-beta
TSF:MSG:SEND,101-101-0-0,s=255,c=3,t=6,pt=1,l=1,sg=0,ft=1,st=OK:0

The Internal Presentation Message with Lib Version as payload always fails. (!TSF: c=0, t=17, st=NACK:2.0.1-beta)

The message is sent immediately after the previously sent internal message (TSF: c=3, t=15, st=OK:0100) (t=15 => I_REQUEST_SIGNING).
The Gateway responds the Signing Preference Message to the node exactly at the same time the node tries to send the Lib Version Presentation Message to the Gateway. Seems that won't work. There's no buffering?
I added a 1s delay for test purpose in MySensorsCore.cpp to give time to finish the response from the gateway before sending the Lib Version Presentation Message.

New message dump from my test-node after changes:

TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
TSF:MSG:READ,0-0-3,s=255,c=3,t=15,pt=6,l=2,sg=0:0100
TSF:MSG:SEND,3-3-0-0,s=255,c=0,t=17,pt=0,l=10,sg=0,ft=0,st=OK:2.0.1-beta
TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=6,pt=1,l=1,sg=0,ft=0,st=OK:0

Now a new message appears in the node, the response message from the Gateway to the Signing Preference Message, and the presentation of the Lib Version works as expected.

TSF:MSG:READ,0-0-3,s=255,c=3,t=15,pt=6,l=2,sg=0:0100 
TSF:MSG:SEND,3-3-0-0,s=255,c=0,t=17,pt=0,l=10,sg=0,ft=0,st=OK:2.0.1-beta

In MySensorsCore.cpp at line 216 I added a wait(1000); to mitigate the issue.

	// Send signing preferences for this node to the GW
	signerPresentation(_msgTmp, GATEWAY_ADDRESS);

wait(1000);

		// Send presentation for this radio node
	#if defined(MY_REPEATER_FEATURE)
		(void)present(NODE_SENSOR_ID, S_ARDUINO_REPEATER_NODE);
	#else
		(void)present(NODE_SENSOR_ID, S_ARDUINO_NODE);
	#endif

I guess the final patch would look different and would need to be looked into by @Anticimex or @hek. :smiley:

[Anticimex]

@jpaulin the signing backend is already waiting for the GW to send a message. So if it is not waiting long enough I believe the existing delay should be increased instead: https://github.com/mysensors/MySensors/blob/development/core/MySigning.cpp#L158

[tekka]

@Anticimex This is a conditional wait(), i.e. only when signing is enabled: https://github.com/mysensors/MySensors/blob/development/core/MySigning.cpp#L154-L160

Since the GW always replies to signing preferences, but the node only waits if signing is enabled - this message will eventually collide with the following lib version message, as seen above.

I suggest removing the surrounding #ifdef.

[Anticimex]

@tekka true. Moving the existing delay outside the preprocessor condition should help.

[Anticimex]

@jpaulin could you please file a pull request with the delay moved outside the preprocessor condition (as you have the rig to verify the change works)?

[jpaulin]

@Anticimex
I don't know how to file a pull request, so I put the test results here.

Modified as follows to remove the preprocessor condition at: https://github.com/mysensors/MySensors/blob/development/core/MySigning.cpp#L154-L160

// #if defined(MY_SIGNING_FEATURE)
    // If we do support signing, wait for the gateway to tell us how it prefer us to transmit our messages
    if (destination == GATEWAY_ADDRESS) {
	    SIGN_DEBUG(PSTR("Waiting for GW to send signing preferences...\n"));
	    wait(2000, C_INTERNAL, I_SIGNING_PRESENTATION);
}
// #endif

solves the issue.

At the same time the internal message received from the gateway seems to be erroneously transferred to the receive() function in a sketch. Adding to the sketch

void receive(const MyMessage &message) {
    Serial.println("something came in");
}

gets the message dump:

2310 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
2332 TSF:MSG:READ,0-0-3,s=255,c=3,t=15,pt=6,l=2,sg=0:0100
something came in
2409 TSF:MSG:SEND,3-3-0-0,s=255,c=0,t=17,pt=0,l=10,sg=0,ft=0,st=OK:2.0.1-beta
2496 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=6,pt=1,l=1,sg=0,ft=0,st=OK:0

To solve this I made the following change to _processInternalMessages(void) in MySensorsCore.cpp.
Line https://github.com/mysensors/MySensors/blob/development/core/MySensorsCore.cpp#L407 is replaced with:

	else if (type == I_SIGNING_PRESENTATION) {
	}
	else return false;

The message dump now looks like this:

2250 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
2269 TSF:MSG:READ,0-0-3,s=255,c=3,t=15,pt=6,l=2,sg=0:0100
2331 TSF:MSG:SEND,3-3-0-0,s=255,c=0,t=17,pt=0,l=10,sg=0,ft=0,st=OK:2.0.1-beta
2441 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=6,pt=1,l=1,sg=0,ft=0,st=OK:0

At the API description page https://www.mysensors.org/download/serial_api_20 seems to be another error.
For the internal message t=15 the name description

I_REQUEST_SIGNING 	15 	Used between sensors when initialting signing.

should be changed to

I_SIGNING_PRESENTATION 	15 	Provides signing related preferences.

This I think is relevant both for the master and the development branch.

[Anticimex]

@jpaulin thanks for the updates. I will have a look at making a pr when I get opportunity. @hek does the documentation issue sound familiar? ;)

[Anticimex]

@jpaulin I have made a pull request. My solution differs slightly from your as the I_SIGNING_PRESENTATION should never reach the _processInternalMessages function. I do however not have the ability to test so I would appreciate if you could test the PR for me?
Thanks for finding and pointing out the flaws! :D

[jpaulin]

@Anticimex
I updated my node and GW with your pull request and made some basic tests and it seems to work ok with my sketch. I added MY_DEBUG_VERBOSE_SIGNING and got the following messages.

From the node:

2205 TSM:READY
2220 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
2226 Waiting for GW to send signing preferences...
2280 TSF:MSG:READ,0-0-3,s=255,c=3,t=15,pt=6,l=2,sg=0:0100
2285 Received signing presentation, but signing is not supported (message ignored)
2349 TSF:MSG:SEND,3-3-0-0,s=255,c=0,t=17,pt=0,l=10,sg=0,ft=0,st=OK:2.0.1-beta
2450 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=6,pt=1,l=1,sg=0,ft=0,st=OK:0

From the GW:

0;255;3;0;9;TSF:MSG:PINGED,ID=3,HP=1
0;255;3;0;9;TSF:MSG:SEND,0-0-3-3,s=255,c=3,t=25,pt=1,l=1,sg=0,ft=0,st=OK:1
0;255;3;0;9;TSF:MSG:READ,3-3-0,s=255,c=3,t=15,pt=6,l=2,sg=0:0100
0;255;3;0;9;Informing node 3 that we do not require signatures because we do not support it
0;255;3;0;9;TSF:MSG:SEND,0-0-3-3,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
0;255;3;0;9;TSF:MSG:READ,3-3-0,s=255,c=0,t=17,pt=0,l=10,sg=0:2.0.1-beta
0;255;3;0;9;Sending message on topic: my_RFM69_gw1-out/3/255/0/0/17
0;255;3;0;9;TSF:MSG:READ,3-3-0,s=255,c=3,t=6,pt=1,l=1,sg=0:0

Do you need some more testing?

[Anticimex]

@jpaulin thanks for testing. The change should have little effect for people using signing. The issue is for people who does not use it as the node did not wait for a gw response in that case.

[korttoma]

tried the latest development branch on my test system and the results are good so far. Now I just need to update my "real" system also. Thanks to everyone that participated in solving this!!

[korttoma]

Updated now also 2 nodes in my "real" system and both now successfully reported Lib version. I did not update the GW. Maybe it is safe to say that the Lib reporting problem is solved now.

Now I will try to look in to the sofSerial RFM69 solution that @scalz is working on, do you have any eta on when you will try to include our solution in the official MySensors development branch?
Any recommendations for the wiring of an W5100/RFM69 Gateway?

[scalz]

i have removed my stuff for the moment..
for integration in mysensors, it's coreteam work in progress, thx to @Tekka ;)
Once, someone smart told me that 1+1=3, quite true :) and with Tekka i think we can say = 4 ahah
So for an eta.., i prefer not saying anything lol, just i'm back on this for the week, i was playing with other stuff..

[jpaulin]

@korttoma I have an up and running RFM69 GW with W5100 (actually a W5500, but it's the same). I'm not using softSerial in my setup. With this it is possible to integrate with eg. Moteino or Anarduino, which comes with the radio chip preinstalled on the Arduino compatible boards. But a standard arduino with a separate chip would work as well (lower price). The GW-RFM69 has been stable and flawless, as far as I can see, for quite some time. The SPI for the RFM69 and W5500 uses the following pins in my setup.

RFM69W:

• SCK = 13
• MISO = 12
• MOSI = 11
• CS = 10
• IRQ = 2

W5500:

• SCK = 13
• MISO = 12
• MOSI = 11
• CS = 7

I'll try to share wire diagrams and software setup later. I had to do some tweakings in the ethernet.h file to make it work.

[korttoma]

@jpaulin I am very interested in your solution. Could it work with a normal Arduino Pro mini 3,3V version? Just so that I know if I need to order some parts or can work with what I have.

[jpaulin]

@korttoma
Yes, it should work with an Arduino Pro Mini 3.3V. Personally I'm using for the moment an Anarduino. But it's almost the same.

[jpaulin]

@korttoma
My RFM69 - Ethernet - GW has the following setup.

On the RFM69 radio I'm using D10 as CS pin (Chip Select). This is because in my case the RFM69 radio is hard-wired to pin D10, as with eg. a Moteino or Anarduino. But this is not mandatory. If you use an Arduino Pro Mini you could select another pin for the RFM69 CS-pin on the Arduino, if you want.

The Ethernet W5100 uses the Arduino IDE pre-installed library Ethernet.h. In the library the CS-pin is as well fixed to D10. CS of course has to be unique for each SPI slave device. I changed directly in the Ethenet.h library the CS pin to be D7. But D9, D8 or some other pin is ok to use as well.

Take care, if none of your SPI Slave devices (radio and W5100) uses D10 as CS-pin. In this case make sure to set the D10 pin to OUTPUT in your sketch to keep the Arduino as SPI Master. On the ATmega328 the D10 pin is hard-wired to be the SS pin (Slave Select). The Arduino has to always be the SPI Master and the devices SPI Slaves on the SPI bus to get everything to work.

Another consideration is:
The RFM69 radio uses the SPI functions during the interrupt sequence (Pin D2). The Ethernet.h library won't work correctly if another device uses SPI signalling during an interrupt sequence. The fix I'm using is to block interrupts from the radio-unit during SPI usage of the W5100 unit. I haven't seen any flaws with this fix so far. I've found other people on the internet using the same fix too without issues.

To use RFM69 CS=D10 pin and W5100 CS=D9 pin do the following change in the W5100.h that is part of the Ethernet library.
If you use a Windows computer you'll find the file here:

C:\Program Files (x86)\Arduino\libraries\Ethernet\src\utility\w5100.h

Look for the following lines at the end of the file.

#else
  inline static void initSS()    { DDRB  |=  _BV(2); };
  inline static void setSS()     { PORTB &= ~_BV(2); };
  inline static void resetSS()   { PORTB |=  _BV(2); };
#endif

If you use D9 as CS pin for the W5100 module change as follows:

#else
  inline static void initSS()    { DDRB  |=  _BV(1); };
  inline static void setSS()     { cli(); PORTB &= ~_BV(1); };
  inline static void resetSS()   { PORTB |=  _BV(1); sei(); };
#endif

If you uses, as I do, D7 as CS pin then change as follows:

#else
  inline static void initSS()    { DDRD  |=  _BV(7); };
  inline static void setSS()     { cli(); PORTD &= ~_BV(7); };
  inline static void resetSS()   { PORTD |=  _BV(7); sei(); };
#endif

If you want to keep D10 for the W5100 just add the cli(); and sei(); macros. (cli = clear interrupt. sei = set interrupt).

If you want to use another SPI CS pin than D10 on the RFM69 radio (eg. D9) it should be safe to modify it directly in the sketch with the preprocessor constant:

#define MY_RF69_SPI_CS 9

For the MOSI, MISO and SCK pins use as mentioned above, they are as well hard-wired in the ATmega328 chip to D11,D12 and D13.

If you have any doubts please let me know.

[korttoma]

Hi @jpaulin thanks for this, I will test it immediately.

Would you mind sharing your complete sketch? The MySensors W5100 GW example has all this Soft SPI stuff, I might make a mistake clearing out all the unnecessary parts.

Btw, do you see any problem with me adding some local "sensors" to this type of GW? I would like to add 4 relays and 4 buttons to it also.

[korttoma]

It is alive :D

I managed to get a W5100 RFM69 Gateway working.
I wanted minimal change to the w5100.h so I just added the cli(); and sei(); macros.
(I would still like to see your sketch @jpaulin to compare)

There are some random failures in the RF communication but these might be because of the breadboard setup. I just have one temp sensor sending in some value now to test with.