[SOLVED] After upgrade to mysensors 2.1 problems with radio module might manifest itself more.

[Nicklas Starkel]

So I took the plunge and sent the same sketch as I had when using mysensors 2.0 to my Arduino Mega with w5100 ethernet shield.
It is the MQTT sketch.

After this I could not use any nodes with signing.

LOGS as follow.

Gateway:

0;255;3;0;9;Attempting MQTT connection...
0;255;3;0;9;MQTT connected
0;255;3;0;9;Sending message on topic: MyMQTT/0/255/0/0/18
0;255;3;0;9;TSF:MSG:READ,6-6-0,s=255,c=0,t=17,pt=0,l=5,sg=0:2.1.0
0;255;3;0;9;!TSF:MSG:SIGN VERIFY FAIL
0;255;3;0;9;TSF:MSG:READ,6-6-0,s=255,c=3,t=6,pt=1,l=1,sg=0:0
0;255;3;0;9;!TSF:MSG:SIGN VERIFY FAIL
0;255;3;0;9;TSF:MSG:READ,6-6-0,s=255,c=3,t=11,pt=0,l=17,sg=0:Sensebender Micro
0;255;3;0;9;!TSF:MSG:SIGN VERIFY FAIL
0;255;3;0;9;TSF:MSG:READ,6-6-0,s=255,c=3,t=26,pt=1,l=1,sg=0:2
0;255;3;0;9;!TSF:MSG:SEND,0-0-6-6,s=255,c=3,t=16,pt=0,l=0,sg=0,ft=0,st=NACK:
0;255;3;0;9;!TSF:MSG:SIGN FAIL
0;255;3;0;9;TSF:MSG:READ,6-6-0,s=255,c=3,t=26,pt=1,l=1,sg=0:2
0;255;3;0;9;TSF:MSG:SEND,0-0-6-6,s=255,c=3,t=16,pt=0,l=0,sg=0,ft=0,st=OK:
0;255;3;0;9;TSF:MSG:READ,6-6-0,s=255,c=3,t=17,pt=6,l=25,sg=0:192E7CA3C75EA05198759BA9F6FF17532DA77693DED0692059
0;255;3;0;9;!TSF:MSG:SEND,0-0-6-6,s=255,c=3,t=27,pt=1,l=1,sg=1,ft=0,st=NACK:1
0;255;3;0;9;TSF:MSG:READ,6-6-0,s=255,c=3,t=26,pt=1,l=1,sg=0:2
0;255;3;0;9;!TSF:MSG:SEND,0-0-6-6,s=255,c=3,t=16,pt=0,l=0,sg=1,ft=0,st=NACK:
0;255;3;0;9;!TSF:MSG:SIGN FAIL
0;255;3;0;9;TSF:MSG:READ,6-6-0,s=1,c=1,t=0,pt=7,l=5,sg=0:26.9
0;255;3;0;9;!TSF:MSG:SIGN VERIFY FAIL
0;255;3;0;9;TSF:MSG:READ,6-6-0,s=0,c=1,t=1,pt=2,l=2,sg=0:34
0;255;3;0;9;!TSF:MSG:SIGN VERIFY FAIL

And this is a NODE:

0 MCO:BGN:INIT NODE,CP=RNNNAS-,VER=2.1.0
3 TSM:INIT
4 TSF:WUR:MS=0
11 TSM:INIT:TSP OK
13 TSM:INIT:STATID=6
15 TSF:SID:OK,ID=6
17 TSM:FPAR
53 TSF:MSG:SEND,6-6-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
2060 !TSM:FPAR:NO REPLY
2062 TSM:FPAR
2105 TSF:MSG:SEND,6-6-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
4112 !TSM:FPAR:NO REPLY
4114 TSM:FPAR
4118 TSF:MSG:SEND,6-6-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
6125 !TSM:FPAR:NO REPLY
6127 TSM:FPAR
6130 TSF:MSG:SEND,6-6-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
8138 !TSM:FPAR:FAIL
8139 TSM:FAIL:CNT=1
8141 TSM:FAIL:PDT
18144 TSM:FAIL:RE-INIT
18146 TSM:INIT
18153 TSM:INIT:TSP OK

If I remove the below signing code in the GW it all works OK.

#define MY_SIGNING_SOFT
#define MY_SIGNING_SOFT_RANDOMSEED_PIN 7
#define MY_SIGNING_REQUEST_SIGNATURES

Anyone else with the same problem (ping @Anticimex )?
I downgraded my arduino boards to 1.6.11 as per instructions in some other posts.
I also ran the signing sketch again (software backend) on my nodes, but to no avail.

//Nicklas

[Anticimex]

@Nicklas-Starkel the log shows many NACKs. Signing require reliable message exchanges. This is also mentioned in the signing troubleshooting guide: https://ci.mysensors.org/job/Verifiers/job/MySensors/job/master/Doxygen_HTML/group__MySigningTroubleshootinggrp.html

[Nicklas Starkel]

@Anticimex

I had built the NRF tester and had perfect coverage with no drops in my apartment before (with signing!) from any of my NRFs.

To test further I even put one of those plateboards for testing NRF to my gateway. Added external power from another source.
Did the same to my NRF tester but still it fails.
Have put them close and also gone around in my apartment to eliminate distance signal error, still the same.
So only difference is mysensors 2.1.
However, since no one else is complaining then maybe the problem IS with my setup.
Or could also be that no one else has updated

I'm using Arduino 1.6.11, maybe I should install 1.8.1?

[Anticimex]

@Nicklas-Starkel
Well, I am not the right person to answer that as I have not been involved in the radio driver development but I believe @tekka could contribute with some statement whether there are known changes between the versions that could affect transmission quality.

[tekka]

@Anticimex @Nicklas-Starkel
Soft- and ATSHA204A-based signing using RF24 were tested for the 2.1.0 release and are confirmed to work.
Test setup:

• Random HMAC key, stored in EEPROM and ATSHA204 using personalizer sketch.
• Controller: MYSController build 3315
• GW: ESP8266, soft signing, nRF24L01+PA
• Node1: Arduino Uno, soft signing, nRF24L01+
• Node2: Sensebender, ATSHA204 signing, nRF24L01+

Result:
==> signing ok with matching HMAC keys
==> signing fails with scrambled HMAC keys

[petewill]

@Nicklas-Starkel I recently updated my Mega sensor (that had been running for months without any issues) to 2.1 and experienced issues with NACKs too. It would work for a little while (half a day or so) then the NACKs would start appearing. A few hours after the NACKs showed up it would then only produce NACKs. I seemed to have finally resolved the issue by switching the 4.7uF capacitor with a 47uF on the NRF radio. It has been running for 3 days without a single NACK. I know it's not the same issue you're having but maybe something to try. By the way, the capacitor is soldered right to the radio pin headers.

[Nicklas Starkel]

@tekka Im not saying it doesn't work, but it doesn't work

I have spent some hours yesterday and today testing different possibilities.
However, as soon as I enable soft signing it just my nodes get no reply or partial reply sometimes.

If I disable softsigning in GW, everything works perfect.

I would guess that the problem could be with my hardware as everything except my ethernetshield is china clones.
But since I had used everything and everything worked without a flaw (with signing) and the only thing I did was update library via arduino, and uploaded the same sketch (without editing it) to the GW.
I would beg to differ that something is not broken with mysensors 2.1. Or atleast maybe compability has been lowered.
Or mysensors 2.1 changed something in the use of the NRF chips (the + as well as PA with big antenna!).

I uninstalled arduino completely, but had some problems installing 2.0 as it complained about some stuff about my arduino mega. But when I installed 2.1 it worked with mega which is also weird.

It is no biggie that signing doesn't work for me, but It bugs me that this should be simple and it isn't.

To sum up:
My NRF works without signing. It even works behind steel heat radiators across my flat (yes, did that for testing purposes!)
Switch on signing and I can barely connect to my GW.

[Anticimex]

@Nicklas-Starkel
I can only repeat what I said earlier. You have NACKs. That indicate that messages are not coming through. It is NOT a signing issue but a rf issue. The only reason it looks like a signing issue is because when you use signing, messages are significantly longer and are therefore statistically more difficult to transmit/receive. Improve decoupling of your radios and you'll probably be fine like suggested by Pete. I also explained this in the signing troubleshooting guide I linked above.
In 2.x significant changes are made to the rf drivers. These are all for the better but could mean power consumption of the radio has a different profile which inadequate decoupling will have trouble dealing with.

[Nicklas Starkel]

@Anticimex I understand that the problem is not with signing itself.
I also understand that with longer messages with low power things it gets extra difficult.

What I am saying is mysensors 2.1 did something and my hardware can't cope with it.

I have tried socket adapters for NRF modules.
I have tried giving them external power.
I tried without socket adapters.
Giving external power as well.
I have tried 47uF capacitator.

Mind you, I have tried several different hardware pieces as well just to make sure.

I will redo everything above again just to make sure and report back.

[Anticimex]

@Nicklas-Starkel
Then I suggest you rename the topic to draw more attention to the core issue so others who might have had similar problems with 2.x can share their experiences as well. All who don't use signing but might have radio troubles in 2.x won't realize that this is discussed here otherwise.

[Nicklas Starkel]

Well so I deleted whole Arduino to make sure I got everything fresh.
Installed arduino and mysensors 2.0 via library manager and also mysensors HW arduino manager in preferences.

Updated my gateway and all nodes work with signing ON.
The nodes are a mixture now with mysensors 2.0 as well as 2.1.

I updated library to 2.1 in my GW and instantly I have failed messages with Signing ON.

To make sure, I powered the GW from one source and NRF separately from external source (5v computer usb)
No changes. I tried different NRF as well as NRF PA with big antenna.
I even tried different antennas (used to other applications). With mysensors 2.0 they work, with 2.1 fails.

I took a node and powered it similar with external power source (3x1.5v AAA) for the NRF but it would still give me fails.

When I go back to mysensors 2.0 everything works great again.

I guess I'll stick with 2.0 and wait and see if other people upgrade their stuff and use signing might see the same problem with their hardware.

[Anticimex]

@Nicklas-Starkel ok. Well like I already said, your problem (at least according to your logs) have nothing to do with signing, but with rf issues. If you were to send messages that were larger you most likely would have the same problem, signing on or off, so I can just stress that referencing signing just shadows the real issue. Signing is only a way of extending the message size to provoke the issue. Signing has been verified to work with 2.1 as @tekka stated, so I (as the security guy) consider the issue settled from a signing perspective.

[Nicklas Starkel]

@Anticimex maybe I was not clear enough or used the wrong words?
I still understand that the problem is not with signing.
I pointed out that using signing provokes the error.
The error is still an underlying issue weather it be my hardware or it be NRF module code in mysensors 2.1.

[tekka]

@Nicklas-Starkel Did you also try with different GW hardware, i.e. ESP8266 or a regular Uno?

[Nicklas Starkel]

@tekka that was my next goal

I just built an UNO with an ethernet shield and low and behold its working with 47uF capacitor.
Conclusion:
MEGA does not work with NRF plate adapter nor 47uF capacitor.
UNO works with 47uF capacitor only (not with plate).

All nodes are reporting in (mixed 2.0 and 2.1).
I use @AWI quality meter with NRF plate board to test as well and it is OK.

So same NRF hardware yields different results on UNO and MEGA for me.

[Anticimex]

@Nicklas-Starkel
Any difference in how you power the radio?

[Nicklas Starkel]

@Anticimex
I power either with 3xAAA (4.5V) if I'm using the plate.
If I'm using only capacitor I power with 2xAA (3.3V).
Tried both versions with same results. Only difference is Arduino/MEGA.
All batteries fresh and also checked with voltmeter

[Anticimex]

@Nicklas-Starkel
I mean do you power the radio by dedicated regulator or directly from mcu?

[Nicklas Starkel]

@Anticimex I not sure I follow you and perhaps I am doing it wrong
I power my arduinos via their usb jacket, 5V/2.1A
If I'm using the NRF plate it has its own dedicated regulator hence its getting 3xAAA 4.5V.
Otherwise I am using 2xAA which is around 3.3V connected to VCC/GND of the radio module and GND is also connected to the arduino.

[Anticimex]

@Nicklas-Starkel
I owe you an apology. It appears there for some configuration combinations does exist a bug for signing in 2.1. @tekka found a problem where the signing presentation in some situations would not be sent to the correct node.
That said, you do appear to also have issues with the radio (the NACKs) but you are correct in that there is an issue with signing. It will be resolved in 2.1.1.
Thank you for reporting, and again, please accept my sincere apology for failing to look more thoroughly through your logs.

[Nicklas Starkel]

@Anticimex , eh, no apologies needed. I'm thankful mysensors is opensource and actually FREE!
Cool that you found a bug and if I helped, I'm glad

[Nicklas Starkel]

@Anticimex and @tekka .
I just upgraded my GW (MEGA) and most sensors.
I do not know what you did, but my range is incredible now (and everything works!)
I had good coverage in my apartment before, but if I went outside my front door it started failing (steel security door).
Now, I can not only go outside the door, I can actually go 3 stairs down in my building with AWIs NRF tester.
Same hardware is used and nothing has changed from my part.

Kudos and thank you.

[Anticimex]

@Nicklas-Starkel nice to hear! We are constantly trying to improve the performance in various aspects. I think it is @tekka that deserves the credit for increased rf performance as he has done a tremendous work with the drivers.