Raspberry Gateway 2.1.1 + signing

[Anticimex]

@t3chie looks like the personalization has not finished successfully. How do you store the hmac key to the rPi? If you don't use the personalizer sketch you either need to write the checksum yourself or disable the checksum validation. It could be that the method for personalizing on rPi does not calculate and store a checksum. So either that has to change, or the checksum check should be disabled on rPi/Linux.

[t3chie]

@Anticimex Sorry for that - when I changed Pi's I forgot to personalize the GW. But that didn't made it working:
mysgw: Starting gateway...
mysgw: Protocol version - 2.2.0-beta
mysgw: MCO:BGN:INIT GW,CP=RNNG-SQX,VER=2.2.0-beta
mysgw: !SGN:PER:TAMPERED
mysgw: !SGN:INI:BND FAIL
mysgw: TSF:LRT:OK
mysgw: TSM:INIT
mysgw: TSF:WUR:MS=0
mysgw: TSM:INIT:TSP OK
mysgw: TSM:INIT:GW MODE
mysgw: TSM:READY:ID=0,PAR=0,DIS=0
mysgw: MCO:REG:NOT NEEDED
mysgw: Listening for connections on 0.0.0.0:5003
mysgw: MCO:BGN:STP
mysgw: MCO:BGN:INIT OK,TSP=1
mysgw: TSF:MSG:READ,118-205-14,s=1,c=2,t=118,pt=6,l=0,sg=1:
mysgw: TSF:MSG:REL MSG
mysgw: !TSF:RTE:14 UNKNOWN
mysgw: SGN:SGN:118!=0 NUS
mysgw: !TSF:MSG:SEND,118-0-14-14,s=1,c=2,t=118,pt=6,l=0,sg=1,ft=0,st=NACK:
mysgw: TSF:MSG:READ,138-29-188,s=181,c=6,t=66,pt=5,l=16,sg=0:3636023473
mysgw: TSF:MSG:REL MSG
mysgw: !TSF:RTE:188 UNKNOWN
mysgw: SGN:SGN:138!=0 NUS
mysgw: !TSF:MSG:SEND,138-0-188-188,s=181,c=6,t=66,pt=5,l=16,sg=0,ft=0,st=NACK:3636023473
mysgw: TSF:MSG:READ,184-237-134,s=214,c=0,t=229,pt=6,l=13,sg=1:5A403EDF050D2265DF00000000
mysgw: !TSF:MSG:PVER,3=2
mysgw: TSF:MSG:READ,233-187-65,s=93,c=1,t=52,pt=1,l=1,sg=1:222
mysgw: !TSF:MSG:PVER,1=2
mysgw: TSF:MSG:READ,177-160-211,s=159,c=6,t=135,pt=4,l=11,sg=1:-2047481336
mysgw: !TSF:MSG:PVER,0=2
mysgw: TSF:MSG:READ,166-111-53,s=187,c=3,t=196,pt=5,l=9,sg=0:204581490
mysgw: TSF:MSG:REL MSG
mysgw: !TSF:RTE:53 UNKNOWN
mysgw: SGN:SGN:166!=0 NUS
mysgw: !TSF:MSG:SEND,166-0-53-53,s=187,c=3,t=196,pt=5,l=9,sg=0,ft=0,st=NACK:204581490
mysgw: TSF:MSG:READ,223-72-13,s=199,c=4,t=110,pt=5,l=4,sg=0:3736138177
mysgw: !TSF:MSG:PVER,1=2
mysgw: TSF:MSG:READ,151-70-68,s=120,c=3,t=166,pt=0,l=15,sg=1:?rQ?0??
mysgw: !TSF:MSG:PVER,0=2

Isn't the Node ID reported changing between random values although it should only see ID 100 ?

[Anticimex]

@t3chie it still reports !SGN:PER:TAMPERED so my original question(s) are still valid :)

[t3chie]

@Anticimex I use:

./bin/mysgw --set-soft-hmac-key D94AB9BF54D266CA517C23C870FF7A6...
./bin/mysgw --set-soft-serial-key 2200002200...

How can I calculate the checksum or disable it?

[Anticimex]

@t3chie just comment https://github.com/mysensors/MySensors/blob/development/core/MySigningAtsha204Soft.cpp#L127

I'm pretty sure the problem is that on Linux, a checksum is never calculated, so I will patch the init function of the software signing backend to ignore that calculation on Linux.
I do also note that you seem to get NACKs. I can already now say that those have to go away for signing to work. See here.

[t3chie]

@Anticimex Have tried to comment out line 127, but as you said no joy. I know that the radio I am using is not the best but I am still not sure if the HW is the culprit here.

I have put that RF24 module onto a Nano with itead RF Shield and loaded a 2.1.1 Gateway Sketch including signing - that works against the same node I am testing the Pi.
I am using the interrupt variant of the Pi cabling - could that be the problem?

[Anticimex]

@t3chie if you read the troubleshooting guide which I linked to, the topmost and most common issue is radio problems and these are not signing related. They are because you have a flaky radio connection (reasons could be many, and there are a ton of threads on the forum that discuss these). You see these when you enable signing because then full size messages are transmitted and not the relatively small messages you typically transmit.

[t3chie]

@Anticimex Well, I have read all troubleshooting material many times and agree that mostly it is HW. This time however it was stupidity on my side!
I mixed up signing with encryption on the Pi compile time flags.
To summarize:
On 2.1.1, the architecture needs to be set and signing works (have not tested encryption yet).
On dev, architecture needs to be left alone (not set) and checksum calculation needs to be commented out (line 127 of MySigningAtsha204Soft.cpp).

[Anticimex]

@t3chie ah, ok. Yes, those NUS messages indicated that something was really mixed up :)
I will file a pull request to fix the checksum issue. Thanks for reporting!

[Anticimex]

@t3chie Could I ask you a big favour to test my PR on your linux gw and node?
https://github.com/mysensors/MySensors/pull/885

[t3chie]

@Anticimex Sure, can do.
Would you refresh my memory - how can I check out the tree with that pullrequest?

[Anticimex]

@t3chie thanks!
git checkout -b fallberg-signing development
git pull https://github.com/fallberg/MySensors.git signing

[t3chie]

@Anticimex I did:
git clone https://github.com/mysensors/MySensors.git --branch development
cd MySensors
git pull https://github.com/fallberg/MySensors.git signing

Then compile with:
./configure --my-gateway=ethernet
--my-transport=nrf24
--my-rf24-ce-pin=22
--my-rf24-cs-pin=24
--my-rf24-irq-pin=15
--my-rf24-pa-level=RF24_PA_LOW
--my-leds-err-pin=12
--my-leds-rx-pin=16
--my-leds-tx-pin=18
--my-rf24-channel=76
--my-debug=enable
--my-signing=software
--my-signing-request-signatures
--my-signing-request-gw-signatures-from-all
--my-signing-debug

Result with one node restarting while others are already running:

mysgw: TSF:MSG:READ,100-100-255,s=255,c=3,t=7,pt=0,l=0,sg=0:
mysgw: TSF:MSG:BC
mysgw: TSF:MSG:FPAR REQ,ID=100
mysgw: TSF:PNG:SEND,TO=0
mysgw: TSF:CKU:OK
mysgw: TSF:MSG:GWL OK
mysgw: SGN:SKP:MSG CMD=3,TYPE=8
mysgw: TSF:MSG:SEND,0-0-100-100,s=255,c=3,t=8,pt=1,l=1,sg=1,ft=0,st=OK:0
mysgw: TSF:MSG:READ,100-100-0,s=255,c=3,t=24,pt=1,l=1,sg=0:1
mysgw: SGN:SKP:MSG CMD=3,TYPE=24
mysgw: TSF:MSG:PINGED,ID=100,HP=1
mysgw: SGN:SKP:MSG CMD=3,TYPE=25
mysgw: TSF:MSG:SEND,0-0-100-100,s=255,c=3,t=25,pt=1,l=1,sg=1,ft=0,st=OK:1
mysgw: TSF:MSG:READ,100-100-0,s=255,c=3,t=15,pt=6,l=2,sg=0:0101
mysgw: SGN:SKP:MSG CMD=3,TYPE=15
mysgw: SGN:PRE:SGN REQ,FROM=100
mysgw: !SGN:PRE:WHI NREQ,FROM=100 REJ
mysgw: SGN:PRE:SGN REQ,TO=100
mysgw: SGN:PRE:WHI NREQ,TO=100
mysgw: SGN:SKP:MSG CMD=3,TYPE=15
mysgw: TSF:MSG:SEND,0-0-100-100,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0101
mysgw: SGN:PRE:XMT,TO=100
mysgw: TSF:MSG:READ,100-100-0,s=255,c=3,t=16,pt=0,l=0,sg=0:
mysgw: SGN:SKP:MSG CMD=3,TYPE=16
mysgw: !SGN:NCE:GEN
mysgw: TSF:MSG:READ,100-100-0,s=255,c=3,t=16,pt=0,l=0,sg=0:
mysgw: SGN:SKP:MSG CMD=3,TYPE=16
mysgw: !SGN:NCE:GEN
mysgw: TSF:MSG:READ,100-100-0,s=255,c=3,t=16,pt=0,l=0,sg=0:
mysgw: SGN:SKP:MSG CMD=3,TYPE=16
mysgw: !SGN:NCE:GEN
mysgw: TSF:MSG:READ,100-100-0,s=255,c=3,t=16,pt=0,l=0,sg=0:
mysgw: SGN:SKP:MSG CMD=3,TYPE=16
mysgw: !SGN:NCE:GEN
mysgw: TSF:MSG:READ,100-100-0,s=1,c=3,t=16,pt=0,l=0,sg=0:
mysgw: SGN:SKP:MSG CMD=3,TYPE=16
mysgw: !SGN:NCE:GEN
mysgw: TSF:MSG:READ,100-100-0,s=255,c=3,t=26,pt=1,l=1,sg=0:2
mysgw: SGN:SKP:MSG CMD=3,TYPE=26
mysgw: SGN:SKP:MSG CMD=3,TYPE=16
mysgw: TSF:MSG:SEND,0-0-100-100,s=255,c=3,t=16,pt=0,l=0,sg=1,ft=0,st=OK:
mysgw: SGN:SGN:NCE REQ,TO=100
mysgw: TSF:MSG:READ,100-100-0,s=255,c=3,t=17,pt=6,l=25,sg=1:<NONCE>
mysgw: SGN:SKP:MSG CMD=3,TYPE=17
mysgw: SGN:NCE:FROM=100
mysgw: SGN:BND:NONCE=4BD28D2B47813291B354D4A856B07E7A6B496CF86FA2FF969D7C2D115FAC09DF
mysgw: SGN:BND:HMAC=07FF305FFEB4D1489C3C9BA63A281890DDD850573B406FB691C09C23DC12DD00
mysgw: SGN:BND:SIG WHI,ID=0
mysgw: SGN:BND:SIG WHI,SERIAL=000000000000000000
mysgw: SGN:SGN:SGN
mysgw: TSF:MSG:SEND,0-0-100-100,s=255,c=3,t=27,pt=1,l=1,sg=1,ft=0,st=OK:1

Looks OK for me, right?

[Anticimex]

@t3chie I think so. There are a few errors in the log though but I am not sure if that is due to other nodes on your network not being configured properly.

[Anticimex]

@t3chie it also looks like your gw is not always succeeding in generating nonces.

[Anticimex]

@t3chie could you please include the startup messages from the gateway? Looks like it still does not initialize properly. It should not have a problem generating nonces if it thinks personalization is OK.

[t3chie]

@Anticimex

mysgw: Starting gateway...
mysgw: Protocol version - 2.2.0-beta
mysgw: MCO:BGN:INIT GW,CP=RNNG-SQ-,VER=2.2.0-beta
mysgw: SGN:PER:OK
mysgw: SGN:PER:OK
mysgw: TSF:LRT:OK
mysgw: TSM:INIT
mysgw: TSF:WUR:MS=0
mysgw: TSM:INIT:TSP OK
mysgw: TSM:INIT:GW MODE
mysgw: TSM:READY:ID=0,PAR=0,DIS=0
mysgw: MCO:REG:NOT NEEDED
mysgw: Listening for connections on 0.0.0.0:5003
mysgw: MCO:BGN:STP
mysgw: MCO:BGN:INIT OK,TSP=1
mysgw: New connection from 127.0.0.1
mysgw: Client 0 connected
mysgw: Client 0: 255;255;3;0;20;
mysgw: SGN:SKP:MSG CMD=3,TYPE=20
mysgw: TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
mysgw: Client 0: 0;255;3;0;2;
mysgw: TSF:MSG:READ,198-198-0,s=255,c=3,t=21,pt=1,l=1,sg=0:0
mysgw: SGN:SKP:MSG CMD=3,TYPE=21
mysgw: Client 0: 0;255;3;0;2;
mysgw: Client 0: 0;255;3;0;2;
mysgw: TSF:MSG:READ,100-100-255,s=255,c=3,t=7,pt=0,l=0,sg=0:
mysgw: TSF:MSG:BC
mysgw: TSF:MSG:FPAR REQ,ID=100
mysgw: TSF:PNG:SEND,TO=0
mysgw: TSF:CKU:OK
mysgw: TSF:MSG:GWL OK
mysgw: SGN:SKP:MSG CMD=3,TYPE=8
mysgw: TSF:MSG:SEND,0-0-100-100,s=255,c=3,t=8,pt=1,l=1,sg=0,ft=0,st=OK:0
mysgw: TSF:MSG:READ,100-100-0,s=255,c=3,t=24,pt=1,l=1,sg=0:1
mysgw: SGN:SKP:MSG CMD=3,TYPE=24
mysgw: TSF:MSG:PINGED,ID=100,HP=1
mysgw: SGN:SKP:MSG CMD=3,TYPE=25
mysgw: TSF:MSG:SEND,0-0-100-100,s=255,c=3,t=25,pt=1,l=1,sg=0,ft=0,st=OK:1
mysgw: TSF:MSG:READ,100-100-0,s=255,c=3,t=15,pt=6,l=2,sg=0:0101
mysgw: SGN:SKP:MSG CMD=3,TYPE=15
mysgw: SGN:PRE:SGN REQ,FROM=100
mysgw: !SGN:PRE:WHI NREQ,FROM=100 REJ
mysgw: SGN:PRE:SGN REQ,TO=100
mysgw: SGN:PRE:WHI NREQ,TO=100
mysgw: SGN:SKP:MSG CMD=3,TYPE=15
mysgw: TSF:MSG:SEND,0-0-100-100,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0101
mysgw: SGN:PRE:XMT,TO=100
mysgw: TSF:MSG:READ,100-100-0,s=255,c=3,t=16,pt=0,l=0,sg=0:
mysgw: SGN:SKP:MSG CMD=3,TYPE=16
mysgw: !SGN:NCE:GEN
mysgw: Client 0: 0;255;3;0;2;

PS:
I am on Skype with the same Nick if you want more synchronous communication.

[Anticimex]

@t3chie hm, odd. And you get the validation message twice. I must have messed something up. I'll have a closer look tonight and get back to you. Thanks for helping out!

[Anticimex]

@t3chie I believe I found the issue on my pr. Sloppy coding is all I can say... I will notify you once I have resolved it. In the meantime you can delete your copy of my branch (I tend to force push to my pull requests).

git checkout development
git branch -D fallberg-signing

[Anticimex]

@t3chie I've pushed updates now. It should resolve the problems. I accidentally excluded the part where the backends are initialized, and there are safeguards against that so it refused to calculate nonces for signing. You should now get a "SGN:INI:BND OK" after "SGN:PER:OK"