[security] Introducing signing support to MySensors

[Anticimex]

@scalz Thanks.
Could you please also post the construction code of your MySensor object.
And also your MyConfig.h changes (where you enable the signing functionality). Don't post your private key though
The initial setup is all signed and well but then your node seem to not even attempt to sign the data it sends (because it should have sent a c=3,t=16 and waited for a c=3,t=17 prior to any sensor data).
I see from the first two messages that the gateway has been informed that your sensor require signatures (of messages from the gateway). And the gateway do respond that it also require signautres but for some reason your sensor do not sign the sensor values (the presentation handshaking is transmitted in signed form as expected).
I don't have a quick answer to why your node does not sign it's sensor data. Could you also share your code that transmits (in your sketch that is). Perhaps the easiest is to post the entire sketch (WITHOUT your private key).

I am afraid I have to prioritize my newborn son, so I hope for understanding if my replies will be delayed. @hek might know if something has happened on dev branch that could affect signing. I did thoroughly test it when it was submitted but I have not been keeping track on activities on dev for a while now.

[scalz]

@Anticimex: Congratulation for your son! it is not hurry, no problem. and thank you for taking time.
So maybe I am missing something, because I have no key in the sketch??? I am using ATSHA hardware chip.. The key is stored by using SHAPersonalizer, isn't it??

I have not made any changes in Myconfig.h as it is default enabled. So it is like this:

// Disable to completly disable signing functionality in library
#define MY_SIGNING_FEATURE

And the node sketch:

#include <SPI.h>
#include <MyTransportNRF24.h>
#include <MyHwATMega328.h>
#include <MySensor.h> 
#include <DHT.h>  
#include <MySigningAtsha204.h>

#define CHILD_ID_HUM 0
#define CHILD_ID_TEMP 1
#define HUMIDITY_SENSOR_DIGITAL_PIN 5
unsigned long SLEEP_TIME = 30000; // Sleep time between reads (in milliseconds)

MyTransportNRF24 radio;  // NRFRF24L01 radio driver
MyHwATMega328 hw; // Select AtMega328 hardware profile
MySigningAtsha204 signer; // Select HW ATSHA signing backend
MySensor gw(radio, hw, signer);

DHT dht;
float lastTemp;
float lastHum;
boolean metric = true; 
MyMessage msgHum(CHILD_ID_HUM, V_HUM);
MyMessage msgTemp(CHILD_ID_TEMP, V_TEMP);


void setup()  
{ 
  gw.begin(NULL, 10);
  dht.setup(HUMIDITY_SENSOR_DIGITAL_PIN); 

  // Send the Sketch Version Information to the Gateway
  gw.sendSketchInfo("Humidity", "1.0");

  // Register all sensors to gw (they will be created as child devices)
  gw.present(CHILD_ID_HUM, S_HUM);
  gw.present(CHILD_ID_TEMP, S_TEMP);
  
  metric = gw.getConfig().isMetric;
}

void loop()      
{  
  delay(dht.getMinimumSamplingPeriod());

  float temperature = dht.getTemperature();
  if (isnan(temperature)) {
      Serial.println("Failed reading temperature from DHT");
  } else if (temperature != lastTemp) {
    lastTemp = temperature;
    if (!metric) {
      temperature = dht.toFahrenheit(temperature);
    }
    gw.send(msgTemp.set(temperature, 1));
    Serial.print("T: ");
    Serial.println(temperature);
  }
  
  float humidity = dht.getHumidity();
  if (isnan(humidity)) {
      Serial.println("Failed reading humidity from DHT");
  } else if (humidity != lastHum) {
      lastHum = humidity;
      gw.send(msgHum.set(humidity, 1));
      Serial.print("H: ");
      Serial.println(humidity);
  }

  gw.sleep(SLEEP_TIME); //sleep a bit
}

and in Serial Gateway, I have only changed this:

// Message signing driver (signer needed if MY_SIGNING_FEATURE is turned on in MyConfig.h)
//MySigningNone signer;
//MySigningAtsha204Soft signer;
MySigningAtsha204 signer;

// Hardware profile 
MyHwATMega328 hw;

// Construct MySensors library (signer needed if MY_SIGNING_FEATURE is turned on in MyConfig.h)
// To use LEDs blinking, uncomment WITH_LEDS_BLINKING in MyConfig.h
#ifdef WITH_LEDS_BLINKING
MySensor gw(transport, hw , signer, RADIO_RX_LED_PIN, RADIO_TX_LED_PIN, RADIO_ERROR_LED_PIN);
#else
MySensor gw(transport, hw , signer);
#endif

[Anticimex]

@scalz Thanks
Correct. With HW ATSHA204A no key in the sketch is necessary.
And your setup seem ok since there are signed messages being passed successfully so the HW seem ok.

Perhaps you could add a debug print that dumps your messages in the gateway before they are checked for signatures in MySensor::process?
Just put debug(PSTR("read: %d-%d-%d s=%d,c=%d,t=%d,pt=%d,l=%d,sg=%d\n"), msg.sender, msg.last, msg.destination, msg.sensor, mGetCommand(msg), msg.type, mGetPayloadType(msg), mGetLength(msg), mGetSigned(msg)); before #ifdef MY_SIGNING_FEATURE.
You could also do the corresponding thing in your nodes MySensor::sendRoute function so we can see what data you send in order to determine why your node has decided it should not be signed.

It could be you stumbled over a library bug because I see in the code that in sendRoute the if case looks funky.
It says:
if (DO_SIGN(message.destination) && message.sender == nc.nodeId && !mGetAck(message) && mGetLength(msg) &&
but I think it should say
if (DO_SIGN(message.destination) && message.sender == nc.nodeId && !mGetAck(message) && mGetLength(message) &&
Try to change this and see what happens. I have a strong suspicion that your node's msg buffer is empty, and sendRoute therefore don't sign the message because it gets the length from the wrong buffer.

@hek if @scalz tests this successfully, could you patch it? I think it is a bug that it checks mGetLength(msg) and not mGetLength(message) in MySensor::sendRoute.

[hek]

@Anticimex
I could probably fix it tonight if it turns out wrong.

[scalz]

@Anticimex @Hek
Thank you very much for debugging!
First, I tried to add the debug watch. And like you said it seems there was no value.
Then, I looked at sendRoute like you advised, and oh!, it makes sense. A tiny small bug is living there !
So I changed mGetLength(msg) with mGetLength(message)
And now... it seems to work well! yeah! I am so happy

If you want below is my GW logs (with the debug watch too), but I think it's ok now. there is no unwanted "nosign" anymore.

0;0;3;0;9;gateway started, id=0, parent=0, distance=0
0;0;3;0;14;Gateway startup complete.
0;0;3;0;9;read: 0-0-0 s=0,c=0,t=0,pt=0,l=0,sg=0
0;0;3;0;9;read: 10-10-0 s=255,c=3,t=15,pt=2,l=2,sg=0:1
0;0;3;0;9;send: 0-0-10-10 s=255,c=3,t=15,pt=2,l=2,sg=0,st=ok:1
0;0;3;0;9;read: 0-0-10 s=255,c=3,t=15,pt=2,l=2,sg=0
0;0;3;0;9;read: 10-10-0 s=255,c=3,t=16,pt=0,l=0,sg=0:
0;0;3;0;9;send: 0-0-10-10 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:01F428B
0;0;3;0;9;read: 0-0-10 s=255,c=3,t=17,pt=6,l=25,sg=0
0;0;3;0;9;read: 10-10-0 s=255,c=0,t=17,pt=0,l=6,sg=1:1.5 b1
10;255;0;0;17;1.5 b1
0;0;3;0;9;read: 10-10-0 s=255,c=0,t=17,pt=0,l=6,sg=0
0;0;3;0;9;read: 10-10-0 s=255,c=3,t=16,pt=0,l=0,sg=0:
0;0;3;0;9;send: 0-0-10-10 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:013E014
0;0;3;0;9;read: 0-0-10 s=255,c=3,t=17,pt=6,l=25,sg=0
0;0;3;0;9;read: 10-10-0 s=255,c=3,t=6,pt=1,l=1,sg=1:0
10;255;3;0;6;0
0;0;3;0;9;read: 10-10-0 s=255,c=3,t=6,pt=1,l=1,sg=0
0;0;3;0;9;read: 10-10-0 s=255,c=3,t=16,pt=0,l=0,sg=0:
0;0;3;0;9;send: 0-0-10-10 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:0174D82
0;0;3;0;9;read: 0-0-10 s=255,c=3,t=17,pt=6,l=25,sg=0
0;0;3;0;9;read: 10-10-0 s=255,c=3,t=11,pt=0,l=8,sg=1:Humidity
10;255;3;0;11;Humidity
0;0;3;0;9;read: 10-10-0 s=255,c=3,t=11,pt=0,l=8,sg=0
0;0;3;0;9;read: 10-10-0 s=255,c=3,t=16,pt=0,l=0,sg=0:
0;0;3;0;9;send: 0-0-10-10 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:01827E7
0;0;3;0;9;read: 0-0-10 s=255,c=3,t=17,pt=6,l=25,sg=0
0;0;3;0;9;read: 10-10-0 s=255,c=3,t=12,pt=0,l=3,sg=1:1.0
10;255;3;0;12;1.0
0;0;3;0;9;read: 10-10-0 s=255,c=3,t=12,pt=0,l=3,sg=0
0;0;3;0;9;read: 10-10-0 s=0,c=0,t=7,pt=0,l=0,sg=0:
10;0;0;0;7;
0;0;3;0;9;read: 10-10-0 s=0,c=0,t=7,pt=0,l=0,sg=0
0;0;3;0;9;read: 10-10-0 s=1,c=0,t=6,pt=0,l=0,sg=0:
10;1;0;0;6;
0;0;3;0;9;read: 10-10-0 s=1,c=0,t=6,pt=0,l=0,sg=0
0;0;3;0;9;read: 10-10-0 s=1,c=3,t=16,pt=0,l=0,sg=0:
0;0;3;0;9;send: 0-0-10-10 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:01E4D70
0;0;3;0;9;read: 0-0-10 s=255,c=3,t=17,pt=6,l=25,sg=0
0;0;3;0;9;read: 10-10-0 s=1,c=1,t=0,pt=7,l=5,sg=1:23.0
10;1;1;0;0;23.0
0;0;3;0;9;read: 10-10-0 s=1,c=1,t=0,pt=7,l=5,sg=0
0;0;3;0;9;read: 10-10-0 s=0,c=3,t=16,pt=0,l=0,sg=0:
0;0;3;0;9;send: 0-0-10-10 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:0148CE9
0;0;3;0;9;read: 0-0-10 s=255,c=3,t=17,pt=6,l=25,sg=0
0;0;3;0;9;read: 10-10-0 s=0,c=1,t=1,pt=7,l=5,sg=1:36.0
10;0;1;0;1;36.0

I have just a last question regarding signature :

• in the future, when Mysensors new version will be released, does the controller will have to check the signature and to store the PSK? I think so, as the logs shows it is crypted. For doing this we should inspire ourselves
  with your atsha_soft, isn't it? It is just curiosity of mine, as I am looking at your libs, and I am thinking about what changes will be involved on controller side when the moment will come for plugin update...

Now, I have to check the atsha soft version too, then ota with the recently great improvement of tekka..., and understand all new changes in the message format...lots of things.
I like this lib! 1.4 is smart, but dev branch looks very well done and much more optimized.

See you soon!

[Anticimex]

@scalz Great! Thanks for finding and testing this.
I am not sure I understand your question. You mean to store the PSK of atsha_soft in encrypted form in the sketch? That is difficult since the encryption secret then needs to be stored somewhere as well.
I would recommend (if you do use atsha_soft) to burn the fuses to prevent memory readout to protect the PSK.
Normally, this implementation is stored in the gateway which should be physically protected in any case, so the need to protekt PSK in the gateway is less critical than in a node which may be located "outside".

Secure OTA is currently unsupported. It might be supported in the future.

[scalz]

Oh, sorry for my bad english!
Is signing just between a Node and a Gateway. or is the controller involved too? Maybe it's a dumb question, I have not checked yet the changes in message.
For OTA, I understand it is not suppported by signature as you explained in your tuto. But, in the other hand, I think that for initiating an ota, it should need a signed message, so I think it is good.

[Anticimex]

@scalz You decide if your component will require signatures when you create your signer. That is, if you want your gateway to be able to sign messages sent to an actuator (like a lock) but you don't want your gateway to require signed messages from your nodes, you pass a argument to the constructor like this:
MySigningAtsha204 signer(false);
That will

• Configure that node/gw to support signed message processing using HW ATSHA
• Have that node/gw inform others that it does not require signed messages
• Still permit that node/gw to sign messages sent to other nodes/gw who do requre signing

If you stick to default constructor arguments or pass true, the difference is that the node/gw will also inform others that it requires signed messages, and will ignore/discard any message that is not signed which it receive (and is targeted to that particular node/gw). It will not affect messages which are to be relayed, so the node will still work as a relay for unsigned messages.

[scalz]

@Anticimex:
ok, it is good to know. You have thought about everything!
If signer(true) for gateway, Will the controller (Jeedom, Domoticz..) see a crypted payload in serial transmission or will it work like before (I understand there are some changes in message structure, but what about crypted payload, does the controller need to decrypt?)

[Anticimex]

@scalz We only sign messages. There is no encryption involved. And the signature is stored without affecting the message contents so it is perfectly compatible with all controllers. The only thing that could affect a controller is the new sign bit in the message header but I don't think that will make any difference as it is not relevant to the controller. It is the gateway that manages this for it.

[scalz]

ok, it is new for me, so I am mixing vocabulary, but I understand the concept. Thank you, and now I have my answer. no big changes for controller side, great! It makes sense, but I was not sure.

[hek]

Fix pushed to development.

[Anticimex]

Thanks for that!

[jsondag]

I put together a garage door opening using code from korttoma. I used soft signing, and it works great. I was wanting to use a real atsha though.

I bought some atsha204a chips from amazon, and the chips are marked with only "3eas", and a "Y" in one corners. They are soic-8. Is that how this should be marked? I wired one up according to the data sheet. Gnd to Gnd, VCC to 5V, and SDA to A3. I ran the personalizer to generate a key and it said "Failed to wake device." in the serial console. Am I doing something wrong? or did I just get the wrong chip from the seller?

[Anticimex]

@jsondag I don't have a good explanation on the software signing not working. Maybe inadequate decoupling leads to a too noisy power rail?
About the HW issue, it is important that you use the single wire version of atsha204. NOT the i2c version. The i2c version also has an scl line while the single wire only has sda (and power and ground). That means your soic8 should have 5 unused pads. Atmel ordering code for that variant is ATSHA204A-SSHCZ-T but unfortunately it is not printed on the case. So unless you find that information from Amazon, I am afraid it is very difficult to determine the type of the chips you've got.

[jsondag]

@Anticimex. Software signing is working fine.

Thanks for the information though. I didn't realize the soic 8 couldn't do one wire. It's a ATSHA204-SH-DA-B according to the listing.

EDIT:
Looking at the datasheet, it says that the SCL pin can be ignored for single wire interface. perhaps I'll wire another one up and give it a go.

Just connect, SDA to A3, correct?

[Anticimex]

@jsondag ok. From what I could read there are two different order codes for i2c and one-wire so I don't think you can take an i2c variant board and treat it as a one-wire board by just ignoring the scl pin. But if the board really is single-wire, the pinout will be the same as the i2c version, you can just ignore the scl pin as it is NC for the one-wire variant. And from the data sheet, SHDAB is i2c not "single-wire" so I am afraid you cannot use my libs for those. I only have drivers for single-wire chips i am afraid.

[TD22057]

I just received my chips today from digikey (part num ATSHA204A-STUCZ-TCT-ND) which are the small 3 pin versions. Holy cow these are things are tiny. These are going to be a lot more difficult than I expected to hand solder.

[Anticimex]

@TD22057 If you don't have a "inverse" tweezer to fixate the chip to the board, just glue it in place. With proper iron temp (at least 300 deg C, I use 330) it's very doable. Much easier than smd resistors/caps if you ask me (just don't forget to solder after you glue if you do that)

[TD22057]

@Anticimex said:

@TD22057 If you don't have a "inverse" tweezer to fixate the chip to the board, just glue it in place. With proper iron temp (at least 300 deg C, I use 330) it's very doable. Much easier than smd resistors/caps if you ask me (just don't forget to solder after you glue if you do that)

Thanks - I'll try that as soon as everything arrives. Watching parts trickle in from aliexpress shippers is killing me (I think I've become spoiled on Amazon prime shipping).

[tomkxy]

First of all I would like to thank Anticimex for this great piece of work!

I am playing around with signing right now. I set up a temp & humid sensor with soft signing support as well as a MQTT gateway with soft signing.

In principle it seems to work. At least sensor values are published to the MQTT broker. However, in the gateway output I see nonce tr errors and sign failures.

Started!
0;0;3;0;9;read: 21-21-0 s=1,c=3,t=16,pt=0,l=0,sg=0:
0;0;3;0;9;send: 0-0-21-21 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:0107FD8
0;0;3;0;9;read: 21-21-0 s=1,c=1,t=0,pt=7,l=5,sg=1:25.1
publish: MyMQTT/21/1/V_TEMP 25.1
0;0;3;0;9;read: 21-21-0 s=2,c=3,t=16,pt=0,l=0,sg=0:
0;0;3;0;9;send: 0-0-21-21 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:012B06D
0;0;3;0;9;send: 0-0-21-21 s=1,c=3,t=16,pt=0,l=0,sg=0,st=ok:
0;0;3;0;9;read: 21-21-0 s=2,c=1,t=1,pt=7,l=5,sg=1:59.0
publish: MyMQTT/21/2/V_HUM 59.0
0;0;3;0;9;sign fail
0;0;3;0;9;send: 0-0-21-21 s=2,c=3,t=16,pt=0,l=0,sg=0,st=fail:
0;0;3;0;9;nonce tr err
0;0;3;0;9;read: 21-21-0 s=255,c=3,t=17,pt=6,l=25,sg=0:01D5F523C2778AA
0;0;3;0;9;read: 21-21-0 s=1,c=3,t=16,pt=0,l=0,sg=0:
0;0;3;0;9;send: 0-0-21-21 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:0104881
0;0;3;0;9;read: 21-21-0 s=1,c=1,t=0,pt=7,l=5,sg=1:25.1
publish: MyMQTT/21/1/V_TEMP 25.1
0;0;3;0;9;read: 21-21-0 s=2,c=3,t=16,pt=0,l=0,sg=0:
0;0;3;0;9;send: 0-0-21-21 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:019A79B
0;0;3;0;9;send: 0-0-21-21 s=1,c=3,t=16,pt=0,l=0,sg=0,st=ok:
0;0;3;0;9;read: 21-21-0 s=2,c=1,t=1,pt=7,l=5,sg=1:59.2
publish: MyMQTT/21/2/V_HUM 59.2
0;0;3;0;9;sign fail
0;0;3;0;9;send: 0-0-21-21 s=2,c=3,t=16,pt=0,l=0,sg=0,st=fail:
0;0;3;0;9;nonce tr err
0;0;3;0;9;read: 21-21-0 s=255,c=3,t=17,pt=6,l=25,sg=0:0120D83204D1A06
0;0;3;0;9;read: 21-21-0 s=1,c=3,t=16,pt=0,l=0,sg=0:
0;0;3;0;9;send: 0-0-21-21 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:01AB761
0;0;3;0;9;read: 21-21-0 s=1,c=1,t=0,pt=7,l=5,sg=1:25.1
publish: MyMQTT/21/1/V_TEMP 25.1
0;0;3;0;9;read: 21-21-0 s=2,c=3,t=16,pt=0,l=0,sg=0:
0;0;3;0;9;send: 0-0-21-21 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:013CD41
0;0;3;0;9;read: 21-21-0 s=2,c=1,t=1,pt=7,l=5,sg=1:59.3
publish: MyMQTT/21/2/V_HUM 59.3
0;0;3;0;9;send: 0-0-21-21 s=1,c=3,t=16,pt=0,l=0,sg=0,st=fail:
0;0;3;0;9;nonce tr err
0;0;3;0;9;send: 0-0-21-21 s=2,c=3,t=16,pt=0,l=0,sg=0,st=fail:
0;0;3;0;9;nonce tr err

I am somehow irritated concerning those failures logged after the publish logs. They seem not to be related to receiving and publishing the sensor data. What can be source of that? Why might the gateway trying to transmit?

A second question is related to personalization of the ATSHA204. Am I right that the key is displayed in the SlotConfig00 - SlotConfig0F?

The second personalization step fails with the message "Data lock failed". What could be the reason for that?

[Anticimex]

@tomkxy Thanks.
"nonce tr err" is sent if sendRoute() fails to send a nonce request. So it suggests the radio link is somewhat unreliable and that in turn causes signing to fail. From what I can see you want to sign messages in both directions, so maybe traimsission from the MQTT gateway is not as reliable as reception? message type 16 (command 3) is a nonce request and sometimes it succeeds

0;0;3;0;9;send: 0-0-21-21 s=1,c=3,t=16,pt=0,l=0,sg=0,st=ok:

and sometimes it fail (with that message as consequence):

0;0;3;0;9;send: 0-0-21-21 s=2,c=3,t=16,pt=0,l=0,sg=0,st=fail:
0;0;3;0;9;nonce tr err

But the failing one is to a different sensor (2) than the first one (1), so perhaps that node is down or at a long range? But last in your log you also fail a transmission to sensor 1. In any case, the nonce tr err is because of failing radio transmissions and as such not a signing issue as such. Sometimes it works (when there are no radio issues:

0;0;3;0;9;read: 21-21-0 s=1,c=3,t=16,pt=0,l=0,sg=0: (request for nonce to GW)
0;0;3;0;9;send: 0-0-21-21 s=255,c=3,t=17,pt=6,l=25,sg=0,st=ok:01AB761 (GW sends nonce back)
0;0;3;0;9;read: 21-21-0 s=1,c=1,t=0,pt=7,l=5,sg=1:25.1 (signed message with that nonce is received)
publish: MyMQTT/21/1/V_TEMP 25.1 (message/signature is accepted and forwarded)

"sign fail" is typically reported if nonce exchange times out. You can adjust the timeout using MY_VERIFICATION_TIMEOUT_MS if you have many hops or for other reasons cannot process messages fast enough.
I have not tested MQTT gateway myself, and perhaps it uses the MySensors library differently but I have tested both serial and ethernet gateway. But in this case, it probably fails because you never got the request for the nonce out so your sender never got a chance to send a nonce back, and therefore signing cannot succeed.
So the key is to determine why the transmission of the nonce request fail (a radio problem since you get st=fail). Or implement a retransmission at a higher level in your sketch if the radio link is "lossy". The signing backend assumes a working channel and do not handle retransmission for you on failures.

Regarding personalization, key is never displayed. It is stored in slot 0. Slotconfigs are used to set the permissions of the slots. They don't store keys themselves.
You should get the reason for the data lock failure in the code that is sent together with the message. It is the ATSHA circuit itself that provide this value (or the library communicating with it in case of communication problem).

[tomkxy]

Thanks for your fast reply. Regarding the communication failures: I was testing this with both transmitters lying side by side. Do you have any idea why the gateway might want to have a nonce. Is it part of the sensor protocol?
The two sensors displayed in the log are basically the same sensor (DHT22 providing temp and humidity).

With respect to the key topic: The first personalization step provides the following output:

Device revision: 00020009
Device serial:   {0x01,0x23,0x48,0xC9,0x51,0x6A,0x1A,0x06,0xEE}
012348C9516A1A06EE
Chip configuration:
           SN[0:1]           |         SN[2:3]           | 01   23 | 48   C9   
                          Revnum                         | 00   09   04   00   
                          SN[4:7]                        | 51   6A   1A   06   
    SN[8]    |  Reserved13   | I2CEnable | Reserved15    | EE | 13 | 00 | 00   
  I2CAddress |  TempOffset   |  OTPmode  | SelectorMode  | C8 | 00 | 55 | 00   
         SlotConfig00        |       SlotConfig01        | 8F   80 | 80   A1   
         SlotConfig02        |       SlotConfig03        | 82   E0 | A3   60   
         SlotConfig04        |       SlotConfig05        | 94   40 | A0   85   
         SlotConfig06        |       SlotConfig07        | 86   40 | 87   07   
         SlotConfig08        |       SlotConfig09        | 0F   00 | 89   F2   
         SlotConfig0A        |       SlotConfig0B        | 8A   7A | 0B   8B   
         SlotConfig0C        |       SlotConfig0D        | 0C   4C | DD   4D   
         SlotConfig0E        |       SlotConfig0F        | C2   42 | AF   8F   
  UseFlag00  | UpdateCount00 | UseFlag01 | UpdateCount01 | FF | 00 | FF | 00   
  UseFlag02  | UpdateCount02 | UseFlag03 | UpdateCount03 | FF | 00 | FF | 00   
  UseFlag04  | UpdateCount04 | UseFlag05 | UpdateCount05 | FF | 00 | FF | 00   
  UseFlag06  | UpdateCount06 | UseFlag07 | UpdateCount07 | FF | 00 | FF | 00   
                      LastKeyUse[0:3]                    | FF   FF   FF   FF   
                      LastKeyUse[4:7]                    | FF   FF   FF   FF   
                      LastKeyUse[8:B]                    | FF   FF   FF   FF   
                      LastKeyUse[C:F]                    | FF   FF   FF   FF   
  UserExtra  |    Selector   | LockValue |  LockConfig   | 00 | 00 | 55 | 55  

Can I provide any key in the second personalization step or does it have to be same than in the first step being generated?

[Anticimex]

@tomkxy The protocol is described in the first post. Any node that is sending a signed message to another node (that has requested to get signed messages) will ask for a nonce. So you have configured your gateway to require signed messages as well as your node.
Regarding the personalization, I do t understand your question. The output is the chip configuration. It does not list any keys. I am not sure I understand what you mean by second step. You can generate a random key and that key you store and after you store it you have an option to also lock it, but then you have no way of ever changing it. I have documented the personalization flow also in the sketch itself.

[Anticimex]

I should emphasise that if you personalize multiple atsha:s you have to have the same key stored in all of them. But the sketch offer to generate a random key (but you can skip that and use any key or password you like). But the key must be the same for all members of a secure network that want to talk to each other.

[tomkxy]

@Anticimex: The output shown is related to the sketch configuration you described in your description:

Set the following sketch configuration of the personalizer:
Enable LOCK_CONFIGURATION
Disable LOCK_DATA
Enable SKIP_KEY_STORAGE
Disable SKIP_UART_CONFIGURATION
Disable USER_KEY_DATA

Execute the sketch on the “master” device to obtain a randomized key. Save this key to >a secure location and keep it confidential so that you can retrieve it if you need to
personalize more devices later on.

I was wondering that I did not see any explicit reference to a key. So may be I just have to retry.

I should emphasise that if you personalize multiple atsha:s you have to have the same >key stored in all of them. But the sketch offer to generate a random key (but you can >skip that and use any key or password you like). But the key must be the same for all >members of a secure network that want to talk to each other.
Thanks for that clarification. I was aware of that.

[Anticimex]

@tomkxy are you sure you have enabled all those options? According to the dump, your configuration is still not locked and therefore no randomized key can be generated. Are there no more output from the sketch than that? You mentioned failure to lock data zone but I cannot see that message. And you cannot lock datazone without locking configuration.
After the sketch locks configuration it will print a randomized key in the log.

[tomkxy]

@Anticimex: Thanks for your patience and your support.
I rerun it with the output and sketch configuration listed below. May be the reason that it is not working that my poor soldering skills bricked the device.

Extract from sketch (1st run):

#include <sha204_library.h>
#include <sha204_lib_return_codes.h>

// The pin the ATSHA204 is connected on
#define ATSHA204_PIN 17 // A3

// Uncomment this to enable locking the configuration zone.
// *** BE AWARE THAT THIS PREVENTS ANY FUTURE CONFIGURATION CHANGE TO THE CHIP ***
// It is still possible to change the key, and this also enable random key generation
#define LOCK_CONFIGURATION

// Uncomment this to enable locking the data zone.
// *** BE AWARE THAT THIS PREVENTS THE KEY TO BE CHANGED ***
// It is not required to lock data, key cannot be retrieved anyway, but by locking
// data, it can be guaranteed that nobody even with physical access to the chip,
// will be able to change the key.
//#define LOCK_DATA

// Uncomment this to skip key storage (typically once key has been written once)
#define SKIP_KEY_STORAGE

// Uncomment this to skip key data storage (once configuration is locked, key
// will aways randomize)
// Uncomment this to skip key generation and use 'user_key_data' as key instead.
//#define USER_KEY_DATA

// Uncomment this for boards that lack UART
// IMPORTANT: No confirmation will be required for locking any zones with this
// configuration!
// Also, key generation is not permitted in this mode as there is no way of
// presenting the generated key.
//#define SKIP_UART_CONFIRMATION

#if defined(SKIP_UART_CONFIRMATION) && !defined(USER_KEY_DATA)
#error You have to define USER_KEY_DATA for boards that does not have UART
#endif
…

Output:

ATSHA204 personalization sketch for MySensors usage.
----------------------------------------------------
Device revision: 00020009
Device serial:   {0x01,0x23,0x48,0xC9,0x51,0x6A,0x1A,0x06,0xEE}
012348C9516A1A06EE
Skipping configuration write and lock (configuration already locked).
Chip configuration:
           SN[0:1]           |         SN[2:3]           | 01   23 | 48   C9   
                          Revnum                         | 00   09   04   00   
                          SN[4:7]                        | 51   6A   1A   06   
    SN[8]    |  Reserved13   | I2CEnable | Reserved15    | EE | 13 | 00 | 00   
  I2CAddress |  TempOffset   |  OTPmode  | SelectorMode  | C8 | 00 | 55 | 00   
         SlotConfig00        |       SlotConfig01        | 8F   80 | 80   A1   
         SlotConfig02        |       SlotConfig03        | 82   E0 | A3   60   
         SlotConfig04        |       SlotConfig05        | 94   40 | A0   85   
         SlotConfig06        |       SlotConfig07        | 86   40 | 87   07   
         SlotConfig08        |       SlotConfig09        | 0F   00 | 89   F2   
         SlotConfig0A        |       SlotConfig0B        | 8A   7A | 0B   8B   
         SlotConfig0C        |       SlotConfig0D        | 0C   4C | DD   4D   
         SlotConfig0E        |       SlotConfig0F        | C2   42 | AF   8F   
  UseFlag00  | UpdateCount00 | UseFlag01 | UpdateCount01 | FF | 00 | FF | 00   
  UseFlag02  | UpdateCount02 | UseFlag03 | UpdateCount03 | FF | 00 | FF | 00   
  UseFlag04  | UpdateCount04 | UseFlag05 | UpdateCount05 | FF | 00 | FF | 00   
  UseFlag06  | UpdateCount06 | UseFlag07 | UpdateCount07 | FF | 00 | FF | 00   
                      LastKeyUse[0:3]                    | FF   FF   FF   FF   
                      LastKeyUse[4:7]                    | FF   FF   FF   FF   
                      LastKeyUse[8:B]                    | FF   FF   FF   FF   
                      LastKeyUse[C:F]                    | FF   FF   FF   FF   
  UserExtra  |    Selector   | LockValue |  LockConfig   | 00 | 00 | 55 | 00   
Disable SKIP_KEY_STORAGE to store key.
Data not locked. Define LOCK_DATA to lock for real.
--------------------------------
Personalization is now complete.
Configuration is LOCKED
Data is UNLOCKED

Second run with the following sketch settings (key used removed):

#include <sha204_library.h>
#include <sha204_lib_return_codes.h>

// The pin the ATSHA204 is connected on
#define ATSHA204_PIN 17 // A3

// Uncomment this to enable locking the configuration zone.
// *** BE AWARE THAT THIS PREVENTS ANY FUTURE CONFIGURATION CHANGE TO THE CHIP ***
// It is still possible to change the key, and this also enable random key generation
#define LOCK_CONFIGURATION

// Uncomment this to enable locking the data zone.
// *** BE AWARE THAT THIS PREVENTS THE KEY TO BE CHANGED ***
// It is not required to lock data, key cannot be retrieved anyway, but by locking
// data, it can be guaranteed that nobody even with physical access to the chip,
// will be able to change the key.
#define LOCK_DATA

// Uncomment this to skip key storage (typically once key has been written once)
//#define SKIP_KEY_STORAGE

// Uncomment this to skip key data storage (once configuration is locked, key
// will aways randomize)
// Uncomment this to skip key generation and use 'user_key_data' as key instead.
#define USER_KEY_DATA

// Uncomment this for boards that lack UART
// IMPORTANT: No confirmation will be required for locking any zones with this
// configuration!
// Also, key generation is not permitted in this mode as there is no way of
// presenting the generated key.
#define SKIP_UART_CONFIRMATION

#if defined(SKIP_UART_CONFIRMATION) && !defined(USER_KEY_DATA)
#error You have to define USER_KEY_DATA for boards that does not have UART
#endif

#ifdef USER_KEY_DATA
#define MY_HMAC_KEY 0x…

const uint8_t user_key_data[32] = {MY_HMAC_KEY};
#endif
const int sha204Pin = ATSHA204_PIN;
atsha204Class sha204(sha204Pin);

Device revision: 00020009
Device serial:   {0x01,0x23,0x48,0xC9,0x51,0x6A,0x1A,0x06,0xEE}
012348C9516A1A06EE
Skipping configuration write and lock (configuration already locked).
Chip configuration:
           SN[0:1]           |         SN[2:3]           | 01   23 | 48   C9   
                          Revnum                         | 00   09   04   00   
                          SN[4:7]                        | 51   6A   1A   06   
    SN[8]    |  Reserved13   | I2CEnable | Reserved15    | EE | 13 | 00 | 00   
  I2CAddress |  TempOffset   |  OTPmode  | SelectorMode  | C8 | 00 | 55 | 00   
         SlotConfig00        |       SlotConfig01        | 8F   80 | 80   A1   
         SlotConfig02        |       SlotConfig03        | 82   E0 | A3   60   
         SlotConfig04        |       SlotConfig05        | 94   40 | A0   85   
         SlotConfig06        |       SlotConfig07        | 86   40 | 87   07   
         SlotConfig08        |       SlotConfig09        | 0F   00 | 89   F2   
         SlotConfig0A        |       SlotConfig0B        | 8A   7A | 0B   8B   
         SlotConfig0C        |       SlotConfig0D        | 0C   4C | DD   4D   
         SlotConfig0E        |       SlotConfig0F        | C2   42 | AF   8F   
  UseFlag00  | UpdateCount00 | UseFlag01 | UpdateCount01 | FF | 00 | FF | 00   
  UseFlag02  | UpdateCount02 | UseFlag03 | UpdateCount03 | FF | 00 | FF | 00   
  UseFlag04  | UpdateCount04 | UseFlag05 | UpdateCount05 | FF | 00 | FF | 00   
  UseFlag06  | UpdateCount06 | UseFlag07 | UpdateCount07 | FF | 00 | FF | 00   
                      LastKeyUse[0:3]                    | FF   FF   FF   FF   
                      LastKeyUse[4:7]                    | FF   FF   FF   FF   
                      LastKeyUse[8:B]                    | FF   FF   FF   FF   
                      LastKeyUse[C:F]                    | FF   FF   FF   FF   
  UserExtra  |    Selector   | LockValue |  LockConfig   | 00 | 00 | 55 | 00   
Using this user supplied key:
#define MY_HMAC_KEY 0x…..
Writing key to slot 0...
Data lock failed. Response: D3
Halting!

[Anticimex]

@tomkxy I'm on phone so I cannot check the response code right now but you could look it up in the atsha datasheet or in the software. I have not tried to lock data myself and I do not recommend it because it makes it impossible to change the key later on if it is comprised. From what I can see, everything looks good except the locking of the datazone (your key)

[Anticimex]

@tomkxy I should add that from the logs I see that you do have successfully stored your key, so unless you really want to lock down the key, personalization is done and successful.

[tomkxy]

@Anticimex: Thanks a lot. I didn't intend to lock the data zone.

Btw, I did some tests regarding the nonce failure the gateway showed. I think the reason is rather simple. The gateway tried - for reasons I still don't understand - send data to the sensor for which it tried to get a nonce. The sensor however was powered down which is probably the reason why no nonce was sent. At least this error didn't show up when I removed the power down and replaced it by a simple call to delay.

[Anticimex]

@tomkxy Ah, I see. Well then that matter should be sorted.
Regarding your radio issue, yes, if the GW wants to send data to your node, it needs to be up&running. From the log you sent, I cannot determine what data the GW tried to send though, since it (because your node has told the GW it require it) wants to sign the message to send, and fails, the actual message is never showed in the log.
Perhaps you could send a log with your node continuously powered and we could see at least the type of message your GW tries to send. That could help to identify why it tries to send the message in the first place.

[tomkxy]

@Anticimex: I started a new thread http://forum.mysensors.org/topic/1782/gateway-is-sending-command-1-type-1-c-1-t-1-message-to-sensor
because it looks like it is not really related to signing.
In the post I included the log output of the message for which the nonce is being requested. It looks like the gateway node is sending back the humidity with a c=1 command. Somehow this does not make sense to me.

[Anticimex]

@tomkxy I see. Perhaps you request ACK or something like that. But ACKs are not signed. I have seen to that, so probably not. I am not too well into the non-signing aspects of the network though, so hopefully someone else perhaps can give a hint on what's wrong. Could be the sketch itself. I am glad we sorted the signing issues anyway

[tomkxy]

@Anticimex: I found the issue with respect to the gateway sending a message back to the sensor. This is due to the gateway subscribing to all topics on the MQTT broker. Once it receives a message it sends it to the sensor as SET message. I have to resolve that one.

I think, I still have an issue with signing which I need to get sorted myself first. I will let you know.

[Anticimex]

@tomkxy Alright. Good to know that you found the root cause for the message issue.
Regarding signing, from what I could see at least some signed messages did come through, so you should at least have the proper shared states (keys) and configs.

[tomkxy]

I did a few test with my ATSHA204. I could get it working on an Uno. The I took the chip and placed it on my breadboard with my ProMini. I tried to run the SHA Personaliser Sketch again just to make sure that the chip is somehow working and I received an error message that the device cannot be woken up.

Since it worked on the Uno (including running the SHA Personalizer) it must be related to the ProMini (3.3V / 8 MHz).

To what pin do I have to connect the data line?

[Anticimex]

@tomkxy That sounds strange. I have run on ProMini 3.3V/8MHz without issues.
Default is to use A3 for the ATSHA204A. It does not really matter which pin you connect it as long as it is usable for digital I/O (and update MY_ATSHA204_PIN accordingly or provide your pin to the MySigningAtsha204 constructor).

[tomkxy]

How do I change the pin definition if I use A3 on the ProMini?

[Anticimex]

A3 is default setting. You can find the definition of MY_ATSHA204_PIN in MyConfig.h.

[tomkxy]

Yep, it is

`#define MY_ATSHA204_PIN 17 // A3 - pin where ATSHA204 is attached
``
I am just irritate by the 17. So this does map to A3 even on ProMini?

[Anticimex]

@tomkxy yes it does.

[tomkxy]

@Anticimex Unfortunately, I am now in a total state of mess. It seems that nothing works anymore the moment I turn on any signing. For sure, I have a problem with ATSHA204 and the ProMini which I need to sort out separately. I tried them all with my Uno (reading out the config) which worked. When I try to just read the config with my ProMini I get the error cannot wake up device.

But now, even soft signing did not work any longer (it worked first, than I made changes and afte. I just receive nonce transmission errors from the sensor to the gateway although both radios are side by side and transmission without signing works perfectly.

Btw, what is stored in the EPROM and under what circumstances do I have to clear EPROM first. I changed between the various signing feature (soft signing, HW signing, signing required etc.) back and forth and it seems that this info is stored in EPROM and somehow not cleared?? How does the sensor know that the gateway requires signing? Will it get the info out of EPROM? Will this information be updated during presentation?

I think you did a great job on that implementation, the more frustrating it is that I cannot get it to work reliably. I am out for today. May be I find some time tomorrow getting anot ProMini prepared to checkout the issue with the chips.

[Anticimex]

@tomkxy The information stored in EEPROM is not specific to any backend. It just informs the node what other nodes require signed messages. At startup, a node broadcasts it's preference to the gateway which then updates it's EEPROM table and replies with its own preference back so the node knows if the gateway wants signed messages as well. If the preferences differ from what is stored in EEPROM already, it is the updated preferences that will replace the stored preferences.
The usecases for this is if you were to deploy a new sensor which require signing, it would inform gateway of this at startup. But if you restart your gateway it would loose this unless it was stored in EEPROM (the same goes the other way around) so the EEPROM is used so that the signing rules in the network gets preserved even if nodes dissappear or restart.
If you suspect the EEPROM to contain corrupt data, you can clear it with the cleareeorom sketch/example.
The typical circumstance you need to clear EEPROM under is when you switch library version, and the reserved region of EEPROM in the library change (and you also use EEPROM in your sketch). Then the library might take some of your sketch data for "it's own" and that can/will lead to unpredictable results.

[scalz]

Just to say, that I tried some weeks ago ATSHA204A with pro mini and had no problem too. Maybe you should try clear eeprom and retry. Or try with another pro mini. But it should work I think.
And for those who want to test excellent work of @Anticimex and @Tekka, I have posted a simple/basic not expensive, 1.8$ for 3 breakouts here : https://oshpark.com/shared_projects/lvvxsHSW . It can be useful for existing nodes too.

See you soon

[TD22057]

@scalz said:

Just to say, that I tried some weeks ago ATSHA204A with pro mini and had no problem too. Maybe you should try clear eeprom and retry. Or try with another pro mini. But it should work I think.
And for those who want to test excellent work of @Anticimex and @Tekka, I have posted a simple/basic not expensive, 1.8$ for 3 breakouts here : https://oshpark.com/shared_projects/lvvxsHSW . It can be useful for existing nodes too.

Good idea - I'm finding soldering the ATSHA chips to be a real pain and a small breakout board would help with that. Is the eeprom pads on your board for people who are just using a mega328 chip? Doesn't a pro-mini already have enough eeprom on board?

[scalz]

@TDD22057: eeprom on the breakout is for the new ota in Mysensors (over the air upload sketch) . It is not related to authentication.
I made this cheap breakout firstly because I wanted to test the new Mysensors features easily. and I thought it could be useful in some specific case where you don't need to make a pcb for one specific usercase so you use a veroboard and so you can easily add these new features with less pain...
Happy it can help some people

[tomkxy]

Regarding my issue with the ATSHA204A, I did some testing with the following results. I soldered the ATSHA204A on a small breakout board and loaded the SHAPersonalizer sketch.

I tested three configurations:

1. Breakout wired to the ProMini without breadboard use
2. Breakout wired to a breadboard where the ProMini was plugged
3. Breakout and ProMini both plugged to breadboard and connected by wires.

Config 3 does not work!!!
That means the breadboard significantly changes the electrical characteristics. Is anybody on the forum who is able to explain that?

[scalz]

@tomkxy: it is strange. My first tests have been done in you config 3 (atsha breakout and promini both plugged to breadboard). And it worked well for me. Could it be your wire quality??? I had problems with some Dupont wire once...

[tomkxy]

Well , could be... I had to use different wires female - female, male - female, male - male (for config 3). For the male-male wire I have no other option at the moment.

However, I am glad it works now.

[Anticimex]

@tomkxy said:

Well , could be... I had to use different wires female - female, male - female, male - male (for config 3). For the male-male wire I have no other option at the moment.

However, I am glad it works now.

Good news! Regarding breadboard issues, it could be that you need an additional decoupling on the VCC line to the atsha204a. And also perhaps a pullup on the SDA signal. Breadboards usually put some more requirements on noise suppression.

[otto001]

Hi,

I seem to be too stupid (or drunk ) to get this working with MySigningAtsha204Soft
Is anyone willing to provide a working MyConfig.h and EthernetGateway.ino for this?
There seemed to be some changes and I am sure, I am just too dumb...

[tomkxy]

Here you go....However, I do not have EthernetGateway but a MQTTClientGateway. May be it is of help.
What is your specific problem?

myconfig.h (key changed)

/**
 * The MySensors Arduino library handles the wireless radio link and protocol
 * between your home built sensors/actuators and HA controller of choice.
 * The sensors forms a self healing radio network with optional repeaters. Each
 * repeater and gateway builds a routing tables in EEPROM which keeps track of the
 * network topology allowing messages to be routed to nodes.
 *
 * Created by Henrik Ekblad <henrik.ekblad@mysensors.org>
 * Copyright (C) 2013-2015 Sensnology AB
 * Full contributor list: https://github.com/mysensors/Arduino/graphs/contributors
 *
 * Documentation: http://www.mysensors.org
 * Support Forum: http://forum.mysensors.org
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * version 2 as published by the Free Software Foundation.
 */


#ifndef MyConfig_h
#define MyConfig_h
#include <stdint.h>

// Enable debug flag for debug prints. This will add a lot to the size of the final sketch but good
// to see what is actually is happening when developing
#define DEBUG

// Serial output baud rate (for debug prints and serial gateway)
#define BAUD_RATE 115200


/**********************************
*  Over the air firmware updates
***********************************/

// The following define enables the safe over-the-air firmware update feature
// which requires external flash and the DualOptiBoot bootloader.
// Note: You can still have OTA FW updates without external flash but it
// requires the MYSBootloader and disabled MY_OTA_FIRMWARE_FEATURE
//#define MY_OTA_FIRMWARE_FEATURE
// Slave select pin for external flash
#define MY_OTA_FLASH_SS 8
// Flash jdecid
#define MY_OTA_FLASH_JDECID 0x1F65


/**********************************
*  Information LEDs blinking
***********************************/
// This feature enables LEDs blinking on message receive, transmit
// or if some error occured. This was commonly used only in gateways,
// but now can be used in any sensor node. Also the LEDs can now be
// disabled in the gateway.

//#define WITH_LEDS_BLINKING

// The following setting allows you to inverse the blinking feature WITH_LEDS_BLINKING
// When WITH_LEDS_BLINKING_INVERSE is enabled LEDSs are normally turned on and switches
// off when blinking

//#define WITH_LEDS_BLINKING_INVERSE


// default LEDs blinking period in milliseconds
#define DEFAULT_LED_BLINK_PERIOD 300
// The RX LED default pin
#define DEFAULT_RX_LED_PIN 6
// The TX LED default pin
#define DEFAULT_TX_LED_PIN 5
// The Error LED default pin
#define DEFAULT_ERR_LED_PIN 4


/**********************************
*  Message Signing Settings
***********************************/
// Disable to completly disable signing functionality in library
#define MY_SIGNING_FEATURE

// Define a suitable timeout for a signature verification session
// Consider the turnaround from a nonce being generated to a signed message being received
// which might vary, especially in networks with many hops. 5s ought to be enough for anyone.
#define MY_VERIFICATION_TIMEOUT_MS 5000

// Enable to turn on whitelisting
// When enabled, a signing node will salt the signature with it's unique signature and nodeId.
// The verifying node will look up the sender in a local table of trusted nodes and
// do the corresponding salting in order to verify the signature.
// For this reason, if whitelisting is enabled on one of the nodes in a sign-verify pair, both
// nodes have to implement whitelisting for this to work.
// Note that a node can still transmit a non-salted message (i.e. have whitelisting disabled)
// to a node that has whitelisting enabled (assuming the receiver does not have a matching entry
// for the sender in it's whitelist)
//#define MY_SECURE_NODE_WHITELISTING

// MySigningAtsha204 default setting
#define MY_ATSHA204_PIN 17 // A3 - pin where ATSHA204 is attached

// MySigningAtsha204Soft default settings
#define MY_RANDOMSEED_PIN 7 // A7 - Pin used for random generation (do not connect anything to this)

// Key to use for HMAC calculation in MySigningAtsha204Soft (32 bytes)
#define MY_HMAC_KEY 0x10,0x23,0xBB,0x78,0x77,0x35,0xB0,0x01,0x70,0x47,0xF1,0xDE,0x21,0x94,0x54,0x67,0xEE,0x36,0x72,0x00,0x97,0x12,0xA0,0x0A,0x0F,0x09,0x03,0xE2,0x00,0x31,0xE4,0x41


/**********************************
*  NRF24L01 Driver Defaults
***********************************/
#define RF24_CE_PIN		   9
#define RF24_CS_PIN		   10
#define RF24_PA_LEVEL 	   RF24_PA_MAX
#define RF24_PA_LEVEL_GW   RF24_PA_LOW
// RF channel for the sensor net, 0-127
#define RF24_CHANNEL	   76
//RF24_250KBPS for 250kbs, RF24_1MBPS for 1Mbps, or RF24_2MBPS for 2Mbps
#define RF24_DATARATE 	   RF24_250KBPS
// This is also act as base value for sensor nodeId addresses. Change this (or channel) if you have more than one sensor network.
#define RF24_BASE_RADIO_ID ((uint64_t)0xA8A8E1FC00LL)

// Enable SOFTSPI for NRF24L01 when using the W5100 Ethernet module
//#define SOFTSPI
#ifdef SOFTSPI
	// Define the soft SPI pins used for NRF radio
	const uint8_t SOFT_SPI_MISO_PIN = 16;
    const uint8_t SOFT_SPI_MOSI_PIN = 15;
    const uint8_t SOFT_SPI_SCK_PIN = 14;
#endif


/**********************************
*  RFM69 Driver Defaults
***********************************/
// Default network id. Use the same for all nodes that will talk to each other
#define RFM69_NETWORKID     100

// Default frequency to use. This must match the hardware version of the RFM69 radio (uncomment one):
// #define RFM69_FREQUENCY   RF69_433MHZ
#define RFM69_FREQUENCY   RF69_868MHZ
//#define FREQUENCY     RF69_915MHZ

// Enable this for encryption of packets
//#define RFM69_ENABLE_ENCRYPTION
#define RFM69_ENCRYPTKEY    "sampleEncryptKey" //exactly the same 16 characters/bytes on all nodes!


#endif

/* 				MyMQTT Client Gateway 0.1b
 Created by Norbert Truchsess <norbert.truchsess@t-online.de>
 Based on MyMQTT-broker gateway created by Daniel Wiegert <daniel.wiegert@gmail.com>
 Based on MySensors Ethernet Gateway by Henrik Ekblad <henrik.ekblad@gmail.com>
 http://www.mysensors.org
 Requires MySensors lib 1.4b
 * Change below; TCP_IP, TCP_PORT, TCP_MAC
 This will listen on your selected TCP_IP:TCP_PORT below, Please change TCP_MAC your liking also.
 *1 -> NOTE: Keep first byte at x2, x6, xA or xE (replace x with any hex value) for using Local Ranges.
 *2 You can use standard pin set-up as MySensors recommends or if you own a IBOARD you may change
 the radio-pins below if you hardware mod your iBoard. see [URL BELOW] for more details.
 http://forum.mysensors.org/topic/224/iboard-cheap-single-board-ethernet-arduino-with-radio/5
 * Don't forget to look at the definitions in libraries\MySensors\MyMQTT.h!
 define TCPDUMP and connect serial interface if you have problems, please write on
 http://forum.mysensors.org/ and explain your problem, include serial output. Don't forget to
 turn on DEBUG in libraries\MySensors\MyConfig.h also.
 MQTT_FIRST_SENSORID is for 'DHCP' server in MyMQTT. You may limit the ID's with FIRST and LAST definition.
 If you want your manually configured below 20 set MQTT_FIRST_SENSORID to 20.
 To disable: set MQTT_FIRST_SENSORID to 255.
 MQTT_BROKER_PREFIX is the leading prefix for your nodes. This can be only one char if like.
 MQTT_SEND_SUBSCRIPTION is if you want the MyMQTT to send a empty payload message to your nodes.
 This can be useful if you want to send latest state back to the MQTT client. Just check if incoming
 message has any length or not.
 Example: if (msg.type==V_LIGHT && strlen(msg.getString())>0) otherwise the code might do strange things.
 * Address-layout is : [MQTT_BROKER_PREFIX]/[NodeID]/[SensorID]/V_[SensorType]
 NodeID and SensorID is uint8 (0-255) number.
 Last segment is translation of the sensor type, look inside MyMQTT.cpp for the definitions.
 User can change this to their needs. We have also left some space for custom types.
 Special: (sensor 255 reserved for special commands)
 You can receive a node sketch name with MyMQTT/20/255/V_Sketch_name (or version with _version)
 To-do:
 Special commands : clear or set EEPROM Values, Send REBOOT and Receive reboot for MyMQTT itself.
 Be able to send ACK so client returns the data being sent.
 ... Please come with ideas!
 What to do with publish messages.
 Test in more MQTT clients, So far tested in openhab and MyMQTT for Android (Not my creation)
 - http://www.openhab.org/
 - https://play.google.com/store/apps/details?id=at.tripwire.mqtt.client&hl=en
 ... Please notify me if you use this broker with other software.
 * How to set-up Openhab and MQTTGateway:
 http://forum.mysensors.org/topic/303/mqtt-broker-gateway

 Changes by Thomas Krebs <thkrebs@gmx.de>
 - Add signing support from MySensors 1.5 and update for MySensors 1.5;
 - Restructured code back to a C like implementation following the existing MQTTGateway
 */

#include <SPI.h>
#include <MySensor.h>
#include "MyMQTTClient.h"
#include "PubSubClient.h"
#include <Ethernet.h>
#include <DigitalIO.h>
#include <MsTimer2.h>
#include <Time.h>


#ifdef MY_SIGNING_FEATURE
#include <MySigningNone.h>
#include <MySigningAtsha204Soft.h>
#include <MySigningAtsha204.h>
#endif

//#define DSRTC
#ifdef DSRTC
#include <Wire.h>
#include <DS1307RTC.h>  // a basic DS1307 library that returns time as a time_t
#endif


/*
 * To configure MQTTClientGateway.ino to use an ENC28J60 based board include
 * 'UIPEthernet.h' (SPI.h required for MySensors anyway). The UIPEthernet-library can be downloaded
 * from: https://github.com/ntruchsess/arduino_uip
 */

//#include <UIPEthernet.h>
/*
 * To execute MQTTClientGateway.ino on Yun uncomment Bridge.h and YunClient.h.
 * Do not include Ethernet.h or SPI.h in this case.
 * On Yun there's no need to configure local_ip and mac in the sketch
 * as this is configured on the linux-side of Yun.
 */

//#include <Bridge.h>
//#include <YunClient.h>
// * Use this for IBOARD modded to use standard MISO/MOSI/SCK, see note *1 above!
/*
 #define RADIO_CE_PIN        3			// radio chip enable
 #define RADIO_SPI_SS_PIN    8			// radio SPI serial select
 #define RADIO_ERROR_LED_PIN A2  		// Error led pin
 #define RADIO_RX_LED_PIN    A1  		// Receive led pin
 #define RADIO_TX_LED_PIN    A0  		// the PCB, on board LED*/

// * Use this for default configured pro mini / nano etc :
//

//#define RADIO_CE_PIN        5		// radio chip enable
//#define RADIO_SPI_SS_PIN    6		// radio SPI serial select
//#define RADIO_ERROR_LED_PIN 7		// Error led pin
//#define RADIO_RX_LED_PIN    8		// Receive led pin
//#define RADIO_TX_LED_PIN    9		// the PCB, on board LED*/

// CE_PIN and SPI_SS_PIN for Mega
#define RADIO_CE_PIN        48			// radio chip enable
#define RADIO_SPI_SS_PIN    49			// radio SPI serial select
#define RADIO_ERROR_LED_PIN A2  		// Error led pin
#define RADIO_RX_LED_PIN    A1  		// Receive led pin
#define RADIO_TX_LED_PIN    A0  		// the PCB, on board LED*/

//replace with ip of server you want to connect to, comment out if using 'remote_host'
uint8_t remote_ip[] =  { 192, 168, 178, 74 };  // Mosquitto broker

//replace with hostname of server you want to connect to, comment out if using 'remote_ip'
//char* remote_ip = "server.local";
//replace with the port that your server is listening on
#define remote_port 1883
//replace with arduinos ip-address. Comment out if Ethernet-startup should use dhcp. Is ignored on Yun
uint8_t local_ip[] = {192, 168, 178, 11};
//replace with ethernet shield mac. It's mandatory every device is assigned a unique mac. Is ignored on Yun
uint8_t mac[] = { 0xA2, 0xAE, 0xAD, 0xA0, 0xA0, 0xA2 };

//////////////////////////////////////////////////////////////////

#if defined remote_ip && defined remote_host
#error "cannot define both remote_ip and remote_host at the same time!"
#endif

#ifdef _YUN_CLIENT_H_
YunClient ethClient;
#else
EthernetClient ethClient;
#endif

////////////////////////////////////////////////////////////////
// NRFRF24L01 radio driver (set low transmit power by default)
MyTransportNRF24 transport(RADIO_CE_PIN, RADIO_SPI_SS_PIN, RF24_PA_LEVEL_GW);
//MyTransportRFM69 transport;

// Message signing driver (signer needed if MY_SIGNING_FEATURE is turned on in MyConfig.h)
//MySigningNone signer;
MySigningAtsha204Soft signer;
//MySigningAtsha204 signer;

// Hardware profile
MyHwATMega328 hw;

MyMessage msg;
char convBuf[MAX_PAYLOAD * 2 + 1];
uint8_t buffsize;
char buffer[MQTT_MAX_PACKET_SIZE];


////////////////////////////////////////////////////////////////

volatile uint8_t countRx;
volatile uint8_t countTx;
volatile uint8_t countErr;


////////////////////////////////////////////////////////////////

void processMQTTMessages(char* topic, byte* payload, unsigned int length);
PubSubClient client(remote_ip, remote_port, processMQTTMessages, ethClient);

////////////////////////////////////////////////////////////////


// Declare and initialize MySensor instance
// Construct MyMQTTClient (signer needed if MY_SIGNING_FEATURE is turned on in MyConfig.h, if signing
// feature not to be used, uncomment)
// To use LEDs blinking, uncomment WITH_LEDS_BLINKING in MyConfig.h
MySensor gw(transport, hw
#ifdef MY_SIGNING_FEATURE
            , signer
#endif
#ifdef WITH_LEDS_BLINKING
            , RADIO_RX_LED_PIN, RADIO_TX_LED_PIN, RADIO_ERROR_LED_PIN
#endif
           );

/*
 * setup
 */
void setup()
{
  countRx = 0;
  countTx = 0;
  countErr = 0;

  Ethernet.begin(mac, local_ip);
  //Bridge.begin();
  delay(1000);   // Wait for Ethernet to get configured.

  begin();
}

/*
 * loop
 */
void loop()
{
  if (!client.connected())
  {
    client.connect("MySensor");
    client.subscribe(MQTT_TOPIC_MASK);
  }
  client.loop();
  gw.process();
}

/*
 * processRadioMessage
 *
 * Receives radio message, parses it and forwards it to the MQTT broker
 */
void processRadioMessage(const MyMessage &message)
{ 
    rxBlink(1);
    sendMQTT(message);
}

/*
 * sendMQTT
 * Handles processing of radio messages and eventually publishes it to the MQTT broker
 */
void sendMQTT(const MyMessage &inMsg)
{
  MyMessage msg = inMsg;
  buffsize = 0;
  if (!client.connected())
    return;			//We have no connections - return
  if (msg.isAck())
  {
#ifdef DEBUG
    Serial.println("msg is ack!");
#endif
    if (msg.sender == 255 && mGetCommand(msg) == C_INTERNAL
        && msg.type == I_ID_REQUEST)
    {
      // TODO: sending ACK request on id_response fucks node up. doesn't work.
      // The idea was to confirm id and save to EEPROM_LATEST_NODE_ADDRESS.
    }
  }
  else
  {
    // we have to check every message if its a newly assigned id or not.
    // Ack on I_ID_RESPONSE does not work, and checking on C_PRESENTATION isn't reliable.
    uint8_t newNodeID = gw.loadState(EEPROM_LATEST_NODE_ADDRESS) + 1;
    if (newNodeID <= MQTT_FIRST_SENSORID)
      newNodeID = MQTT_FIRST_SENSORID;
    if (msg.sender == newNodeID)
    {
      gw.saveState(EEPROM_LATEST_NODE_ADDRESS, newNodeID);
    }
    if (mGetCommand(msg) == C_INTERNAL)
    {
      if (msg.type == I_CONFIG)
      {
        txBlink(1);
        if (!gw.sendRoute(
              build(msg, GATEWAY_ADDRESS, msg.sender, 255, C_INTERNAL,
                    I_CONFIG, 0).set("M")))
          errBlink(1);
      }
      else if (msg.type == I_TIME)
      {
#ifdef DEBUG
        Serial.println("I_TIME requested!");
#endif
        txBlink(1);
        if (!gw.sendRoute(
              build(msg, GATEWAY_ADDRESS, msg.sender, 255, C_INTERNAL,
                    I_TIME, 0).set(now())))
          errBlink(1);
      }
      else if (msg.type == I_ID_REQUEST && msg.sender == 255)
      {
        uint8_t newNodeID = gw.loadState(EEPROM_LATEST_NODE_ADDRESS) + 1;
        if (newNodeID <= MQTT_FIRST_SENSORID)
          newNodeID = MQTT_FIRST_SENSORID;
        if (newNodeID >= MQTT_LAST_SENSORID)
          return; // Sorry no more id's left :(
        txBlink(1);
        if (!gw.sendRoute(
              build(msg, GATEWAY_ADDRESS, msg.sender, 255, C_INTERNAL,
                    I_ID_RESPONSE, 0).set(newNodeID)))
          errBlink(1);
      }
      else if (msg.type == I_BATTERY_LEVEL)
      {
        strcpy_P(buffer, mqtt_prefix);
        buffsize += strlen_P(mqtt_prefix);
        buffsize += sprintf(&buffer[buffsize], "/%i/255/BATTERY_LEVEL\0", msg.sender );
        msg.getString(convBuf);
#ifdef DEBUG
        Serial.print("publish: ");
        Serial.print((char*) buffer);
        Serial.print(" ");
        Serial.println((char*) convBuf);
#endif
        client.publish(buffer, convBuf);
      }
      else if (msg.type == I_SKETCH_NAME)
      {
        strcpy_P(buffer, mqtt_prefix);
        buffsize += strlen_P(mqtt_prefix);
        buffsize += sprintf(&buffer[buffsize], "/%i/255/SKETCH_NAME\0", msg.sender );
        msg.getString(convBuf);
#ifdef DEBUG
        Serial.print("publish: ");
        Serial.print((char*) buffer);
        Serial.print(" ");
        Serial.println((char*) convBuf);
#endif
        client.publish(buffer, convBuf);
      }
      else if (msg.type == I_SKETCH_VERSION)
      {
        strcpy_P(buffer, mqtt_prefix);
        buffsize += strlen_P(mqtt_prefix);
        buffsize += sprintf(&buffer[buffsize], "/%i/255/SKETCH_VERSION\0", msg.sender );
        msg.getString(convBuf);
#ifdef DEBUG
        Serial.print("publish: ");
        Serial.print((char*) buffer);
        Serial.print(" ");
        Serial.println((char*) convBuf);
#endif
        client.publish(buffer, convBuf);
      }

    }
    else if (mGetCommand(msg) != 0)
    {
      if (mGetCommand(msg) == 3)
        msg.type = msg.type + (S_FIRSTCUSTOM - 10);	//Special message

      if (msg.type > VAR_TOTAL)
        msg.type = VAR_TOTAL;		// If type > defined types set to unknown.
      strcpy_P(buffer, mqtt_prefix);
      buffsize += strlen_P(mqtt_prefix);
      buffsize += sprintf(&buffer[buffsize], "/%i/%i/V_%s\0", msg.sender,
                          msg.sensor, getType(convBuf, &VAR_Type[msg.type]));
      msg.getString(convBuf);
#ifdef DEBUG
      Serial.print("publish: ");
      Serial.print((char*) buffer);
      Serial.print(" ");
      Serial.println((char*) convBuf);
#endif
      client.publish(buffer, convBuf);
    }
  }
}

/*
 * build
 * Constructs a radio message
 */
inline MyMessage& build(MyMessage &msg, uint8_t sender, uint8_t destination, uint8_t sensor,
                        uint8_t command, uint8_t type, bool enableAck)
{
  msg.destination = destination;
  msg.sender = sender;
  msg.sensor = sensor;
  msg.type = type;
  mSetCommand(msg, command);
  mSetRequestAck(msg, enableAck);
  mSetAck(msg, false);
  return msg;
}

/*
 * getType
 */
char *getType(char *b, const char **index)
{
  char *q = b;
  char *p = (char *) pgm_read_word(index);
  while (*q++ = pgm_read_byte(p++))
    ;
  *q = 0;
  return b;
}

/*
 * begin
 * wraps MySensors begin method; setup of RTC and led timers interrupt
 */
void begin()
{
#ifdef DEBUG
  Serial.begin(BAUD_RATE);
#endif
#ifdef DSRTC
  // Attach RTC
  setSyncProvider(RTC.get);   // the function to get the time from the RTC
  setSyncInterval(60);
#endif

  gw.begin(processRadioMessage, 0, true, 0);

  MsTimer2::set(200, ledTimersInterrupt);
  MsTimer2::start();

#ifdef DEBUG
  Serial.print(getType(convBuf, &VAR_Type[S_FIRSTCUSTOM]));
#endif
}

/*
 * processMQTTMessages
 * message handler for the PubSubClient
 */
void processMQTTMessages(char* topic, byte* payload, unsigned int length)
{
  processMQTTMessage(topic, payload, length);
}

/*
 * processMQTTMessage
 * processes MQTT messages, parses the topic, extracts radio address out of topic and sends them
 * to the respective radio
 */
void processMQTTMessage(char* topic, byte* payload,
                        unsigned int length)
{
  char *str, *p;
  uint8_t i = 0;
  buffer[0] = 0;
  buffsize = 0;
  uint8_t cmd = -1;

  for (str = strtok_r(topic, "/", &p); str && i < 5;
       str = strtok_r(NULL, "/", &p))
  {
    switch (i)
    {
      case 0:
        {
          if (strcmp_P(str, mqtt_prefix) != 0)
          { //look for MQTT_PREFIX
            return;			//Message not for us or malformatted!
          }
          break;
        }
      case 1:
        {
          msg.destination = atoi(str);	//NodeID
          break;
        }
      case 2:
        {
          msg.sensor = atoi(str);		//SensorID
          break;
        }
      case 3:
        {
          char match = 0;			//SensorType
          //strcpy(str,(char*)&str[2]);  //Strip VAR_

          for (uint8_t j = 0;
               strcpy_P(convBuf, (char*) pgm_read_word(&(VAR_Type[j]))); j++)
          {
            if (strcmp((char*) &str[2], convBuf) == 0)
            { //Strip VAR_ and compare
              match = j;
              break;
            }
            if (j >= VAR_TOTAL)
            { // No match found!
              match = VAR_TOTAL;	// last item.
              break;
            }
          }
          msg.type = match;
          break;
        }
       case 4:
          {
            // support the command get and set; get will be mapped to a C_REQ and set to C_SET
            if (strcmp(str,MQTT_CMD_SET) == 0)
            {
              cmd = C_SET;
            }
            else if (strcmp(str,MQTT_CMD_GET) == 0) {
              cmd = C_REQ;
            } 
            else {
#ifdef DEBUG
              Serial.print("Received unsupported command - ignore: ");
              Serial.println(str);
#endif              
            }
          } 
    }
    i++;
  }						//Check if packge has payload

  if (cmd != -1) {
    char* ca;
    ca = (char *)payload;
    ca += length;
    *ca = '\0';

    msg.set((const char*)payload);			//Payload

    txBlink(1);
    // inject time
    if ((msg.destination == 0) && (msg.sensor == 199)) {
      unsigned long epoch = atol((char*)payload);
      if (epoch > 10000) {
#ifdef DSRTC
        RTC.set(epoch); // this sets the RTC to the time from controller - which we do want periodically
#endif
        setTime(epoch);
      }
#ifdef DEBUG
      Serial.print("Time recieved ");
      Serial.println(epoch);
#endif
    }
    //
    if (!gw.sendRoute(
          build(msg, GATEWAY_ADDRESS, msg.destination, msg.sensor, C_SET, msg.type,
                0)))
      errBlink(1);
  }
}

//////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Led handling
//////////////////////////////////////////////////////////////////////////////////////////////////////////////

/*
 * ledTimersIntterupt
 */
void ledTimersInterrupt() {
  if (countRx && countRx != 255) {
    // switch led on
    digitalWrite(RADIO_RX_LED_PIN, LOW);
  } else if (!countRx) {
    // switching off
    digitalWrite(RADIO_RX_LED_PIN, HIGH);
  }
  if (countRx != 255) {
    countRx--;
  }

  if (countTx && countTx != 255) {
    // switch led on
    digitalWrite(RADIO_TX_LED_PIN, LOW);
  } else if (!countTx) {
    // switching off
    digitalWrite(RADIO_TX_LED_PIN, HIGH);
  }
  if (countTx != 255) {
    countTx--;
  }

  if (countErr && countErr != 255) {
    // switch led on
    digitalWrite(RADIO_ERROR_LED_PIN, LOW);
  } else if (!countErr) {
    // switching off
    digitalWrite(RADIO_ERROR_LED_PIN, HIGH);
  }
  if (countErr != 255) {
    countErr--;
  }
}


void rxBlink(uint8_t cnt)
{
  if (countRx == 255)
  {
    countRx = cnt;
  }
}

void txBlink(uint8_t cnt)
{
  if (countTx == 255)
  {
    countTx = cnt;
  }
}

void errBlink(uint8_t cnt)
{
  if (countErr == 255)
  {
    countErr = cnt;
  }
}

[otto001]

@tomkxy Thanks, that helped! I seemed to have two problems:

1. I did not know where to place the necessary includes! Thanks for the examples! Helped me a lot

2)Regarding my problems from above (error: 'DEFAULT_CE_PIN' was not declared in this scope): I removed the whole libraries/MySensors dir and copied it again from the recent devel branch - this helped, error is gone and everything is compiling fine now

Thanks again and cheers,

Otto

[Anticimex]

@otto001 Great! And thanks @tomkxy for helping out. MySensors teamwork at it's best. This is what it's all about

[otto001]

Hi again,

and thanks again. signing seems to work now, unfortunately the sketch is too big for an UNO and I am right now struggling with the pinout and code adaptions to get it working with my W5100 shield and a mega...
/edit: got it working.

MyConfig.h:

#ifdef SOFTSPI
    // Define the soft SPI pins used for NRF radio
    //MEGA
    const uint8_t SOFT_SPI_MISO_PIN = 15;
    const uint8_t SOFT_SPI_MOSI_PIN = 14;
    const uint8_t SOFT_SPI_SCK_PIN = 16;
#endif

EthernetGateway.ino:

#define RADIO_CE_PIN        35 // 5  // radio chip enable
#define RADIO_SPI_SS_PIN    34 // 6  // radio SPI serial select

And exactly those pins I connected from radio to the mega... (just in case that anyone else is in need of this)

Might be a little OT, but maybe someone here knows a solution for a strange problem:
Whenever I am using an UNO or MEGA with my W5100 shield, I have to reset after every power-on to make the sketch run. Does anyone know this? Is there a solution? I am trying to adapt optiboot to wait additional 500ms when powering on, but no luck so far...

Cheers,

Otto

[TD22057]

@otto001 said:

Might be a little OT, but maybe someone here knows a solution for a strange problem:
Whenever I am using an UNO or MEGA with my W5100 shield, I have to reset after every power-on to make the sketch run. Does anyone know this? Is there a solution? I am trying to adapt optiboot to wait additional 500ms when powering on, but no luck so far...

Check here: https://www.sparkfun.com/products/11166 Their "version 2" shield has some notes about fixing the power up reset problem with regular internet shields. Not sure what it is but maybe their notes and schematic can clue you in to a help you implement a fix on your current board.

[TD22057]

@TD22057 said:

@otto001 said:

Might be a little OT, but maybe someone here knows a solution for a strange problem:
Whenever I am using an UNO or MEGA with my W5100 shield, I have to reset after every power-on to make the sketch run. Does anyone know this? Is there a solution? I am trying to adapt optiboot to wait additional 500ms when powering on, but no luck so far...

Check here: https://www.sparkfun.com/products/11166 Their "version 2" shield has some notes about fixing the power up reset problem with regular internet shields. Not sure what it is but maybe their notes and schematic can clue you in to a help you implement a fix on your current board.

And is the explanation of the fix that needs to be made:
http://www.hobbyist.co.nz/?q=ethernet-shield-w5100
http://forum.arduino.cc/index.php?topic=215798.msg1607035#msg1607035

FYI I just ordered a Mega myself for a gateway since I figured the $7 to get one is a better expenditure of my time/money than me trying to get the MqttClientGateway + signing + RF69 radio + debug print to all fit on an Uno so I'll be doing this mod as well.

[fleinze]

@Anticimex
I think I may have found a bug in the code:
I wanted to build a gateway that accepts both signed and unsigned messages. For this I used MySigningAtsha204Soft signer(false);.
Now the gateway did accept unsigned messages, but also all messages sent to the gateway by another node with signing where unsigned.
After changing this line in MySensor.cpp:

if (signer.requestSignatures() && DO_SIGN(msg.sender))

to

if (signer.requestSignatures() || DO_SIGN(msg.sender))

it worked as I wanted it. The gateway accepts unsigned messages from nodes but if a node expects signed messages, the messages to the gateway are also signed.
This line means that the gateway requests signing from the node if he always requests signing or if the node requests signing.
What do you think?

[Anticimex]

@fleinze I am not sure I understand the basic problem. What do you mean by " but also all messages sent to the gateway by another node with signing where unsigned.". The signing is based on requirement. A node (or gateway) either require messages sent to it to be signed or not. You have told your gateway that it is not supposed to require signed messages so it will accept unsigned messages sent to it. It is up to the node to configure if the node require signed messages in return. The gateway configuration has nothing to do with the preferences of a node. If you tell your gateway that it does not require signing, no messages to it will be signed. The opposite also holds true. If you configure it to require signed messages, all messages have to be signed. That is "working as designed".

[Anticimex]

@fleinze after reading the post a few times I think I understand better you want nodes who require signed messages from the gateway to send signed messages to the gateway as well even though the gateway is configured to not require it. I don't have a problem with that change as long as it is also portable to the nodes (the same code works equally well for both gateways and nodes usecases). You are welcome to put a pull request on the code so I can have a closer look. If approved I will update the head post to reflect this changed behaviour.

[fleinze]

The change only effects gateways. I will put a pull request, thx.

[Anticimex]

@fleinze do you have some special reason for why you want this? I don't see a real benefit and it would slightly increase the traffic. I suppose it would add some symmetry to node-gw communication.

[Anticimex]

@fleinze I am quite stressed out for the moment so I forget my own design
What you actually should do is to configure the gateway to require signed messages. It will then do so, but only from nodes that in turn require signed messages. So you should not need to change anything, just set the GW to require signatures.

[fleinze]

@Anticimex If I do so, this is the output of the Gateway:

0;0;3;0;9;gateway started, id=0, parent=0, distance=0
0;0;3;0;14;Gateway startup complete.
0;0;3;0;9;no sign
0;0;3;0;9;no sign
0;0;3;0;9;no sign
0;0;3;0;9;no sign
0;0;3;0;9;read: 2-2-0 s=0,c=0,t=6,pt=0,l=0,sg=0:
2;0;0;0;6;
0;0;3;0;9;read: 2-2-0 s=1,c=0,t=30,pt=0,l=0,sg=0:
2;1;0;0;30;
0;0;3;0;9;no sign
0;0;3;0;9;no sign
0;0;3;0;9;no sign

I tried to compile the node with deactivated signing feature and also with MySigningNone.
The messages get rejected because in MySensor.cpp line 570 (developement branch) it is not checked if the sender requires signing:

	if (signer.requestSignatures() && msg.destination == nc.nodeId && mGetLength(msg) && !mGetAck(msg) &&
		(mGetCommand(msg) != C_INTERNAL ||
		 (msg.type != I_GET_NONCE_RESPONSE && msg.type != I_GET_NONCE && msg.type != I_REQUEST_SIGNING &&
		  msg.type != I_ID_REQUEST && msg.type != I_ID_RESPONSE &&
		  msg.type != I_FIND_PARENT && msg.type != I_FIND_PARENT_RESPONSE)))

My thought is that if I have a mixed network (signing and non-signing nodes) for some sensors I do not need signing (i.e. temperature-sensors). But if I have a button sensor that can actually switch something on or off (via the controller), it would be a security benefit if the messages from the sensor to the gateway are signed.

[Anticimex]

@fleinze If you have upgraded the library version, the signing table might have shifted in EEPROM. You then need to run the clear EEPROM sketch to reset the stored state in order for the gw/nodes to re-learn the existing signing preferences of the network.

[Anticimex]

@fleinze said:

My thought is that if I have a mixed network (signing and non-signing nodes) for some sensors I do not need signing (i.e. temperature-sensors). But if I have a button sensor that can actually switch something on or off (via the controller), it would be a security benefit if the messages from the sensor to the gateway are signed.

This is the exact usecase for the gw default behavior to only require signing from nodes that require signing in return. But I also got "no sign" errors after I flashed development branch yesterday on node/gw and only after I wiped the GW EEPROM I got it back online. So please try that. If it still does not work, I have to look closer, and see why this has broken because it has been working like that when I submitted the signing behavior.

[fleinze]

@Anticimex I tried to make it work:

• took a fresh copy of the development branch
• activated signing in MyConfig.h
• flashed the ClearEepromConfig sketch to both the test-gateway and the test-node.
• built the gateway with MySigningAtsha204Soft signer; (requires signing by default)
• built the node with MySigningNone signer;

Still got the no sign message from the gateway...
Please have a look at this.

[Anticimex]

@fleinze mysigningnone is a special case. It won't make signatures, so could you instead try with either hard or soft atsha?

[Anticimex]

@fleinze also, you can mix hard and soft atsha signing in a network, but you can't mix with "none". If you mark a node that it can do signing or require signing and use the "none" backend, nodes that use atsha (hard or soft) will not accept the signatures.

[fleinze]

@Anticimex
Ok, I built the node with MySigningAtsha204Soft signer(false); so it does not require signing and the gateway with MySigningAtsha204Soft signer;
-> no sign error.
How do I correctly build a node that does not sign in a mixed network (signing and non-signing nodes mixed)?

[Anticimex]

@fleinze
In a network where you have "mixed" nodes, I would suggest something like this:
The gateway has signing enabled and is set to require signatures and uses either hard or soft ATSHA (since you earlier wrote that you wanted the GW to sign messages to nodes that signed messages in return).
For all nodes that need to be secure, enable signing and pick either hard or soft ATSHA (depending on the node hardware) and set them to require signatures.
For all nodes that do not need to be secure, just disable signing alltogether, or pick any signing backend and set it to NOT require signatures.
The result should be like this:
A "insecure" node sends and receives unsigned messages to/from GW.
A "secure" node sends and receives signed messages to/from GW.
The GW will send signed messages to all nodes that has reported to the GW that they require signatures.
The GW will send unsigned messages to all nodes that has reported to the GW that they do NOT require signatures.

[fleinze]

@Anticimex In your original post you said:

However, the difference is that the gateway will only require signed messages from nodes it knows in turn require signed messages.

I made pull request 208 to match this behavior in the code:
https://github.com/mysensors/Arduino/pull/208

• If node is not a gateway, everything is as always. (1 || x)
• if node is gateway and the sender requires signed messages, check signature (0 || 1)
• if node is gateway and the sender does not requires signed messages, do not check signature (0 || 0)
  please consider adding this.

[Anticimex]

@fleinze Ok, lets discuss this in the PR and conclude it in this thread later.

[Anticimex]

Ok, the problem is identified and @fleinze has provided a fix that resolves the issue which is now merged to the development branch.
Thanks for finding the issue and fixing it!
There is no change in the signing behavior as it is documented. Now gateway does as it is supposed to.

[mfalkvidd]

Great work @fleinze and @Anticimex !
Reading this thread (yes, all of it) has made me ready to start configuring my nodes to use signing. Better start now, while my sensor network is still small

[FotoFieber]

Compiling with the latest dev branch SecureActuator example

#define MY_SIGNING_REQUEST_SIGNATURES

I get following error:

In file included from /Users/mm/Documents/Arduino165/libraries/MySensors/MySensor.h:157:0,
                 from SecureActuator.ino:58:
/Users/mm/Documents/Arduino165/libraries/MySensors/core/MyTransport.cpp: In function 'void transportProcess()':
/Users/mm/Documents/Arduino165/libraries/MySensors/core/MyTransport.cpp:92:10: error: 'MY_IS_GATEWAY' was not declared in this scope
    if ((!MY_IS_GATEWAY || DO_SIGN(sender)) &&
          ^
/Users/mm/Documents/Arduino165/libraries/MySensors/core/MyTransport.cpp:93:20: error: 'nc' was not declared in this scope
     destination == nc.nodeId &&
                    ^
Fehler beim Kompilieren.```

Is this a problem of the code or of my setup?

[Anticimex]

@FotoFieber @hek something related to the recent refactoring? I have not had the opportunity to evaluate the effects on signing myself.

[hek]

@FotoFieber said:

MY_SIGNING_REQUEST_SIGNATURES

Let's see how Jenkins feels about this
https://github.com/mysensors/Arduino/pull/231

[Avamander]

@Anticimex Hello, I found this thread now. I'll ask a few more questions, sorry. This all seems really-really great.

1. If someone wishes to use TMRh20's RF24Mesh instead of MySensors how would one sign the messages sent with that library? Is is even possible? What should be done to use that functionality?

2. Also, as I understand it is possible to emulate the ATSHA204A, on what hardware is is possible (on Uno's ATMega too?)?

3. Is it also possible to use whitelisting with RF24Mesh somehow?

Again, sorry for any dumb questions and my poor English.

[Anticimex]

@Avamander

1. Signing as described in this post is specific to users of the MySensors library. I suppose you could use any rf backend you like but you will need to use the MySensors library to get the nonce exchange and such. Or manually port the signing specifics out from the MySensors library and integrate them into another library. It's all open source.
2. The software emulated atsha signing backend (for MySensors) is compatible with any Arduino product.
3. See 1.

Lastly, there are no dumb questions, only dumb answers

[Silver978]

Hello,

firstly, I must say that you're doing a great job and this section about security is very interesting.
Sorry, but I want to ask a question: as regards the whitelisting system, how can I add more nodes to the list of the trusted nodes of the gateway?
The part of code is this:

[...]
#ifdef MY_SECURE_NODE_WHITELISTING
whitelist_entry_t node_whitelist[] = {
  { //I want to add nodes HERE:
    .nodeId = 55, // Just some value, this need to be changed  to the NodeId of the trusted node
    .serial = {0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09} } // This need to change to the serial of the trusted node
};
MySigningAtsha204 signer(true, 1, node_whitelist);  // Select ATSHA204A software signing backend with one entry in the whitelist
[...]

but the gateway continue to fail the verification of the trusted nodes that I inserted.

To sum up, how can I add more nodeIDs and serials in addition to the number "55" presented in the example of the code?

Thank you very much in advance,
Silver978

[Anticimex]

@Silver978 Thanks!
Try something like this:

whitelist_entry_t node_whitelist[] = {
  {
    .nodeId = 55, // Just some value, this need to be changed  to the NodeId of the trusted node
    .serial = {0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09} // This need to change to the serial of the trusted node
  },
  {
    .nodeId = 56, // Just some value, this need to be changed  to the NodeId of the trusted node
    .serial = {0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09} // This need to change to the serial of the trusted node
  },
  {
    .nodeId = 57, // Just some value, this need to be changed  to the NodeId of the trusted node
    .serial = {0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09} // This need to change to the serial of the trusted node
  }
};
MySigningAtsha204 signer(true, 3, node_whitelist);  // Select ATSHA204A software signing backend with three entries in the whitelist

Note the changed argument in the constructor (3 and not 1)

[Silver978]

@Anticimex I tried also this syntax but I haven't changed the argument of the signer, so that number indicates how many nodes the gateway must trust?
Thank you!

[Anticimex]

@Silver978 The number gives the number of entries in the whitelist the backend is expected to iterate over. Having a number for this allows the use of a potenitally very large whitelist but where "uninteresting" entries to this particular node can be ignored by having them last in the list and make sure the number does not cover those entries when searching the list.

[Silver978]

@Anticimex Perfect, understood! Very good explanation and super-fast support, fantastic
Thank you very much again!

[Anticimex]

@Silver978

[siod]

I would like to add security to my sensor network as well so I am very glad I found your thread. Your explanation are very detailed but I´m a guy who needs more practical examples. For example I would like to know, where exactly I have to activate this. I am using an arduino as an MQTT to Ethernet gateway and Mosquitto running on a raspberry pi. So all I have to do is activate the soft version in Myconfig.h, if not already activated, set a serial, re-upload to my arduino gateway and that´s it? I guess the Nodes have to get flashed again to get the serials as well, right?

[Anticimex]

@siod correct. You have to activate signing in both ends. The serial numbers you only need of you want to use whitelisting which is a bit more complex to set up. At the very least, for software signing, you need to enable it, have an unconnected analog pin on your boards, and set a hmac key.

[Anticimex]

EDIT: Topic post updates with a documentation link for those using the development branch.

[ahmedadelhosni]

Great topic as usual. I have a question please.

You mentioned "Because it is pure-software however, it does not provide as good nonces (it uses the Arduino pseudo-random generator) and the HMAC key is stored in SW and is therefore readable if the memory is dumped. "

Isn't this case also dangerous if the node which contains ATSHA chip was stolen ? The attacker will use the chip to be on the network ?

Thanks.

[Anticimex]

@ahmedadelhosni thanks!
Correct. That is why I also implemented whitelisting and node revocation. It's all described in the topic (and for development branch in doxygen, linked in the topic).

[ahmedadelhosni]

@Anticimex Thanks.

So this gets me to my next question

What I understood is that if I use whitelisting, I need to know the node ID of my sensor door to define it in the GW, correct ?
I guess the below code is in the GW.

whitelist_entry_t node_whitelist[] = {
  { .nodeId = 55, // Just some value, this need to be changed  to the NodeId of the trusted node
    .serial = {0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09} } // This need to change to the serial of the trusted node
};

another question, If I want to remove this node, do I have to reflash the GW and remove it ?

[Anticimex]

Yes.

[ahmedadelhosni]

@Anticimex mmmm
Can't it be implemented in a way so that whitelist is to be defined by the controller ?
For example, an internal message to be used to set the data.

My concern is that auto assignation of nodes is flexible but at the same time you may need to choose which sensor node to be added to your whitelist. Thus if a Controller can show you all your nodes, like vera or domoticz, I guess this option can be added to the plugin to send an internal message to the GW to choose which sensor node to be added.

Am I saying something logic ?

[Anticimex]

Possibly. But that would mean that security features is dictaded by the controller plugins and I do not like that at all.

For months I have been trying to convince the Domoticz people that their ACK timeout is way to short and needs to be configurable for signing to work with Domoticz, but they do not even reply to me.

And I do not trust controllers at all and want to have full control over all configuration aspects of the signing solution. So I do not think it is good to move that logic off the nodes (whitelisting can also be used by a node, communicating with another node).

It is not only a GW that can have a whitelist. So although it might have been flexible to have it configurable, I think it compromises security (who knows, your controller could be hacked to inject a whitelist that permits a rogue node in your network).

Of course, security is at some level compromised anyway if the controller is hacked. But it is not the signing solution that is compromised in this case.

And ultimately, that feature would mean that the level of signing security and signing features for MySensors, becomes controller specific. And that I do not think is a good idea. If I want to change or improve the feature, the controller plugins all also have to be updated. It just becomes a too big turnaround for something as important as this in my opinion.

[ahmedadelhosni]

Yeah I got your point of view and convinced me. The only solution to implement this is that other communities works together to improve flexibility and security. This is not that easy of course as you have said.

Thanks a lot for your time.

[Anticimex]

Thanks for understanding

[ahmedadelhosni]

Also in all cases I see that if your controller is being hacked there will be no use to play with the whitelist The attacker has full control already to blow my house if he wants to

Anyway maybe I need to reflash using signing to understand things more.

Keep it up

[ahmedadelhosni]

Hi again.

I managed to set run the softsign and everything seems fine till now. I just want to understand the system much more if you please.

Now my Gateway sends this data when I turn on Light.
( I am using GatewayW5100 with Domoticz ) Development branch

0;0;3;0;9;Eth: 0;0;3;0;18;PING
0;0;3;0;9;Eth: 2;7;1;0;2;1
0;0;3;0;9;send: 0-0-2-2 s=7,c=3,t=16,pt=0,l=0,sg=0,st=ok:
0;0;3;0;9;read: 2-2-0 s=255,c=3,t=17,pt=6,l=25,sg=0:01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACAC
0;0;3;0;9;Signing backend: ATSHA204Soft
0;0;3;0;9;Message to process: 00020E01020731
0;0;3;0;9;Current nonce: 01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACACAAAAAAAAAAAAAA
0;0;3;0;9;HMAC: 135DBD85528E869ECC86C9C53679795D7FDA7B789DB7A0A74053C94FE8D668F0
0;0;3;0;9;Signature in message: 015DBD85528E869ECC86C9C53679795D7FDA7B789DB7A0A7
0;0;3;0;9;send: 0-0-2-2 s=7,c=1,t=2,pt=0,l=1,sg=1,st=ok:1

I want to know how was the following data calculated please:

• sg=0:01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACAC
• Message to process: 00020E01020731
• Current nonce: 01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACACAAAAAAAAAAAAAA
• HMAC: 135DBD85528E869ECC86C9C53679795D7FDA7B789DB7A0A74053C94FE8D668F0
• Signature in message: 015DBD85528E869ECC86C9C53679795D7FDA7B789DB7A0A7

So how was sg= calculated ?
What is message to process ?
The nonce is depending on analogue signal + what ? to get this value ?
I have already defined a random HMAC, but this one is different. It is a combination of what ?
Finally what is signature in message ?

On the other side I am trying to sniff the data by a serialgateway which is trying to hack the network. It only read the following data:

0;0;3;0;9;read: 2-2-0 s=255,c=3,t=17,pt=6,l=25,sg=0:01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACAC                      
2;255;3;0;17;01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACAC

It is the sg only which it succeeded to read. Also I do understood form the topic that the attacker can read values sent between nodes unencrypted, but here I can't figure out where is the unencrypted data which can be sniffed by the attacker but we don't bother ourselves with cas we protect our network by using signing.

Lots of questions but I am trying to understand the logic and architecture in details to understand what I am doing

Thanks a lot.