[security] Introducing signing support to MySensors


  • Contest Winner

    @tomkxy The protocol is described in the first post. Any node that is sending a signed message to another node (that has requested to get signed messages) will ask for a nonce. So you have configured your gateway to require signed messages as well as your node.
    Regarding the personalization, I do t understand your question. The output is the chip configuration. It does not list any keys. I am not sure I understand what you mean by second step. You can generate a random key and that key you store and after you store it you have an option to also lock it, but then you have no way of ever changing it. I have documented the personalization flow also in the sketch itself.


  • Contest Winner

    I should emphasise that if you personalize multiple atsha:s you have to have the same key stored in all of them. But the sketch offer to generate a random key (but you can skip that and use any key or password you like). But the key must be the same for all members of a secure network that want to talk to each other.



  • @Anticimex: The output shown is related to the sketch configuration you described in your description:

    Set the following sketch configuration of the personalizer:
    Enable LOCK_CONFIGURATION
    Disable LOCK_DATA
    Enable SKIP_KEY_STORAGE
    Disable SKIP_UART_CONFIGURATION
    Disable USER_KEY_DATA

    Execute the sketch on the “master” device to obtain a randomized key. Save this key to >a secure location and keep it confidential so that you can retrieve it if you need to
    personalize more devices later on.

    I was wondering that I did not see any explicit reference to a key. So may be I just have to retry.

    I should emphasise that if you personalize multiple atsha:s you have to have the same >key stored in all of them. But the sketch offer to generate a random key (but you can >skip that and use any key or password you like). But the key must be the same for all >members of a secure network that want to talk to each other.
    Thanks for that clarification. I was aware of that.


  • Contest Winner

    @tomkxy are you sure you have enabled all those options? According to the dump, your configuration is still not locked and therefore no randomized key can be generated. Are there no more output from the sketch than that? You mentioned failure to lock data zone but I cannot see that message. And you cannot lock datazone without locking configuration.
    After the sketch locks configuration it will print a randomized key in the log.



  • @Anticimex: Thanks for your patience and your support.
    I rerun it with the output and sketch configuration listed below. May be the reason that it is not working that my poor soldering skills bricked the device.

    Extract from sketch (1st run):

    
    #include <sha204_library.h>
    #include <sha204_lib_return_codes.h>
    
    // The pin the ATSHA204 is connected on
    #define ATSHA204_PIN 17 // A3
    
    // Uncomment this to enable locking the configuration zone.
    // *** BE AWARE THAT THIS PREVENTS ANY FUTURE CONFIGURATION CHANGE TO THE CHIP ***
    // It is still possible to change the key, and this also enable random key generation
    #define LOCK_CONFIGURATION
    
    // Uncomment this to enable locking the data zone.
    // *** BE AWARE THAT THIS PREVENTS THE KEY TO BE CHANGED ***
    // It is not required to lock data, key cannot be retrieved anyway, but by locking
    // data, it can be guaranteed that nobody even with physical access to the chip,
    // will be able to change the key.
    //#define LOCK_DATA
    
    // Uncomment this to skip key storage (typically once key has been written once)
    #define SKIP_KEY_STORAGE
    
    // Uncomment this to skip key data storage (once configuration is locked, key
    // will aways randomize)
    // Uncomment this to skip key generation and use 'user_key_data' as key instead.
    //#define USER_KEY_DATA
    
    // Uncomment this for boards that lack UART
    // IMPORTANT: No confirmation will be required for locking any zones with this
    // configuration!
    // Also, key generation is not permitted in this mode as there is no way of
    // presenting the generated key.
    //#define SKIP_UART_CONFIRMATION
    
    #if defined(SKIP_UART_CONFIRMATION) && !defined(USER_KEY_DATA)
    #error You have to define USER_KEY_DATA for boards that does not have UART
    #endif
    …
    
    

    Output:

    
    ATSHA204 personalization sketch for MySensors usage.
    ----------------------------------------------------
    Device revision: 00020009
    Device serial:   {0x01,0x23,0x48,0xC9,0x51,0x6A,0x1A,0x06,0xEE}
    012348C9516A1A06EE
    Skipping configuration write and lock (configuration already locked).
    Chip configuration:
               SN[0:1]           |         SN[2:3]           | 01   23 | 48   C9   
                              Revnum                         | 00   09   04   00   
                              SN[4:7]                        | 51   6A   1A   06   
        SN[8]    |  Reserved13   | I2CEnable | Reserved15    | EE | 13 | 00 | 00   
      I2CAddress |  TempOffset   |  OTPmode  | SelectorMode  | C8 | 00 | 55 | 00   
             SlotConfig00        |       SlotConfig01        | 8F   80 | 80   A1   
             SlotConfig02        |       SlotConfig03        | 82   E0 | A3   60   
             SlotConfig04        |       SlotConfig05        | 94   40 | A0   85   
             SlotConfig06        |       SlotConfig07        | 86   40 | 87   07   
             SlotConfig08        |       SlotConfig09        | 0F   00 | 89   F2   
             SlotConfig0A        |       SlotConfig0B        | 8A   7A | 0B   8B   
             SlotConfig0C        |       SlotConfig0D        | 0C   4C | DD   4D   
             SlotConfig0E        |       SlotConfig0F        | C2   42 | AF   8F   
      UseFlag00  | UpdateCount00 | UseFlag01 | UpdateCount01 | FF | 00 | FF | 00   
      UseFlag02  | UpdateCount02 | UseFlag03 | UpdateCount03 | FF | 00 | FF | 00   
      UseFlag04  | UpdateCount04 | UseFlag05 | UpdateCount05 | FF | 00 | FF | 00   
      UseFlag06  | UpdateCount06 | UseFlag07 | UpdateCount07 | FF | 00 | FF | 00   
                          LastKeyUse[0:3]                    | FF   FF   FF   FF   
                          LastKeyUse[4:7]                    | FF   FF   FF   FF   
                          LastKeyUse[8:B]                    | FF   FF   FF   FF   
                          LastKeyUse[C:F]                    | FF   FF   FF   FF   
      UserExtra  |    Selector   | LockValue |  LockConfig   | 00 | 00 | 55 | 00   
    Disable SKIP_KEY_STORAGE to store key.
    Data not locked. Define LOCK_DATA to lock for real.
    --------------------------------
    Personalization is now complete.
    Configuration is LOCKED
    Data is UNLOCKED
    
    

    Second run with the following sketch settings (key used removed):

    
    #include <sha204_library.h>
    #include <sha204_lib_return_codes.h>
    
    // The pin the ATSHA204 is connected on
    #define ATSHA204_PIN 17 // A3
    
    // Uncomment this to enable locking the configuration zone.
    // *** BE AWARE THAT THIS PREVENTS ANY FUTURE CONFIGURATION CHANGE TO THE CHIP ***
    // It is still possible to change the key, and this also enable random key generation
    #define LOCK_CONFIGURATION
    
    // Uncomment this to enable locking the data zone.
    // *** BE AWARE THAT THIS PREVENTS THE KEY TO BE CHANGED ***
    // It is not required to lock data, key cannot be retrieved anyway, but by locking
    // data, it can be guaranteed that nobody even with physical access to the chip,
    // will be able to change the key.
    #define LOCK_DATA
    
    // Uncomment this to skip key storage (typically once key has been written once)
    //#define SKIP_KEY_STORAGE
    
    // Uncomment this to skip key data storage (once configuration is locked, key
    // will aways randomize)
    // Uncomment this to skip key generation and use 'user_key_data' as key instead.
    #define USER_KEY_DATA
    
    // Uncomment this for boards that lack UART
    // IMPORTANT: No confirmation will be required for locking any zones with this
    // configuration!
    // Also, key generation is not permitted in this mode as there is no way of
    // presenting the generated key.
    #define SKIP_UART_CONFIRMATION
    
    #if defined(SKIP_UART_CONFIRMATION) && !defined(USER_KEY_DATA)
    #error You have to define USER_KEY_DATA for boards that does not have UART
    #endif
    
    #ifdef USER_KEY_DATA
    #define MY_HMAC_KEY 0x…
    
    const uint8_t user_key_data[32] = {MY_HMAC_KEY};
    #endif
    const int sha204Pin = ATSHA204_PIN;
    atsha204Class sha204(sha204Pin);
    
    Device revision: 00020009
    Device serial:   {0x01,0x23,0x48,0xC9,0x51,0x6A,0x1A,0x06,0xEE}
    012348C9516A1A06EE
    Skipping configuration write and lock (configuration already locked).
    Chip configuration:
               SN[0:1]           |         SN[2:3]           | 01   23 | 48   C9   
                              Revnum                         | 00   09   04   00   
                              SN[4:7]                        | 51   6A   1A   06   
        SN[8]    |  Reserved13   | I2CEnable | Reserved15    | EE | 13 | 00 | 00   
      I2CAddress |  TempOffset   |  OTPmode  | SelectorMode  | C8 | 00 | 55 | 00   
             SlotConfig00        |       SlotConfig01        | 8F   80 | 80   A1   
             SlotConfig02        |       SlotConfig03        | 82   E0 | A3   60   
             SlotConfig04        |       SlotConfig05        | 94   40 | A0   85   
             SlotConfig06        |       SlotConfig07        | 86   40 | 87   07   
             SlotConfig08        |       SlotConfig09        | 0F   00 | 89   F2   
             SlotConfig0A        |       SlotConfig0B        | 8A   7A | 0B   8B   
             SlotConfig0C        |       SlotConfig0D        | 0C   4C | DD   4D   
             SlotConfig0E        |       SlotConfig0F        | C2   42 | AF   8F   
      UseFlag00  | UpdateCount00 | UseFlag01 | UpdateCount01 | FF | 00 | FF | 00   
      UseFlag02  | UpdateCount02 | UseFlag03 | UpdateCount03 | FF | 00 | FF | 00   
      UseFlag04  | UpdateCount04 | UseFlag05 | UpdateCount05 | FF | 00 | FF | 00   
      UseFlag06  | UpdateCount06 | UseFlag07 | UpdateCount07 | FF | 00 | FF | 00   
                          LastKeyUse[0:3]                    | FF   FF   FF   FF   
                          LastKeyUse[4:7]                    | FF   FF   FF   FF   
                          LastKeyUse[8:B]                    | FF   FF   FF   FF   
                          LastKeyUse[C:F]                    | FF   FF   FF   FF   
      UserExtra  |    Selector   | LockValue |  LockConfig   | 00 | 00 | 55 | 00   
    Using this user supplied key:
    #define MY_HMAC_KEY 0x…..
    Writing key to slot 0...
    Data lock failed. Response: D3
    Halting!
    

  • Contest Winner

    @tomkxy I'm on phone so I cannot check the response code right now but you could look it up in the atsha datasheet or in the software. I have not tried to lock data myself and I do not recommend it because it makes it impossible to change the key later on if it is comprised. From what I can see, everything looks good except the locking of the datazone (your key)


  • Contest Winner

    @tomkxy I should add that from the logs I see that you do have successfully stored your key, so unless you really want to lock down the key, personalization is done and successful.



  • @Anticimex: Thanks a lot. I didn't intend to lock the data zone.

    Btw, I did some tests regarding the nonce failure the gateway showed. I think the reason is rather simple. The gateway tried - for reasons I still don't understand - send data to the sensor for which it tried to get a nonce. The sensor however was powered down which is probably the reason why no nonce was sent. At least this error didn't show up when I removed the power down and replaced it by a simple call to delay.


  • Contest Winner

    @tomkxy Ah, I see. Well then that matter should be sorted.
    Regarding your radio issue, yes, if the GW wants to send data to your node, it needs to be up&running. From the log you sent, I cannot determine what data the GW tried to send though, since it (because your node has told the GW it require it) wants to sign the message to send, and fails, the actual message is never showed in the log.
    Perhaps you could send a log with your node continuously powered and we could see at least the type of message your GW tries to send. That could help to identify why it tries to send the message in the first place.



  • @Anticimex: I started a new thread http://forum.mysensors.org/topic/1782/gateway-is-sending-command-1-type-1-c-1-t-1-message-to-sensor
    because it looks like it is not really related to signing.
    In the post I included the log output of the message for which the nonce is being requested. It looks like the gateway node is sending back the humidity with a c=1 command. Somehow this does not make sense to me.


  • Contest Winner

    @tomkxy I see. Perhaps you request ACK or something like that. But ACKs are not signed. I have seen to that, so probably not. I am not too well into the non-signing aspects of the network though, so hopefully someone else perhaps can give a hint on what's wrong. Could be the sketch itself. I am glad we sorted the signing issues anyway 🙂



  • @Anticimex: I found the issue with respect to the gateway sending a message back to the sensor. This is due to the gateway subscribing to all topics on the MQTT broker. Once it receives a message it sends it to the sensor as SET message. I have to resolve that one.

    I think, I still have an issue with signing which I need to get sorted myself first. I will let you know.


  • Contest Winner

    @tomkxy Alright. Good to know that you found the root cause for the message issue.
    Regarding signing, from what I could see at least some signed messages did come through, so you should at least have the proper shared states (keys) and configs.



  • I did a few test with my ATSHA204. I could get it working on an Uno. The I took the chip and placed it on my breadboard with my ProMini. I tried to run the SHA Personaliser Sketch again just to make sure that the chip is somehow working and I received an error message that the device cannot be woken up.

    Since it worked on the Uno (including running the SHA Personalizer) it must be related to the ProMini (3.3V / 8 MHz).

    To what pin do I have to connect the data line?


  • Contest Winner

    @tomkxy That sounds strange. I have run on ProMini 3.3V/8MHz without issues.
    Default is to use A3 for the ATSHA204A. It does not really matter which pin you connect it as long as it is usable for digital I/O (and update MY_ATSHA204_PIN accordingly or provide your pin to the MySigningAtsha204 constructor).



  • How do I change the pin definition if I use A3 on the ProMini?


  • Contest Winner

    A3 is default setting. You can find the definition of MY_ATSHA204_PIN in MyConfig.h.



  • Yep, it is

    `#define MY_ATSHA204_PIN 17 // A3 - pin where ATSHA204 is attached
    ``
    I am just irritate by the 17. So this does map to A3 even on ProMini?

  • Contest Winner

    @tomkxy yes it does.



  • @Anticimex Unfortunately, I am now in a total state of mess. It seems that nothing works anymore the moment I turn on any signing. For sure, I have a problem with ATSHA204 and the ProMini which I need to sort out separately. I tried them all with my Uno (reading out the config) which worked. When I try to just read the config with my ProMini I get the error cannot wake up device.

    But now, even soft signing did not work any longer (it worked first, than I made changes and afte. I just receive nonce transmission errors from the sensor to the gateway although both radios are side by side and transmission without signing works perfectly.

    Btw, what is stored in the EPROM and under what circumstances do I have to clear EPROM first. I changed between the various signing feature (soft signing, HW signing, signing required etc.) back and forth and it seems that this info is stored in EPROM and somehow not cleared?? How does the sensor know that the gateway requires signing? Will it get the info out of EPROM? Will this information be updated during presentation?

    I think you did a great job on that implementation, the more frustrating it is that I cannot get it to work reliably. I am out for today. May be I find some time tomorrow getting anot ProMini prepared to checkout the issue with the chips.


  • Contest Winner

    @tomkxy The information stored in EEPROM is not specific to any backend. It just informs the node what other nodes require signed messages. At startup, a node broadcasts it's preference to the gateway which then updates it's EEPROM table and replies with its own preference back so the node knows if the gateway wants signed messages as well. If the preferences differ from what is stored in EEPROM already, it is the updated preferences that will replace the stored preferences.
    The usecases for this is if you were to deploy a new sensor which require signing, it would inform gateway of this at startup. But if you restart your gateway it would loose this unless it was stored in EEPROM (the same goes the other way around) so the EEPROM is used so that the signing rules in the network gets preserved even if nodes dissappear or restart.
    If you suspect the EEPROM to contain corrupt data, you can clear it with the cleareeorom sketch/example.
    The typical circumstance you need to clear EEPROM under is when you switch library version, and the reserved region of EEPROM in the library change (and you also use EEPROM in your sketch). Then the library might take some of your sketch data for "it's own" and that can/will lead to unpredictable results.


  • Hardware Contributor

    Just to say, that I tried some weeks ago ATSHA204A with pro mini and had no problem too. Maybe you should try clear eeprom and retry. Or try with another pro mini. But it should work I think.
    And for those who want to test excellent work of @Anticimex and @Tekka, I have posted a simple/basic not expensive, 1.8$ for 3 breakouts here : https://oshpark.com/shared_projects/lvvxsHSW . It can be useful for existing nodes too.

    See you soon


  • Hardware Contributor

    @scalz said:

    Just to say, that I tried some weeks ago ATSHA204A with pro mini and had no problem too. Maybe you should try clear eeprom and retry. Or try with another pro mini. But it should work I think.
    And for those who want to test excellent work of @Anticimex and @Tekka, I have posted a simple/basic not expensive, 1.8$ for 3 breakouts here : https://oshpark.com/shared_projects/lvvxsHSW . It can be useful for existing nodes too.

    Good idea - I'm finding soldering the ATSHA chips to be a real pain and a small breakout board would help with that. Is the eeprom pads on your board for people who are just using a mega328 chip? Doesn't a pro-mini already have enough eeprom on board?


  • Hardware Contributor

    @TDD22057: eeprom on the breakout is for the new ota in Mysensors (over the air upload sketch) . It is not related to authentication.
    I made this cheap breakout firstly because I wanted to test the new Mysensors features easily. and I thought it could be useful in some specific case where you don't need to make a pcb for one specific usercase so you use a veroboard and so you can easily add these new features with less pain...
    Happy it can help some people 😄



  • Regarding my issue with the ATSHA204A, I did some testing with the following results. I soldered the ATSHA204A on a small breakout board and loaded the SHAPersonalizer sketch.

    I tested three configurations:

    1. Breakout wired to the ProMini without breadboard use
    2. Breakout wired to a breadboard where the ProMini was plugged
    3. Breakout and ProMini both plugged to breadboard and connected by wires.

    Config 3 does not work!!!
    That means the breadboard significantly changes the electrical characteristics. Is anybody on the forum who is able to explain that?


  • Hardware Contributor

    @tomkxy: it is strange. My first tests have been done in you config 3 (atsha breakout and promini both plugged to breadboard). And it worked well for me. Could it be your wire quality??? I had problems with some Dupont wire once...



  • Well , could be... I had to use different wires female - female, male - female, male - male (for config 3). For the male-male wire I have no other option at the moment.

    However, I am glad it works now.


  • Contest Winner

    @tomkxy said:

    Well , could be... I had to use different wires female - female, male - female, male - male (for config 3). For the male-male wire I have no other option at the moment.

    However, I am glad it works now.

    Good news! Regarding breadboard issues, it could be that you need an additional decoupling on the VCC line to the atsha204a. And also perhaps a pullup on the SDA signal. Breadboards usually put some more requirements on noise suppression.



  • Hi,

    I seem to be too stupid (or drunk 😉 ) to get this working with MySigningAtsha204Soft 😞
    Is anyone willing to provide a working MyConfig.h and EthernetGateway.ino for this?
    There seemed to be some changes and I am sure, I am just too dumb... 😞



  • Here you go....However, I do not have EthernetGateway but a MQTTClientGateway. May be it is of help.
    What is your specific problem?

    myconfig.h (key changed)

    /**
     * The MySensors Arduino library handles the wireless radio link and protocol
     * between your home built sensors/actuators and HA controller of choice.
     * The sensors forms a self healing radio network with optional repeaters. Each
     * repeater and gateway builds a routing tables in EEPROM which keeps track of the
     * network topology allowing messages to be routed to nodes.
     *
     * Created by Henrik Ekblad <henrik.ekblad@mysensors.org>
     * Copyright (C) 2013-2015 Sensnology AB
     * Full contributor list: https://github.com/mysensors/Arduino/graphs/contributors
     *
     * Documentation: http://www.mysensors.org
     * Support Forum: http://forum.mysensors.org
     *
     * This program is free software; you can redistribute it and/or
     * modify it under the terms of the GNU General Public License
     * version 2 as published by the Free Software Foundation.
     */
    
    
    #ifndef MyConfig_h
    #define MyConfig_h
    #include <stdint.h>
    
    // Enable debug flag for debug prints. This will add a lot to the size of the final sketch but good
    // to see what is actually is happening when developing
    #define DEBUG
    
    // Serial output baud rate (for debug prints and serial gateway)
    #define BAUD_RATE 115200
    
    
    /**********************************
    *  Over the air firmware updates
    ***********************************/
    
    // The following define enables the safe over-the-air firmware update feature
    // which requires external flash and the DualOptiBoot bootloader.
    // Note: You can still have OTA FW updates without external flash but it
    // requires the MYSBootloader and disabled MY_OTA_FIRMWARE_FEATURE
    //#define MY_OTA_FIRMWARE_FEATURE
    // Slave select pin for external flash
    #define MY_OTA_FLASH_SS 8
    // Flash jdecid
    #define MY_OTA_FLASH_JDECID 0x1F65
    
    
    /**********************************
    *  Information LEDs blinking
    ***********************************/
    // This feature enables LEDs blinking on message receive, transmit
    // or if some error occured. This was commonly used only in gateways,
    // but now can be used in any sensor node. Also the LEDs can now be
    // disabled in the gateway.
    
    //#define WITH_LEDS_BLINKING
    
    // The following setting allows you to inverse the blinking feature WITH_LEDS_BLINKING
    // When WITH_LEDS_BLINKING_INVERSE is enabled LEDSs are normally turned on and switches
    // off when blinking
    
    //#define WITH_LEDS_BLINKING_INVERSE
    
    
    // default LEDs blinking period in milliseconds
    #define DEFAULT_LED_BLINK_PERIOD 300
    // The RX LED default pin
    #define DEFAULT_RX_LED_PIN 6
    // The TX LED default pin
    #define DEFAULT_TX_LED_PIN 5
    // The Error LED default pin
    #define DEFAULT_ERR_LED_PIN 4
    
    
    /**********************************
    *  Message Signing Settings
    ***********************************/
    // Disable to completly disable signing functionality in library
    #define MY_SIGNING_FEATURE
    
    // Define a suitable timeout for a signature verification session
    // Consider the turnaround from a nonce being generated to a signed message being received
    // which might vary, especially in networks with many hops. 5s ought to be enough for anyone.
    #define MY_VERIFICATION_TIMEOUT_MS 5000
    
    // Enable to turn on whitelisting
    // When enabled, a signing node will salt the signature with it's unique signature and nodeId.
    // The verifying node will look up the sender in a local table of trusted nodes and
    // do the corresponding salting in order to verify the signature.
    // For this reason, if whitelisting is enabled on one of the nodes in a sign-verify pair, both
    // nodes have to implement whitelisting for this to work.
    // Note that a node can still transmit a non-salted message (i.e. have whitelisting disabled)
    // to a node that has whitelisting enabled (assuming the receiver does not have a matching entry
    // for the sender in it's whitelist)
    //#define MY_SECURE_NODE_WHITELISTING
    
    // MySigningAtsha204 default setting
    #define MY_ATSHA204_PIN 17 // A3 - pin where ATSHA204 is attached
    
    // MySigningAtsha204Soft default settings
    #define MY_RANDOMSEED_PIN 7 // A7 - Pin used for random generation (do not connect anything to this)
    
    // Key to use for HMAC calculation in MySigningAtsha204Soft (32 bytes)
    #define MY_HMAC_KEY 0x10,0x23,0xBB,0x78,0x77,0x35,0xB0,0x01,0x70,0x47,0xF1,0xDE,0x21,0x94,0x54,0x67,0xEE,0x36,0x72,0x00,0x97,0x12,0xA0,0x0A,0x0F,0x09,0x03,0xE2,0x00,0x31,0xE4,0x41
    
    
    /**********************************
    *  NRF24L01 Driver Defaults
    ***********************************/
    #define RF24_CE_PIN		   9
    #define RF24_CS_PIN		   10
    #define RF24_PA_LEVEL 	   RF24_PA_MAX
    #define RF24_PA_LEVEL_GW   RF24_PA_LOW
    // RF channel for the sensor net, 0-127
    #define RF24_CHANNEL	   76
    //RF24_250KBPS for 250kbs, RF24_1MBPS for 1Mbps, or RF24_2MBPS for 2Mbps
    #define RF24_DATARATE 	   RF24_250KBPS
    // This is also act as base value for sensor nodeId addresses. Change this (or channel) if you have more than one sensor network.
    #define RF24_BASE_RADIO_ID ((uint64_t)0xA8A8E1FC00LL)
    
    // Enable SOFTSPI for NRF24L01 when using the W5100 Ethernet module
    //#define SOFTSPI
    #ifdef SOFTSPI
    	// Define the soft SPI pins used for NRF radio
    	const uint8_t SOFT_SPI_MISO_PIN = 16;
        const uint8_t SOFT_SPI_MOSI_PIN = 15;
        const uint8_t SOFT_SPI_SCK_PIN = 14;
    #endif
    
    
    /**********************************
    *  RFM69 Driver Defaults
    ***********************************/
    // Default network id. Use the same for all nodes that will talk to each other
    #define RFM69_NETWORKID     100
    
    // Default frequency to use. This must match the hardware version of the RFM69 radio (uncomment one):
    // #define RFM69_FREQUENCY   RF69_433MHZ
    #define RFM69_FREQUENCY   RF69_868MHZ
    //#define FREQUENCY     RF69_915MHZ
    
    // Enable this for encryption of packets
    //#define RFM69_ENABLE_ENCRYPTION
    #define RFM69_ENCRYPTKEY    "sampleEncryptKey" //exactly the same 16 characters/bytes on all nodes!
    
    
    #endif
    
    
    /* 				MyMQTT Client Gateway 0.1b
     Created by Norbert Truchsess <norbert.truchsess@t-online.de>
     Based on MyMQTT-broker gateway created by Daniel Wiegert <daniel.wiegert@gmail.com>
     Based on MySensors Ethernet Gateway by Henrik Ekblad <henrik.ekblad@gmail.com>
     http://www.mysensors.org
     Requires MySensors lib 1.4b
     * Change below; TCP_IP, TCP_PORT, TCP_MAC
     This will listen on your selected TCP_IP:TCP_PORT below, Please change TCP_MAC your liking also.
     *1 -> NOTE: Keep first byte at x2, x6, xA or xE (replace x with any hex value) for using Local Ranges.
     *2 You can use standard pin set-up as MySensors recommends or if you own a IBOARD you may change
     the radio-pins below if you hardware mod your iBoard. see [URL BELOW] for more details.
     http://forum.mysensors.org/topic/224/iboard-cheap-single-board-ethernet-arduino-with-radio/5
     * Don't forget to look at the definitions in libraries\MySensors\MyMQTT.h!
     define TCPDUMP and connect serial interface if you have problems, please write on
     http://forum.mysensors.org/ and explain your problem, include serial output. Don't forget to
     turn on DEBUG in libraries\MySensors\MyConfig.h also.
     MQTT_FIRST_SENSORID is for 'DHCP' server in MyMQTT. You may limit the ID's with FIRST and LAST definition.
     If you want your manually configured below 20 set MQTT_FIRST_SENSORID to 20.
     To disable: set MQTT_FIRST_SENSORID to 255.
     MQTT_BROKER_PREFIX is the leading prefix for your nodes. This can be only one char if like.
     MQTT_SEND_SUBSCRIPTION is if you want the MyMQTT to send a empty payload message to your nodes.
     This can be useful if you want to send latest state back to the MQTT client. Just check if incoming
     message has any length or not.
     Example: if (msg.type==V_LIGHT && strlen(msg.getString())>0) otherwise the code might do strange things.
     * Address-layout is : [MQTT_BROKER_PREFIX]/[NodeID]/[SensorID]/V_[SensorType]
     NodeID and SensorID is uint8 (0-255) number.
     Last segment is translation of the sensor type, look inside MyMQTT.cpp for the definitions.
     User can change this to their needs. We have also left some space for custom types.
     Special: (sensor 255 reserved for special commands)
     You can receive a node sketch name with MyMQTT/20/255/V_Sketch_name (or version with _version)
     To-do:
     Special commands : clear or set EEPROM Values, Send REBOOT and Receive reboot for MyMQTT itself.
     Be able to send ACK so client returns the data being sent.
     ... Please come with ideas!
     What to do with publish messages.
     Test in more MQTT clients, So far tested in openhab and MyMQTT for Android (Not my creation)
     - http://www.openhab.org/
     - https://play.google.com/store/apps/details?id=at.tripwire.mqtt.client&hl=en
     ... Please notify me if you use this broker with other software.
     * How to set-up Openhab and MQTTGateway:
     http://forum.mysensors.org/topic/303/mqtt-broker-gateway
    
     Changes by Thomas Krebs <thkrebs@gmx.de>
     - Add signing support from MySensors 1.5 and update for MySensors 1.5;
     - Restructured code back to a C like implementation following the existing MQTTGateway
     */
    
    #include <SPI.h>
    #include <MySensor.h>
    #include "MyMQTTClient.h"
    #include "PubSubClient.h"
    #include <Ethernet.h>
    #include <DigitalIO.h>
    #include <MsTimer2.h>
    #include <Time.h>
    
    
    #ifdef MY_SIGNING_FEATURE
    #include <MySigningNone.h>
    #include <MySigningAtsha204Soft.h>
    #include <MySigningAtsha204.h>
    #endif
    
    //#define DSRTC
    #ifdef DSRTC
    #include <Wire.h>
    #include <DS1307RTC.h>  // a basic DS1307 library that returns time as a time_t
    #endif
    
    
    /*
     * To configure MQTTClientGateway.ino to use an ENC28J60 based board include
     * 'UIPEthernet.h' (SPI.h required for MySensors anyway). The UIPEthernet-library can be downloaded
     * from: https://github.com/ntruchsess/arduino_uip
     */
    
    //#include <UIPEthernet.h>
    /*
     * To execute MQTTClientGateway.ino on Yun uncomment Bridge.h and YunClient.h.
     * Do not include Ethernet.h or SPI.h in this case.
     * On Yun there's no need to configure local_ip and mac in the sketch
     * as this is configured on the linux-side of Yun.
     */
    
    //#include <Bridge.h>
    //#include <YunClient.h>
    // * Use this for IBOARD modded to use standard MISO/MOSI/SCK, see note *1 above!
    /*
     #define RADIO_CE_PIN        3			// radio chip enable
     #define RADIO_SPI_SS_PIN    8			// radio SPI serial select
     #define RADIO_ERROR_LED_PIN A2  		// Error led pin
     #define RADIO_RX_LED_PIN    A1  		// Receive led pin
     #define RADIO_TX_LED_PIN    A0  		// the PCB, on board LED*/
    
    // * Use this for default configured pro mini / nano etc :
    //
    
    //#define RADIO_CE_PIN        5		// radio chip enable
    //#define RADIO_SPI_SS_PIN    6		// radio SPI serial select
    //#define RADIO_ERROR_LED_PIN 7		// Error led pin
    //#define RADIO_RX_LED_PIN    8		// Receive led pin
    //#define RADIO_TX_LED_PIN    9		// the PCB, on board LED*/
    
    // CE_PIN and SPI_SS_PIN for Mega
    #define RADIO_CE_PIN        48			// radio chip enable
    #define RADIO_SPI_SS_PIN    49			// radio SPI serial select
    #define RADIO_ERROR_LED_PIN A2  		// Error led pin
    #define RADIO_RX_LED_PIN    A1  		// Receive led pin
    #define RADIO_TX_LED_PIN    A0  		// the PCB, on board LED*/
    
    //replace with ip of server you want to connect to, comment out if using 'remote_host'
    uint8_t remote_ip[] =  { 192, 168, 178, 74 };  // Mosquitto broker
    
    //replace with hostname of server you want to connect to, comment out if using 'remote_ip'
    //char* remote_ip = "server.local";
    //replace with the port that your server is listening on
    #define remote_port 1883
    //replace with arduinos ip-address. Comment out if Ethernet-startup should use dhcp. Is ignored on Yun
    uint8_t local_ip[] = {192, 168, 178, 11};
    //replace with ethernet shield mac. It's mandatory every device is assigned a unique mac. Is ignored on Yun
    uint8_t mac[] = { 0xA2, 0xAE, 0xAD, 0xA0, 0xA0, 0xA2 };
    
    //////////////////////////////////////////////////////////////////
    
    #if defined remote_ip && defined remote_host
    #error "cannot define both remote_ip and remote_host at the same time!"
    #endif
    
    #ifdef _YUN_CLIENT_H_
    YunClient ethClient;
    #else
    EthernetClient ethClient;
    #endif
    
    ////////////////////////////////////////////////////////////////
    // NRFRF24L01 radio driver (set low transmit power by default)
    MyTransportNRF24 transport(RADIO_CE_PIN, RADIO_SPI_SS_PIN, RF24_PA_LEVEL_GW);
    //MyTransportRFM69 transport;
    
    // Message signing driver (signer needed if MY_SIGNING_FEATURE is turned on in MyConfig.h)
    //MySigningNone signer;
    MySigningAtsha204Soft signer;
    //MySigningAtsha204 signer;
    
    // Hardware profile
    MyHwATMega328 hw;
    
    MyMessage msg;
    char convBuf[MAX_PAYLOAD * 2 + 1];
    uint8_t buffsize;
    char buffer[MQTT_MAX_PACKET_SIZE];
    
    
    ////////////////////////////////////////////////////////////////
    
    volatile uint8_t countRx;
    volatile uint8_t countTx;
    volatile uint8_t countErr;
    
    
    ////////////////////////////////////////////////////////////////
    
    void processMQTTMessages(char* topic, byte* payload, unsigned int length);
    PubSubClient client(remote_ip, remote_port, processMQTTMessages, ethClient);
    
    ////////////////////////////////////////////////////////////////
    
    
    // Declare and initialize MySensor instance
    // Construct MyMQTTClient (signer needed if MY_SIGNING_FEATURE is turned on in MyConfig.h, if signing
    // feature not to be used, uncomment)
    // To use LEDs blinking, uncomment WITH_LEDS_BLINKING in MyConfig.h
    MySensor gw(transport, hw
    #ifdef MY_SIGNING_FEATURE
                , signer
    #endif
    #ifdef WITH_LEDS_BLINKING
                , RADIO_RX_LED_PIN, RADIO_TX_LED_PIN, RADIO_ERROR_LED_PIN
    #endif
               );
    
    /*
     * setup
     */
    void setup()
    {
      countRx = 0;
      countTx = 0;
      countErr = 0;
    
      Ethernet.begin(mac, local_ip);
      //Bridge.begin();
      delay(1000);   // Wait for Ethernet to get configured.
    
      begin();
    }
    
    /*
     * loop
     */
    void loop()
    {
      if (!client.connected())
      {
        client.connect("MySensor");
        client.subscribe(MQTT_TOPIC_MASK);
      }
      client.loop();
      gw.process();
    }
    
    /*
     * processRadioMessage
     *
     * Receives radio message, parses it and forwards it to the MQTT broker
     */
    void processRadioMessage(const MyMessage &message)
    { 
        rxBlink(1);
        sendMQTT(message);
    }
    
    /*
     * sendMQTT
     * Handles processing of radio messages and eventually publishes it to the MQTT broker
     */
    void sendMQTT(const MyMessage &inMsg)
    {
      MyMessage msg = inMsg;
      buffsize = 0;
      if (!client.connected())
        return;			//We have no connections - return
      if (msg.isAck())
      {
    #ifdef DEBUG
        Serial.println("msg is ack!");
    #endif
        if (msg.sender == 255 && mGetCommand(msg) == C_INTERNAL
            && msg.type == I_ID_REQUEST)
        {
          // TODO: sending ACK request on id_response fucks node up. doesn't work.
          // The idea was to confirm id and save to EEPROM_LATEST_NODE_ADDRESS.
        }
      }
      else
      {
        // we have to check every message if its a newly assigned id or not.
        // Ack on I_ID_RESPONSE does not work, and checking on C_PRESENTATION isn't reliable.
        uint8_t newNodeID = gw.loadState(EEPROM_LATEST_NODE_ADDRESS) + 1;
        if (newNodeID <= MQTT_FIRST_SENSORID)
          newNodeID = MQTT_FIRST_SENSORID;
        if (msg.sender == newNodeID)
        {
          gw.saveState(EEPROM_LATEST_NODE_ADDRESS, newNodeID);
        }
        if (mGetCommand(msg) == C_INTERNAL)
        {
          if (msg.type == I_CONFIG)
          {
            txBlink(1);
            if (!gw.sendRoute(
                  build(msg, GATEWAY_ADDRESS, msg.sender, 255, C_INTERNAL,
                        I_CONFIG, 0).set("M")))
              errBlink(1);
          }
          else if (msg.type == I_TIME)
          {
    #ifdef DEBUG
            Serial.println("I_TIME requested!");
    #endif
            txBlink(1);
            if (!gw.sendRoute(
                  build(msg, GATEWAY_ADDRESS, msg.sender, 255, C_INTERNAL,
                        I_TIME, 0).set(now())))
              errBlink(1);
          }
          else if (msg.type == I_ID_REQUEST && msg.sender == 255)
          {
            uint8_t newNodeID = gw.loadState(EEPROM_LATEST_NODE_ADDRESS) + 1;
            if (newNodeID <= MQTT_FIRST_SENSORID)
              newNodeID = MQTT_FIRST_SENSORID;
            if (newNodeID >= MQTT_LAST_SENSORID)
              return; // Sorry no more id's left :(
            txBlink(1);
            if (!gw.sendRoute(
                  build(msg, GATEWAY_ADDRESS, msg.sender, 255, C_INTERNAL,
                        I_ID_RESPONSE, 0).set(newNodeID)))
              errBlink(1);
          }
          else if (msg.type == I_BATTERY_LEVEL)
          {
            strcpy_P(buffer, mqtt_prefix);
            buffsize += strlen_P(mqtt_prefix);
            buffsize += sprintf(&buffer[buffsize], "/%i/255/BATTERY_LEVEL\0", msg.sender );
            msg.getString(convBuf);
    #ifdef DEBUG
            Serial.print("publish: ");
            Serial.print((char*) buffer);
            Serial.print(" ");
            Serial.println((char*) convBuf);
    #endif
            client.publish(buffer, convBuf);
          }
          else if (msg.type == I_SKETCH_NAME)
          {
            strcpy_P(buffer, mqtt_prefix);
            buffsize += strlen_P(mqtt_prefix);
            buffsize += sprintf(&buffer[buffsize], "/%i/255/SKETCH_NAME\0", msg.sender );
            msg.getString(convBuf);
    #ifdef DEBUG
            Serial.print("publish: ");
            Serial.print((char*) buffer);
            Serial.print(" ");
            Serial.println((char*) convBuf);
    #endif
            client.publish(buffer, convBuf);
          }
          else if (msg.type == I_SKETCH_VERSION)
          {
            strcpy_P(buffer, mqtt_prefix);
            buffsize += strlen_P(mqtt_prefix);
            buffsize += sprintf(&buffer[buffsize], "/%i/255/SKETCH_VERSION\0", msg.sender );
            msg.getString(convBuf);
    #ifdef DEBUG
            Serial.print("publish: ");
            Serial.print((char*) buffer);
            Serial.print(" ");
            Serial.println((char*) convBuf);
    #endif
            client.publish(buffer, convBuf);
          }
    
        }
        else if (mGetCommand(msg) != 0)
        {
          if (mGetCommand(msg) == 3)
            msg.type = msg.type + (S_FIRSTCUSTOM - 10);	//Special message
    
          if (msg.type > VAR_TOTAL)
            msg.type = VAR_TOTAL;		// If type > defined types set to unknown.
          strcpy_P(buffer, mqtt_prefix);
          buffsize += strlen_P(mqtt_prefix);
          buffsize += sprintf(&buffer[buffsize], "/%i/%i/V_%s\0", msg.sender,
                              msg.sensor, getType(convBuf, &VAR_Type[msg.type]));
          msg.getString(convBuf);
    #ifdef DEBUG
          Serial.print("publish: ");
          Serial.print((char*) buffer);
          Serial.print(" ");
          Serial.println((char*) convBuf);
    #endif
          client.publish(buffer, convBuf);
        }
      }
    }
    
    /*
     * build
     * Constructs a radio message
     */
    inline MyMessage& build(MyMessage &msg, uint8_t sender, uint8_t destination, uint8_t sensor,
                            uint8_t command, uint8_t type, bool enableAck)
    {
      msg.destination = destination;
      msg.sender = sender;
      msg.sensor = sensor;
      msg.type = type;
      mSetCommand(msg, command);
      mSetRequestAck(msg, enableAck);
      mSetAck(msg, false);
      return msg;
    }
    
    /*
     * getType
     */
    char *getType(char *b, const char **index)
    {
      char *q = b;
      char *p = (char *) pgm_read_word(index);
      while (*q++ = pgm_read_byte(p++))
        ;
      *q = 0;
      return b;
    }
    
    /*
     * begin
     * wraps MySensors begin method; setup of RTC and led timers interrupt
     */
    void begin()
    {
    #ifdef DEBUG
      Serial.begin(BAUD_RATE);
    #endif
    #ifdef DSRTC
      // Attach RTC
      setSyncProvider(RTC.get);   // the function to get the time from the RTC
      setSyncInterval(60);
    #endif
    
      gw.begin(processRadioMessage, 0, true, 0);
    
      MsTimer2::set(200, ledTimersInterrupt);
      MsTimer2::start();
    
    #ifdef DEBUG
      Serial.print(getType(convBuf, &VAR_Type[S_FIRSTCUSTOM]));
    #endif
    }
    
    /*
     * processMQTTMessages
     * message handler for the PubSubClient
     */
    void processMQTTMessages(char* topic, byte* payload, unsigned int length)
    {
      processMQTTMessage(topic, payload, length);
    }
    
    /*
     * processMQTTMessage
     * processes MQTT messages, parses the topic, extracts radio address out of topic and sends them
     * to the respective radio
     */
    void processMQTTMessage(char* topic, byte* payload,
                            unsigned int length)
    {
      char *str, *p;
      uint8_t i = 0;
      buffer[0] = 0;
      buffsize = 0;
      uint8_t cmd = -1;
    
      for (str = strtok_r(topic, "/", &p); str && i < 5;
           str = strtok_r(NULL, "/", &p))
      {
        switch (i)
        {
          case 0:
            {
              if (strcmp_P(str, mqtt_prefix) != 0)
              { //look for MQTT_PREFIX
                return;			//Message not for us or malformatted!
              }
              break;
            }
          case 1:
            {
              msg.destination = atoi(str);	//NodeID
              break;
            }
          case 2:
            {
              msg.sensor = atoi(str);		//SensorID
              break;
            }
          case 3:
            {
              char match = 0;			//SensorType
              //strcpy(str,(char*)&str[2]);  //Strip VAR_
    
              for (uint8_t j = 0;
                   strcpy_P(convBuf, (char*) pgm_read_word(&(VAR_Type[j]))); j++)
              {
                if (strcmp((char*) &str[2], convBuf) == 0)
                { //Strip VAR_ and compare
                  match = j;
                  break;
                }
                if (j >= VAR_TOTAL)
                { // No match found!
                  match = VAR_TOTAL;	// last item.
                  break;
                }
              }
              msg.type = match;
              break;
            }
           case 4:
              {
                // support the command get and set; get will be mapped to a C_REQ and set to C_SET
                if (strcmp(str,MQTT_CMD_SET) == 0)
                {
                  cmd = C_SET;
                }
                else if (strcmp(str,MQTT_CMD_GET) == 0) {
                  cmd = C_REQ;
                } 
                else {
    #ifdef DEBUG
                  Serial.print("Received unsupported command - ignore: ");
                  Serial.println(str);
    #endif              
                }
              } 
        }
        i++;
      }						//Check if packge has payload
    
      if (cmd != -1) {
        char* ca;
        ca = (char *)payload;
        ca += length;
        *ca = '\0';
    
        msg.set((const char*)payload);			//Payload
    
        txBlink(1);
        // inject time
        if ((msg.destination == 0) && (msg.sensor == 199)) {
          unsigned long epoch = atol((char*)payload);
          if (epoch > 10000) {
    #ifdef DSRTC
            RTC.set(epoch); // this sets the RTC to the time from controller - which we do want periodically
    #endif
            setTime(epoch);
          }
    #ifdef DEBUG
          Serial.print("Time recieved ");
          Serial.println(epoch);
    #endif
        }
        //
        if (!gw.sendRoute(
              build(msg, GATEWAY_ADDRESS, msg.destination, msg.sensor, C_SET, msg.type,
                    0)))
          errBlink(1);
      }
    }
    
    //////////////////////////////////////////////////////////////////////////////////////////////////////////////
    // Led handling
    //////////////////////////////////////////////////////////////////////////////////////////////////////////////
    
    /*
     * ledTimersIntterupt
     */
    void ledTimersInterrupt() {
      if (countRx && countRx != 255) {
        // switch led on
        digitalWrite(RADIO_RX_LED_PIN, LOW);
      } else if (!countRx) {
        // switching off
        digitalWrite(RADIO_RX_LED_PIN, HIGH);
      }
      if (countRx != 255) {
        countRx--;
      }
    
      if (countTx && countTx != 255) {
        // switch led on
        digitalWrite(RADIO_TX_LED_PIN, LOW);
      } else if (!countTx) {
        // switching off
        digitalWrite(RADIO_TX_LED_PIN, HIGH);
      }
      if (countTx != 255) {
        countTx--;
      }
    
      if (countErr && countErr != 255) {
        // switch led on
        digitalWrite(RADIO_ERROR_LED_PIN, LOW);
      } else if (!countErr) {
        // switching off
        digitalWrite(RADIO_ERROR_LED_PIN, HIGH);
      }
      if (countErr != 255) {
        countErr--;
      }
    }
    
    
    void rxBlink(uint8_t cnt)
    {
      if (countRx == 255)
      {
        countRx = cnt;
      }
    }
    
    void txBlink(uint8_t cnt)
    {
      if (countTx == 255)
      {
        countTx = cnt;
      }
    }
    
    void errBlink(uint8_t cnt)
    {
      if (countErr == 255)
      {
        countErr = cnt;
      }
    }
    
    


  • @tomkxy Thanks, that helped! I seemed to have two problems:

    1. I did not know where to place the necessary includes! Thanks for the examples! Helped me a lot

    2)Regarding my problems from above (error: 'DEFAULT_CE_PIN' was not declared in this scope): I removed the whole libraries/MySensors dir and copied it again from the recent devel branch - this helped, error is gone and everything is compiling fine now 🙂

    Thanks again and cheers,

    Otto


  • Contest Winner

    @otto001 Great! And thanks @tomkxy for helping out. MySensors teamwork at it's best. This is what it's all about 😄



  • Hi again,

    and thanks again. signing seems to work now, unfortunately the sketch is too big for an UNO and I am right now struggling with the pinout and code adaptions to get it working with my W5100 shield and a mega...
    /edit: got it working.

    MyConfig.h:
    
    #ifdef SOFTSPI
        // Define the soft SPI pins used for NRF radio
        //MEGA
        const uint8_t SOFT_SPI_MISO_PIN = 15;
        const uint8_t SOFT_SPI_MOSI_PIN = 14;
        const uint8_t SOFT_SPI_SCK_PIN = 16;
    #endif
    
    EthernetGateway.ino:
    
    #define RADIO_CE_PIN        35 // 5  // radio chip enable
    #define RADIO_SPI_SS_PIN    34 // 6  // radio SPI serial select
    

    And exactly those pins I connected from radio to the mega... (just in case that anyone else is in need of this)

    Might be a little OT, but maybe someone here knows a solution for a strange problem:
    Whenever I am using an UNO or MEGA with my W5100 shield, I have to reset after every power-on to make the sketch run. Does anyone know this? Is there a solution? I am trying to adapt optiboot to wait additional 500ms when powering on, but no luck so far...

    Cheers,

    Otto


  • Hardware Contributor

    @otto001 said:

    Might be a little OT, but maybe someone here knows a solution for a strange problem:
    Whenever I am using an UNO or MEGA with my W5100 shield, I have to reset after every power-on to make the sketch run. Does anyone know this? Is there a solution? I am trying to adapt optiboot to wait additional 500ms when powering on, but no luck so far...

    Check here: https://www.sparkfun.com/products/11166 Their "version 2" shield has some notes about fixing the power up reset problem with regular internet shields. Not sure what it is but maybe their notes and schematic can clue you in to a help you implement a fix on your current board.


  • Hardware Contributor

    @TD22057 said:

    @otto001 said:

    Might be a little OT, but maybe someone here knows a solution for a strange problem:
    Whenever I am using an UNO or MEGA with my W5100 shield, I have to reset after every power-on to make the sketch run. Does anyone know this? Is there a solution? I am trying to adapt optiboot to wait additional 500ms when powering on, but no luck so far...

    Check here: https://www.sparkfun.com/products/11166 Their "version 2" shield has some notes about fixing the power up reset problem with regular internet shields. Not sure what it is but maybe their notes and schematic can clue you in to a help you implement a fix on your current board.

    And is the explanation of the fix that needs to be made:
    http://www.hobbyist.co.nz/?q=ethernet-shield-w5100
    http://forum.arduino.cc/index.php?topic=215798.msg1607035#msg1607035

    FYI I just ordered a Mega myself for a gateway since I figured the $7 to get one is a better expenditure of my time/money than me trying to get the MqttClientGateway + signing + RF69 radio + debug print to all fit on an Uno so I'll be doing this mod as well.



  • @Anticimex
    I think I may have found a bug in the code:
    I wanted to build a gateway that accepts both signed and unsigned messages. For this I used MySigningAtsha204Soft signer(false);.
    Now the gateway did accept unsigned messages, but also all messages sent to the gateway by another node with signing where unsigned.
    After changing this line in MySensor.cpp:

    if (signer.requestSignatures() && DO_SIGN(msg.sender))
    

    to

    if (signer.requestSignatures() || DO_SIGN(msg.sender))
    

    it worked as I wanted it. The gateway accepts unsigned messages from nodes but if a node expects signed messages, the messages to the gateway are also signed.
    This line means that the gateway requests signing from the node if he always requests signing or if the node requests signing.
    What do you think?


  • Contest Winner

    @fleinze I am not sure I understand the basic problem. What do you mean by " but also all messages sent to the gateway by another node with signing where unsigned.". The signing is based on requirement. A node (or gateway) either require messages sent to it to be signed or not. You have told your gateway that it is not supposed to require signed messages so it will accept unsigned messages sent to it. It is up to the node to configure if the node require signed messages in return. The gateway configuration has nothing to do with the preferences of a node. If you tell your gateway that it does not require signing, no messages to it will be signed. The opposite also holds true. If you configure it to require signed messages, all messages have to be signed. That is "working as designed".


  • Contest Winner

    @fleinze after reading the post a few times I think I understand better 🙂 you want nodes who require signed messages from the gateway to send signed messages to the gateway as well even though the gateway is configured to not require it. I don't have a problem with that change as long as it is also portable to the nodes (the same code works equally well for both gateways and nodes usecases). You are welcome to put a pull request on the code so I can have a closer look. If approved I will update the head post to reflect this changed behaviour.



  • 👍 The change only effects gateways. I will put a pull request, thx.


  • Contest Winner

    @fleinze do you have some special reason for why you want this? I don't see a real benefit and it would slightly increase the traffic. I suppose it would add some symmetry to node-gw communication.


  • Contest Winner

    @fleinze I am quite stressed out for the moment so I forget my own design 🙂
    What you actually should do is to configure the gateway to require signed messages. It will then do so, but only from nodes that in turn require signed messages. So you should not need to change anything, just set the GW to require signatures.



  • @Anticimex If I do so, this is the output of the Gateway:

    0;0;3;0;9;gateway started, id=0, parent=0, distance=0
    0;0;3;0;14;Gateway startup complete.
    0;0;3;0;9;no sign
    0;0;3;0;9;no sign
    0;0;3;0;9;no sign
    0;0;3;0;9;no sign
    0;0;3;0;9;read: 2-2-0 s=0,c=0,t=6,pt=0,l=0,sg=0:
    2;0;0;0;6;
    0;0;3;0;9;read: 2-2-0 s=1,c=0,t=30,pt=0,l=0,sg=0:
    2;1;0;0;30;
    0;0;3;0;9;no sign
    0;0;3;0;9;no sign
    0;0;3;0;9;no sign
    

    I tried to compile the node with deactivated signing feature and also with MySigningNone.
    The messages get rejected because in MySensor.cpp line 570 (developement branch) it is not checked if the sender requires signing:

    	if (signer.requestSignatures() && msg.destination == nc.nodeId && mGetLength(msg) && !mGetAck(msg) &&
    		(mGetCommand(msg) != C_INTERNAL ||
    		 (msg.type != I_GET_NONCE_RESPONSE && msg.type != I_GET_NONCE && msg.type != I_REQUEST_SIGNING &&
    		  msg.type != I_ID_REQUEST && msg.type != I_ID_RESPONSE &&
    		  msg.type != I_FIND_PARENT && msg.type != I_FIND_PARENT_RESPONSE)))
    

    My thought is that if I have a mixed network (signing and non-signing nodes) for some sensors I do not need signing (i.e. temperature-sensors). But if I have a button sensor that can actually switch something on or off (via the controller), it would be a security benefit if the messages from the sensor to the gateway are signed.


  • Contest Winner

    @fleinze If you have upgraded the library version, the signing table might have shifted in EEPROM. You then need to run the clear EEPROM sketch to reset the stored state in order for the gw/nodes to re-learn the existing signing preferences of the network.


  • Contest Winner

    @fleinze said:

    My thought is that if I have a mixed network (signing and non-signing nodes) for some sensors I do not need signing (i.e. temperature-sensors). But if I have a button sensor that can actually switch something on or off (via the controller), it would be a security benefit if the messages from the sensor to the gateway are signed.

    This is the exact usecase for the gw default behavior to only require signing from nodes that require signing in return. But I also got "no sign" errors after I flashed development branch yesterday on node/gw and only after I wiped the GW EEPROM I got it back online. So please try that. If it still does not work, I have to look closer, and see why this has broken because it has been working like that when I submitted the signing behavior.



  • @Anticimex I tried to make it work:

    • took a fresh copy of the development branch
    • activated signing in MyConfig.h
    • flashed the ClearEepromConfig sketch to both the test-gateway and the test-node.
    • built the gateway with MySigningAtsha204Soft signer; (requires signing by default)
    • built the node with MySigningNone signer;

    Still got the no sign message from the gateway...
    Please have a look at this.


  • Contest Winner

    @fleinze mysigningnone is a special case. It won't make signatures, so could you instead try with either hard or soft atsha?


  • Contest Winner

    @fleinze also, you can mix hard and soft atsha signing in a network, but you can't mix with "none". If you mark a node that it can do signing or require signing and use the "none" backend, nodes that use atsha (hard or soft) will not accept the signatures.



  • @Anticimex
    Ok, I built the node with MySigningAtsha204Soft signer(false); so it does not require signing and the gateway with MySigningAtsha204Soft signer;
    -> no sign error.
    How do I correctly build a node that does not sign in a mixed network (signing and non-signing nodes mixed)?


  • Contest Winner

    @fleinze
    In a network where you have "mixed" nodes, I would suggest something like this:
    The gateway has signing enabled and is set to require signatures and uses either hard or soft ATSHA (since you earlier wrote that you wanted the GW to sign messages to nodes that signed messages in return).
    For all nodes that need to be secure, enable signing and pick either hard or soft ATSHA (depending on the node hardware) and set them to require signatures.
    For all nodes that do not need to be secure, just disable signing alltogether, or pick any signing backend and set it to NOT require signatures.
    The result should be like this:
    A "insecure" node sends and receives unsigned messages to/from GW.
    A "secure" node sends and receives signed messages to/from GW.
    The GW will send signed messages to all nodes that has reported to the GW that they require signatures.
    The GW will send unsigned messages to all nodes that has reported to the GW that they do NOT require signatures.



  • @Anticimex In your original post you said:

    However, the difference is that the gateway will only require signed messages from nodes it knows in turn require signed messages.

    I made pull request 208 to match this behavior in the code:
    https://github.com/mysensors/Arduino/pull/208

    • If node is not a gateway, everything is as always. (1 || x)
    • if node is gateway and the sender requires signed messages, check signature (0 || 1)
    • if node is gateway and the sender does not requires signed messages, do not check signature (0 || 0)
      please consider adding this.

  • Contest Winner

    @fleinze Ok, lets discuss this in the PR and conclude it in this thread later.


  • Contest Winner

    Ok, the problem is identified and @fleinze has provided a fix that resolves the issue which is now merged to the development branch.
    Thanks for finding the issue and fixing it!
    There is no change in the signing behavior as it is documented. Now gateway does as it is supposed to.


  • Mod

    Great work @fleinze and @Anticimex !
    Reading this thread (yes, all of it) has made me ready to start configuring my nodes to use signing. Better start now, while my sensor network is still small 🙂


  • Hardware Contributor

    Compiling with the latest dev branch SecureActuator example

    #define MY_SIGNING_REQUEST_SIGNATURES

    I get following error:

    In file included from /Users/mm/Documents/Arduino165/libraries/MySensors/MySensor.h:157:0,
                     from SecureActuator.ino:58:
    /Users/mm/Documents/Arduino165/libraries/MySensors/core/MyTransport.cpp: In function 'void transportProcess()':
    /Users/mm/Documents/Arduino165/libraries/MySensors/core/MyTransport.cpp:92:10: error: 'MY_IS_GATEWAY' was not declared in this scope
        if ((!MY_IS_GATEWAY || DO_SIGN(sender)) &&
              ^
    /Users/mm/Documents/Arduino165/libraries/MySensors/core/MyTransport.cpp:93:20: error: 'nc' was not declared in this scope
         destination == nc.nodeId &&
                        ^
    Fehler beim Kompilieren.```
    
    Is this a problem of the code or of my setup?

  • Contest Winner

    @FotoFieber @hek something related to the recent refactoring? I have not had the opportunity to evaluate the effects on signing myself.


  • Admin

    @FotoFieber said:

    MY_SIGNING_REQUEST_SIGNATURES

    Let's see how Jenkins feels about this
    https://github.com/mysensors/Arduino/pull/231



  • @Anticimex Hello, I found this thread now. I'll ask a few more questions, sorry. This all seems really-really great.

    1. If someone wishes to use TMRh20's RF24Mesh instead of MySensors how would one sign the messages sent with that library? Is is even possible? What should be done to use that functionality?

    2. Also, as I understand it is possible to emulate the ATSHA204A, on what hardware is is possible (on Uno's ATMega too?)?

    3. Is it also possible to use whitelisting with RF24Mesh somehow?

    Again, sorry for any dumb questions and my poor English.


  • Contest Winner

    @Avamander

    1. Signing as described in this post is specific to users of the MySensors library. I suppose you could use any rf backend you like but you will need to use the MySensors library to get the nonce exchange and such. Or manually port the signing specifics out from the MySensors library and integrate them into another library. It's all open source.
    2. The software emulated atsha signing backend (for MySensors) is compatible with any Arduino product.
    3. See 1.

    Lastly, there are no dumb questions, only dumb answers 🙂



  • Hello,

    firstly, I must say that you're doing a great job and this section about security is very interesting.
    Sorry, but I want to ask a question: as regards the whitelisting system, how can I add more nodes to the list of the trusted nodes of the gateway?
    The part of code is this:

    [...]
    #ifdef MY_SECURE_NODE_WHITELISTING
    whitelist_entry_t node_whitelist[] = {
      { //I want to add nodes HERE:
        .nodeId = 55, // Just some value, this need to be changed  to the NodeId of the trusted node
        .serial = {0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09} } // This need to change to the serial of the trusted node
    };
    MySigningAtsha204 signer(true, 1, node_whitelist);  // Select ATSHA204A software signing backend with one entry in the whitelist
    [...]
    

    but the gateway continue to fail the verification of the trusted nodes that I inserted.

    To sum up, how can I add more nodeIDs and serials in addition to the number "55" presented in the example of the code?

    Thank you very much in advance,
    Silver978


  • Contest Winner

    @Silver978 Thanks!
    Try something like this:

    whitelist_entry_t node_whitelist[] = {
      {
        .nodeId = 55, // Just some value, this need to be changed  to the NodeId of the trusted node
        .serial = {0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09} // This need to change to the serial of the trusted node
      },
      {
        .nodeId = 56, // Just some value, this need to be changed  to the NodeId of the trusted node
        .serial = {0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09} // This need to change to the serial of the trusted node
      },
      {
        .nodeId = 57, // Just some value, this need to be changed  to the NodeId of the trusted node
        .serial = {0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09} // This need to change to the serial of the trusted node
      }
    };
    MySigningAtsha204 signer(true, 3, node_whitelist);  // Select ATSHA204A software signing backend with three entries in the whitelist
    
    

    Note the changed argument in the constructor (3 and not 1)



  • @Anticimex I tried also this syntax but I haven't changed the argument of the signer, so that number indicates how many nodes the gateway must trust?
    Thank you!


  • Contest Winner

    @Silver978 The number gives the number of entries in the whitelist the backend is expected to iterate over. Having a number for this allows the use of a potenitally very large whitelist but where "uninteresting" entries to this particular node can be ignored by having them last in the list and make sure the number does not cover those entries when searching the list.



  • @Anticimex Perfect, understood! Very good explanation and super-fast support, fantastic 🙂
    Thank you very much again!


  • Contest Winner



  • I would like to add security to my sensor network as well so I am very glad I found your thread. Your explanation are very detailed but I´m a guy who needs more practical examples. For example I would like to know, where exactly I have to activate this. I am using an arduino as an MQTT to Ethernet gateway and Mosquitto running on a raspberry pi. So all I have to do is activate the soft version in Myconfig.h, if not already activated, set a serial, re-upload to my arduino gateway and that´s it? I guess the Nodes have to get flashed again to get the serials as well, right?


  • Contest Winner

    @siod correct. You have to activate signing in both ends. The serial numbers you only need of you want to use whitelisting which is a bit more complex to set up. At the very least, for software signing, you need to enable it, have an unconnected analog pin on your boards, and set a hmac key.


  • Contest Winner

    EDIT: Topic post updates with a documentation link for those using the development branch.



  • Great topic as usual. I have a question please.

    You mentioned "Because it is pure-software however, it does not provide as good nonces (it uses the Arduino pseudo-random generator) and the HMAC key is stored in SW and is therefore readable if the memory is dumped. "

    Isn't this case also dangerous if the node which contains ATSHA chip was stolen ? The attacker will use the chip to be on the network ?

    Thanks.


  • Contest Winner

    @ahmedadelhosni thanks!
    Correct. That is why I also implemented whitelisting and node revocation. It's all described in the topic (and for development branch in doxygen, linked in the topic).



  • @Anticimex Thanks.

    So this gets me to my next question 🙂

    What I understood is that if I use whitelisting, I need to know the node ID of my sensor door to define it in the GW, correct ?
    I guess the below code is in the GW.

    whitelist_entry_t node_whitelist[] = {
      { .nodeId = 55, // Just some value, this need to be changed  to the NodeId of the trusted node
        .serial = {0x01,0x02,0x03,0x04,0x05,0x06,0x07,0x08,0x09} } // This need to change to the serial of the trusted node
    };
    

    another question, If I want to remove this node, do I have to reflash the GW and remove it ?


  • Contest Winner

    Yes.



  • @Anticimex mmmm
    Can't it be implemented in a way so that whitelist is to be defined by the controller ?
    For example, an internal message to be used to set the data.

    My concern is that auto assignation of nodes is flexible but at the same time you may need to choose which sensor node to be added to your whitelist. Thus if a Controller can show you all your nodes, like vera or domoticz, I guess this option can be added to the plugin to send an internal message to the GW to choose which sensor node to be added.

    Am I saying something logic ? 🙂


  • Contest Winner

    Possibly. But that would mean that security features is dictaded by the controller plugins and I do not like that at all.

    For months I have been trying to convince the Domoticz people that their ACK timeout is way to short and needs to be configurable for signing to work with Domoticz, but they do not even reply to me.

    And I do not trust controllers at all and want to have full control over all configuration aspects of the signing solution. So I do not think it is good to move that logic off the nodes (whitelisting can also be used by a node, communicating with another node).

    It is not only a GW that can have a whitelist. So although it might have been flexible to have it configurable, I think it compromises security (who knows, your controller could be hacked to inject a whitelist that permits a rogue node in your network).

    Of course, security is at some level compromised anyway if the controller is hacked. But it is not the signing solution that is compromised in this case.

    And ultimately, that feature would mean that the level of signing security and signing features for MySensors, becomes controller specific. And that I do not think is a good idea. If I want to change or improve the feature, the controller plugins all also have to be updated. It just becomes a too big turnaround for something as important as this in my opinion.



  • Yeah I got your point of view and convinced me. The only solution to implement this is that other communities works together to improve flexibility and security. This is not that easy of course as you have said.

    Thanks a lot for your time.


  • Contest Winner

    Thanks for understanding 🙂



  • Also in all cases I see that if your controller is being hacked there will be no use to play with the whitelist 🙂 The attacker has full control already to blow my house if he wants to 🙂

    Anyway maybe I need to reflash using signing to understand things more.

    Keep it up 🙂



  • Hi again.

    I managed to set run the softsign and everything seems fine till now. I just want to understand the system much more if you please.

    Now my Gateway sends this data when I turn on Light.
    ( I am using GatewayW5100 with Domoticz ) Development branch

    0;0;3;0;9;Eth: 0;0;3;0;18;PING
    0;0;3;0;9;Eth: 2;7;1;0;2;1
    0;0;3;0;9;send: 0-0-2-2 s=7,c=3,t=16,pt=0,l=0,sg=0,st=ok:
    0;0;3;0;9;read: 2-2-0 s=255,c=3,t=17,pt=6,l=25,sg=0:01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACAC
    0;0;3;0;9;Signing backend: ATSHA204Soft
    0;0;3;0;9;Message to process: 00020E01020731
    0;0;3;0;9;Current nonce: 01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACACAAAAAAAAAAAAAA
    0;0;3;0;9;HMAC: 135DBD85528E869ECC86C9C53679795D7FDA7B789DB7A0A74053C94FE8D668F0
    0;0;3;0;9;Signature in message: 015DBD85528E869ECC86C9C53679795D7FDA7B789DB7A0A7
    0;0;3;0;9;send: 0-0-2-2 s=7,c=1,t=2,pt=0,l=1,sg=1,st=ok:1
    

    I want to know how was the following data calculated please:

    • sg=0:01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACAC
    • Message to process: 00020E01020731
    • Current nonce: 01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACACAAAAAAAAAAAAAA
    • HMAC: 135DBD85528E869ECC86C9C53679795D7FDA7B789DB7A0A74053C94FE8D668F0
    • Signature in message: 015DBD85528E869ECC86C9C53679795D7FDA7B789DB7A0A7

    So how was sg= calculated ?
    What is message to process ?
    The nonce is depending on analogue signal + what ? to get this value ?
    I have already defined a random HMAC, but this one is different. It is a combination of what ?
    Finally what is signature in message ?

    On the other side I am trying to sniff the data by a serialgateway which is trying to hack the network. It only read the following data:

    0;0;3;0;9;read: 2-2-0 s=255,c=3,t=17,pt=6,l=25,sg=0:01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACAC                      
    2;255;3;0;17;01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACAC
    

    It is the sg only which it succeeded to read. Also I do understood form the topic that the attacker can read values sent between nodes unencrypted, but here I can't figure out where is the unencrypted data which can be sniffed by the attacker but we don't bother ourselves with cas we protect our network by using signing.

    Lots of questions but I am trying to understand the logic and architecture in details to understand what I am doing 🙂

    Thanks a lot.


  • Contest Winner

    Hi,
    I suggest you read the doxygen documentation for signing carefully as it answers the questions you have. Signing does not in any way prevent anyone from reading your messages. As I have described in the documentation, it provides authenticity. That is, you can trust the sender of a signed message to be yours and not someone else's.



  • @Anticimex Yeah I understand it doesn't prevent anyone from reading my messages, and that's what I am trying to do now. To hack the system depending on the data available. But when I read the post I understood that encryption is not a priority as signing is enough now to prevent an attacker from sending data, but it can read text. I thought I will find clear text describing that I am sending to node X value Y but I found this 2;255;3;0;17;01F470C061A0B9FF3DE248835736E2B85E31C8D6D1844AACAC
    so I wanted to ask.

    Anyway I'll re read the documents more carefully and return to you back please. Thanks.


  • Contest Winner

    Encryption is available for both rf24 and rf69 radios. But encryption and signing are two different things, and I work with signing. And signing is more efficient against hackers than encryption. Encryption is more if you care about privacy.



  • @Anticimex @hek is it possible to encrypt the data sent over nrf modules using this https://github.com/DavyLandman/AESLib ???


  • Contest Winner

    @noelgeorgi AES encryption is already supported for both NRF24 and RFM69 radios. See MyConfig.h on the development branch for the flag to use. Doyxgen documentation I link to in this thread on the topic post also contain the personalization instructions for encryption on the development branch.



  • @Anticimex said:

    @noelgeorgi AES encryption is already supported for both NRF24 and RFM69 radios. See MyConfig.h on the development branch for the flag to use. Doyxgen documentation I link to in this thread on the topic post also contain the personalization instructions for encryption on the development branch.

    Thanks for the info.
    I'm getting the following errors when compiling securitypersonalizer:

    Arduino: 1.6.7 (Linux), Board: "Arduino/Genuino Mega or Mega 2560, ATmega2560 (Mega 2560)"
    
    /home/technoman/Arduino/libraries/MySensors/examples/SecurityPersonalizer/SecurityPersonalizer.ino:83:28: fatal error: sha204_library.h: No such file or directory
     #include <sha204_library.h>
                                ^
    compilation terminated.
    exit status 1
    Error compiling.
    
      This report would have more information with
      "Show verbose output during compilation"
      enabled in File > Preferences.
    
    

    doesnt the ATSHA204 library has ATSHA204.h


  • Contest Winner

    @noelgeorgi master or development branch?



  • @Anticimex development branch


  • Contest Winner

    @noelgeorgi have you made changes to the sketch? Jenkins builds it without problems.



  • @Anticimex no, I'm using the sketch from the examples.....😕
    am i missing some libraries????


  • Contest Winner

    @noelgeorgi if you have cloned the arduino git you have everything you need. I can build that sketch myself and have used it many times. I suspect you have a problem with your environment if you get compilation errors in it. We continously test the repository and build all examples nightly to make sure it all works as it is supposed to. You can see this at ci.mysensors.org



  • @Anticimex sorry for the interruption, i was using rsync to get eveything synced and i might have messed something uop, sha204 library was missing from my libraries folder, now everything works, thanks for the prompt and fast support.


  • Contest Winner

    👍


  • Hardware Contributor

    @Anticimex - thanks for this. I finally got around to enabling signing on everything while I was testing out my new board design. I followed your schematic from your board design when I built mine and I just went through the instructions to configure the crypto chip and everything went smoothly. Thanks for the work and the good instructions!


  • Contest Winner

    @TD22057 that's really nice to hear! Are you using a stable release or the development/beta channel?


  • Hardware Contributor

    @Anticimex
    I used the signing sketch on the dev branch - I'm in the process of shifting my stuff over to the dev branch from 1.5 now.


  • Contest Winner

    Ok, cool. As you might have noticed, personalization has changed a bit on development. Also, whitelisting works slightly differently/better.



  • Isn't blacklisting cool too to use ?
    Like, if we know the "name" of the sensor lost, we push it to the black list and voila ? This way, we don't have to put a long list a agreed sensors if we have only two nodes outside the house.
    ... I think.


  • Contest Winner

    @Pierre-P because if a node is lost, you no longer control that node. So anyone could reprogram it to have it identify itself in any way possible. We have to assume an attacker has full source code access, so they can rewrite the signing algorithm to use a fake serial as salt for the signature to trick the GW to believe it is a new node. A whitelist mean the attacker has to know the ID/serial of one of the nodes you trust. Which they won't know unless they can get access to that node.



  • Thanks for the complet answer !


  • Contest Winner

    @Pierre-P you are welcome. Security is best when it is totally open and the implementation is aware of this. That makes it quite difficult to circumvent, and it also allows to be challenged. That way, with many eyes examining it, it gets stronger and stronger 🙂 I welcome all attempts to crack it. White or black hat style.


  • Hardware Contributor

    Hello all! What would require less flash memory? Signing by ATSHA204A Chip or by software?


  • Contest Winner

    Signing by ATSHA204A require less flash memory, but takes longer to execute (at least on 8MHz AtMega328).


Log in to reply
 

173 out of 491

Suggested Topics

0
Online

11.4k
Users

11.1k
Topics

112.7k
Posts