Skip to content
  • MySensors
  • OpenHardware.io
  • Categories
  • Recent
  • Tags
  • Popular
Skins
  • Light
  • Brite
  • Cerulean
  • Cosmo
  • Flatly
  • Journal
  • Litera
  • Lumen
  • Lux
  • Materia
  • Minty
  • Morph
  • Pulse
  • Sandstone
  • Simplex
  • Sketchy
  • Spacelab
  • United
  • Yeti
  • Zephyr
  • Dark
  • Cyborg
  • Darkly
  • Quartz
  • Slate
  • Solar
  • Superhero
  • Vapor

  • Default (No Skin)
  • No Skin
Collapse
Brand Logo
  1. Home
  2. Development
  3. Browser-based firmware generator

Browser-based firmware generator

Scheduled Pinned Locked Moved Development
108 Posts 14 Posters 28.1k Views 18 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • AnticimexA Anticimex

    @rakeshpai I have never used node manager myself but it appears that it consumes quite some memory. I have to my best effort optimized the signing code to use as little memory as possible. That said, it could very well be possible to optimize it further, but I would then need other eyes on it. I have not looked into node manager, but as I understand it, it relies heavily on c++ (signing code is pretty much just c) and the atmega compiler optimized c++ code horribly bad. So I think that it is in nodemanager the biggest memory savings can be made.

    rakeshpaiR Offline
    rakeshpaiR Offline
    rakeshpai
    wrote on last edited by
    #38

    @Anticimex That's the feeling I had as well, considering that even after I disabled signing, while the sketch fits in memory, it still seemed to complain about low memory. @user2684 does this match your experience too?

    AnticimexA U 2 Replies Last reply
    0
    • rakeshpaiR rakeshpai

      @Anticimex That's the feeling I had as well, considering that even after I disabled signing, while the sketch fits in memory, it still seemed to complain about low memory. @user2684 does this match your experience too?

      AnticimexA Offline
      AnticimexA Offline
      Anticimex
      Contest Winner
      wrote on last edited by
      #39

      @rakeshpai the warning is because the heap and stack is shared. So as the program claims storage for variables (typically globals statics and classes) the ram available for stack shrinks. And at some point it becomes to small. But the tools cannot determine when that happens as such analysis is quite complex. So they warn when it "feels" to be getting tight. But it ultimately comes down to how the program behaves and how the code handles function calls and parameters. So if you get the warning, it does not mean there is a problem. But you have entered what somebody decided to be the "danger zone" (probably based on typical sketch stack usage).

      Do you feel secure today? No? Start requiring some signatures and feel better tomorrow ;)

      1 Reply Last reply
      1
      • rakeshpaiR rakeshpai

        @Anticimex That's the feeling I had as well, considering that even after I disabled signing, while the sketch fits in memory, it still seemed to complain about low memory. @user2684 does this match your experience too?

        U Offline
        U Offline
        user2684
        Contest Winner
        wrote on last edited by
        #40

        @rakeshpai sorry for the delay! NodeManager consumes some good memory indeed but I should have optimized this in v1.5 and the problem should have gone unless a lot of sensors are registered but this is not your case. Are you already using this version?
        As a side note, I've noticed when MY_RF24_ENABLE_ENCRYPTION is defined a good 20% of memory got consumed on a pro mini but I guess this is not something we can do much about :)

        1 Reply Last reply
        0
        • U Offline
          U Offline
          user2684
          Contest Winner
          wrote on last edited by
          #41

          Correction, this has been solved only in the development version :-( (https://forum.mysensors.org/topic/6183/nodemanager-plugin-for-a-rapid-development-of-battery-powered-sensors/168).
          But I'd recommend not to move there because a few things have changed and I'd prefer to provide you with a better recap of the changes to make your life easier. Probably best is that I put together a v1.5.1 version with just that issue fixed for you to use. I'll work on it and let you know between today and tomorrow. Thanks!

          rakeshpaiR 1 Reply Last reply
          0
          • AnticimexA Offline
            AnticimexA Offline
            Anticimex
            Contest Winner
            wrote on last edited by
            #42

            Generally, the beta branch of the MySensors library has much "better" memory footprint than the current release. This is also true for signing.

            Do you feel secure today? No? Start requiring some signatures and feel better tomorrow ;)

            1 Reply Last reply
            0
            • U user2684

              Correction, this has been solved only in the development version :-( (https://forum.mysensors.org/topic/6183/nodemanager-plugin-for-a-rapid-development-of-battery-powered-sensors/168).
              But I'd recommend not to move there because a few things have changed and I'd prefer to provide you with a better recap of the changes to make your life easier. Probably best is that I put together a v1.5.1 version with just that issue fixed for you to use. I'll work on it and let you know between today and tomorrow. Thanks!

              rakeshpaiR Offline
              rakeshpaiR Offline
              rakeshpai
              wrote on last edited by
              #43

              @user2684 My version management is pretty crappy at the moment ;) but I'm simply using the latest version of NodeManager.cpp and NodeManager.h from the master branch on GitHub. Users keep a local copy of NodeManager for purposes of offline availability, but they are forced to update their local versions of NodeManager everyday. So, it's safe to assume that within a margin of error of a day, everyone's on the latest version of your checkins on GitHub. (I know this isn't ideal and I should do version management better, but I too like to live dangerously B).)

              Also, while MySensors itself isn't shipped as part of the download bundle, I've manually used the latest version from the dev branch in GitHub as well, in my arduino libraries folder.

              Maybe I'm just doing something stupid that's causing the memory usage to inflate? I'll wait for whenever you can get a chance to review the output code. Thanks for looking at this!

              U 1 Reply Last reply
              0
              • rakeshpaiR rakeshpai

                @user2684 My version management is pretty crappy at the moment ;) but I'm simply using the latest version of NodeManager.cpp and NodeManager.h from the master branch on GitHub. Users keep a local copy of NodeManager for purposes of offline availability, but they are forced to update their local versions of NodeManager everyday. So, it's safe to assume that within a margin of error of a day, everyone's on the latest version of your checkins on GitHub. (I know this isn't ideal and I should do version management better, but I too like to live dangerously B).)

                Also, while MySensors itself isn't shipped as part of the download bundle, I've manually used the latest version from the dev branch in GitHub as well, in my arduino libraries folder.

                Maybe I'm just doing something stupid that's causing the memory usage to inflate? I'll wait for whenever you can get a chance to review the output code. Thanks for looking at this!

                U Offline
                U Offline
                user2684
                Contest Winner
                wrote on last edited by
                #44

                @rakeshpai great so yes, you are using the latest version and btw, very smart method for keeping it up to date ;-) No, you are not doing anything stupid, it was me doing something stupid when releasing the latest version of NodeManager :-) Of course together with encryption enabled this would end up consuming almost all the memory but the original sin was on my side. I'll definitely work on a hotfix for you, otherwise the code would become useless. And of course I'll review the output code but apologize in advance, real life will be very demanding for me in the next few months so my response will be almost always delayed :-(

                rakeshpaiR 1 Reply Last reply
                0
                • U user2684

                  @rakeshpai great so yes, you are using the latest version and btw, very smart method for keeping it up to date ;-) No, you are not doing anything stupid, it was me doing something stupid when releasing the latest version of NodeManager :-) Of course together with encryption enabled this would end up consuming almost all the memory but the original sin was on my side. I'll definitely work on a hotfix for you, otherwise the code would become useless. And of course I'll review the output code but apologize in advance, real life will be very demanding for me in the next few months so my response will be almost always delayed :-(

                  rakeshpaiR Offline
                  rakeshpaiR Offline
                  rakeshpai
                  wrote on last edited by
                  #45

                  @user2684 No problem with the delays - I've got enough things to do in the meantime. All those real-life things coming in the way all the time, amirite? :)

                  U 1 Reply Last reply
                  0
                  • rakeshpaiR rakeshpai

                    @user2684 No problem with the delays - I've got enough things to do in the meantime. All those real-life things coming in the way all the time, amirite? :)

                    U Offline
                    U Offline
                    user2684
                    Contest Winner
                    wrote on last edited by
                    #46

                    @rakeshpai yes indeed :-)
                    I wonder if you can give a try to https://github.com/user2684/NodeManager/tree/release/1.5.1 before I'll make publicly available. It is basically the same v1.5 version you are using which just applied the memory optimization patch on top. It should save 25% of the memory so it is a huge fix :-)
                    Let me know if works better. Something else I recommend to disable when you are approaching the memory limit is #define DEBUG 1 which will prevent storing all the debug messages giving you another boost.
                    Generally speaking you need to stay pretty distant from the 100% memory utilization with NodeManager: since it is allocating its objects dynamically (which is not a good practice but I couldn't find a better way), you can get weird behavior even if the compiler doesn't not complain.

                    Thanks!

                    rakeshpaiR 1 Reply Last reply
                    1
                    • U user2684

                      @rakeshpai yes indeed :-)
                      I wonder if you can give a try to https://github.com/user2684/NodeManager/tree/release/1.5.1 before I'll make publicly available. It is basically the same v1.5 version you are using which just applied the memory optimization patch on top. It should save 25% of the memory so it is a huge fix :-)
                      Let me know if works better. Something else I recommend to disable when you are approaching the memory limit is #define DEBUG 1 which will prevent storing all the debug messages giving you another boost.
                      Generally speaking you need to stay pretty distant from the 100% memory utilization with NodeManager: since it is allocating its objects dynamically (which is not a good practice but I couldn't find a better way), you can get weird behavior even if the compiler doesn't not complain.

                      Thanks!

                      rakeshpaiR Offline
                      rakeshpaiR Offline
                      rakeshpai
                      wrote on last edited by rakeshpai
                      #47

                      @user2684 Tried with the 1.5.1 release, and it fits!

                      Sketch uses 25,786 bytes (83%) of program storage space. Maximum is 30,720 bytes.
                      Global variables use 1,716 bytes (83%) of dynamic memory, leaving 332 bytes for local variables. Maximum is 2,048 bytes.
                      Low memory available, stability problems may occur.
                      

                      It's still complaining about low memory, and from what I understand of what you said, this is a problem, but this is definitely an awesome direction. It's also a concern that I still haven't added any sensors yet - this is just a bare serial gateway sketch. However, I'm still excited. :metal:

                      I've kept debug off by default in the config for now, so debug messages aren't pushing us to the limit here.

                      EDIT: I must add, the result above is with signing and encryption, and all that good stuff.

                      AnticimexA 1 Reply Last reply
                      0
                      • rakeshpaiR rakeshpai

                        @user2684 Tried with the 1.5.1 release, and it fits!

                        Sketch uses 25,786 bytes (83%) of program storage space. Maximum is 30,720 bytes.
                        Global variables use 1,716 bytes (83%) of dynamic memory, leaving 332 bytes for local variables. Maximum is 2,048 bytes.
                        Low memory available, stability problems may occur.
                        

                        It's still complaining about low memory, and from what I understand of what you said, this is a problem, but this is definitely an awesome direction. It's also a concern that I still haven't added any sensors yet - this is just a bare serial gateway sketch. However, I'm still excited. :metal:

                        I've kept debug off by default in the config for now, so debug messages aren't pushing us to the limit here.

                        EDIT: I must add, the result above is with signing and encryption, and all that good stuff.

                        AnticimexA Offline
                        AnticimexA Offline
                        Anticimex
                        Contest Winner
                        wrote on last edited by
                        #48

                        @rakeshpai looks good. I get that warning myself and I have only seen issues when I push 90% but it also depend on what your sketch does. In any case, the warning is no guarantee for failure so I think it can be considered acceptable in most situations.

                        Do you feel secure today? No? Start requiring some signatures and feel better tomorrow ;)

                        1 Reply Last reply
                        0
                        • rakeshpaiR Offline
                          rakeshpaiR Offline
                          rakeshpai
                          wrote on last edited by
                          #49

                          Output from the gateway's serial:

                          NodeManager v1.5.1
                          INT1 M=255
                          INT2 M=255
                          0;255;3;0;14;Gateway startup complete.
                          0;255;0;0;18;2.2.0-beta
                          RADIO OK
                          0;255;3;0;11;Gateway
                          0;255;3;0;12;1.0
                          PRES I=200, T=23
                          0;200;0;0;23;
                          READY
                          
                          MY I=0 M=1
                          SEND D=0 I=200 C=0 T=48 S=STARTED I=0 F=0.00
                          0;200;1;0;48;STARTED
                          

                          Output from a dummy node's serial:

                          REG I=1 P=14 P=16 T=23
                          PWR G=0 REG I=1 P=14 P=16 T=23
                          PWR G=0 V=12
                          NodeManager v1.5.1
                          INT1 M=255
                          INT2 M=255
                          

                          Ok, so the both aren't talking to each other yet, and I'll investigate that. BUT! This is NodeManager 1.5.1 + Signing + Encryption, flashed on to two Arduino Pro Minis 3.3v, one with a serial gateway sketch, and the other with the node for a simple sketch of a LDR sensor, both generated from the UI!

                          You can't see me here, but I'm doing a little dance! :P

                          While I complete my little dance here, if you can spot anything obvious about why they aren't talking to each other, please let me know.

                          Also, minor UI update: New 'Are you sure?' dialogs when you are performing any deletions on the UI.

                          AnticimexA 1 Reply Last reply
                          0
                          • rakeshpaiR rakeshpai

                            Output from the gateway's serial:

                            NodeManager v1.5.1
                            INT1 M=255
                            INT2 M=255
                            0;255;3;0;14;Gateway startup complete.
                            0;255;0;0;18;2.2.0-beta
                            RADIO OK
                            0;255;3;0;11;Gateway
                            0;255;3;0;12;1.0
                            PRES I=200, T=23
                            0;200;0;0;23;
                            READY
                            
                            MY I=0 M=1
                            SEND D=0 I=200 C=0 T=48 S=STARTED I=0 F=0.00
                            0;200;1;0;48;STARTED
                            

                            Output from a dummy node's serial:

                            REG I=1 P=14 P=16 T=23
                            PWR G=0 REG I=1 P=14 P=16 T=23
                            PWR G=0 V=12
                            NodeManager v1.5.1
                            INT1 M=255
                            INT2 M=255
                            

                            Ok, so the both aren't talking to each other yet, and I'll investigate that. BUT! This is NodeManager 1.5.1 + Signing + Encryption, flashed on to two Arduino Pro Minis 3.3v, one with a serial gateway sketch, and the other with the node for a simple sketch of a LDR sensor, both generated from the UI!

                            You can't see me here, but I'm doing a little dance! :P

                            While I complete my little dance here, if you can spot anything obvious about why they aren't talking to each other, please let me know.

                            Also, minor UI update: New 'Are you sure?' dialogs when you are performing any deletions on the UI.

                            AnticimexA Offline
                            AnticimexA Offline
                            Anticimex
                            Contest Winner
                            wrote on last edited by
                            #50

                            @rakeshpai The first question that comes to mind is, have you double checked your AES personalization? If you run with encryption enabled and have a mismatch in the PSK then nothing comes through in any direciton.
                            You can execute the personalizer without flags set and it should dump the EEPROM data on serial.

                            Do you feel secure today? No? Start requiring some signatures and feel better tomorrow ;)

                            rakeshpaiR 1 Reply Last reply
                            0
                            • AnticimexA Anticimex

                              @rakeshpai The first question that comes to mind is, have you double checked your AES personalization? If you run with encryption enabled and have a mismatch in the PSK then nothing comes through in any direciton.
                              You can execute the personalizer without flags set and it should dump the EEPROM data on serial.

                              rakeshpaiR Offline
                              rakeshpaiR Offline
                              rakeshpai
                              wrote on last edited by
                              #51

                              @Anticimex From what it looks like at the moment, it looks like I'm hitting up against the memory issue. (But I might be wrong.)

                              I don't know why the two nodes weren't communicating in my previous post. However, I decided to start from scratch, and ran the ClearEEPROM sketch on both my arduinos. Then, I disabled personalisation on both the gateway and the node sketches, flashed them to the boards, and watched ID requests come on the gateway's serial output, from the node. (I don't have a controller running.) So, everything looks good so far.

                              Then I ran the SecurityPersonalizer on the gateway, and flashed the gateway sketch to it again. This was the output of the personalizer:

                              +------------------------------------------------------------------------------------+
                              |                           MySensors security personalizer                          |
                              +------------------------------------------------------------------------------------+
                              
                              +------------------------------------------------------------------------------------+
                              |                               Configuration settings                               |
                              +------------------------------------------------------------------------------------+
                              | * Guided personalization/storage of keys in EEPROM                                 |
                              | * Software based personalization (no ATSHA204A usage whatsoever)                   |
                              | * Will not require any UART confirmations                                          |
                              | * Will store HMAC key to EEPROM                                                    |
                              | * Will store AES key to EEPROM                                                     |
                              | * Will store soft serial to EEPROM                                                 |
                              +------------------------------------------------------------------------------------+
                              
                              +------------------------------------------------------------------------------------+
                              |                           Hardware security peripherals                            |
                              +--------------+--------------+--------------+------------------------------+--------+
                              | Device       | Status       | Revision     | Serial number                | Locked |
                              +--------------+--------------+--------------+------------------------------+--------+
                              | AVR          | DETECTED     | N/A          | N/A (generation required)    | N/A    |
                              +--------------+--------------+--------------+------------------------------+--------+
                              
                              +------------------------------------------------------------------------------------+
                              |                                    Key storage                                     |
                              +--------+--------+------------------------------------------------------------------+
                              | Key ID | Status | Key                                                              |
                              +--------+--------+------------------------------------------------------------------+
                              | HMAC   | OK     | 26E779A2876F7FC756C96B60085927EA24F7D8C57E117107BAE92C57080429E1 |
                              | AES    | OK     | 1E4FAF1129029D34C4832F6B48044C8B                                 |
                              | SERIAL | OK     | BED2C17579AF3CDA6D                                               |
                              +--------+--------+------------------------------------------------------------------+
                              
                              +------------------------------------------------------------------------------------+
                              |                                       EEPROM                                       |
                              +--------+--------+------------------------------------------------------------------+
                              | Key ID | Status | Key                                                              |
                              +--------+--------+------------------------------------------------------------------+
                              | HMAC   | OK     | 26E779A2876F7FC756C96B60085927EA24F7D8C57E117107BAE92C57080429E1 |
                              | AES    | OK     | 1E4FAF1129029D34C4832F6B48044C8B                                 |
                              | SERIAL | OK     | BED2C17579AF3CDA6D                                               |
                              +--------+--------+------------------------------------------------------------------+
                              
                              +------------------------------------------------------------------------------------+
                              |                      This nodes whitelist entry on other nodes                     |
                              +------------------------------------------------------------------------------------+
                              {.nodeId = <ID of this node>,.serial = {0xBE,0xD2,0xC1,0x75,0x79,0xAF,0x3C,0xDA,0x6D}}
                              +------------------------------------------------------------------------------------+
                              
                              +------------------------------------------------------------------------------------+
                              |                                  WHAT TO DO NEXT?                                  |
                              +------------------------------------------------------------------------------------+
                              | This device has now been personalized. Run this sketch with its current settings   |
                              | on all the devices in your network that have security enabled.                     |
                              +------------------------------------------------------------------------------------+
                              
                              +------------------------------------------------------------------------------------+
                              |                                  Execution result                                  |
                              +------------------------------------------------------------------------------------+
                              | SUCCESS                                                                            |
                              +------------------------------------------------------------------------------------+
                              

                              So, that's a success. A little confusingly, the output says 'Run this sketch with its current settings on all the devices in your network that have security enabled.' However, won't the device IDs / serial key be different on every device?

                              I then flashed the SecurityPersonalizer and node sketch on the other arduino. The output of the security personalizer is as follows, notably with a different serial key:

                              +------------------------------------------------------------------------------------+
                              |                           MySensors security personalizer                          |
                              +------------------------------------------------------------------------------------+
                              
                              +------------------------------------------------------------------------------------+
                              |                               Configuration settings                               |
                              +------------------------------------------------------------------------------------+
                              | * Guided personalization/storage of keys in EEPROM                                 |
                              | * Software based personalization (no ATSHA204A usage whatsoever)                   |
                              | * Will not require any UART confirmations                                          |
                              | * Will store HMAC key to EEPROM                                                    |
                              | * Will store AES key to EEPROM                                                     |
                              | * Will store soft serial to EEPROM                                                 |
                              +------------------------------------------------------------------------------------+
                              
                              +------------------------------------------------------------------------------------+
                              |                           Hardware security peripherals                            |
                              +--------------+--------------+--------------+------------------------------+--------+
                              | Device       | Status       | Revision     | Serial number                | Locked |
                              +--------------+--------------+--------------+------------------------------+--------+
                              | AVR          | DETECTED     | N/A          | N/A (generation required)    | N/A    |
                              +--------------+--------------+--------------+------------------------------+--------+
                              
                              +------------------------------------------------------------------------------------+
                              |                                    Key storage                                     |
                              +--------+--------+------------------------------------------------------------------+
                              | Key ID | Status | Key                                                              |
                              +--------+--------+------------------------------------------------------------------+
                              | HMAC   | OK     | 26E779A2876F7FC756C96B60085927EA24F7D8C57E117107BAE92C57080429E1 |
                              | AES    | OK     | 1E4FAF1129029D34C4832F6B48044C8B                                 |
                              | SERIAL | OK     | 31D46BF4ADBEF72803                                               |
                              +--------+--------+------------------------------------------------------------------+
                              
                              +------------------------------------------------------------------------------------+
                              |                                       EEPROM                                       |
                              +--------+--------+------------------------------------------------------------------+
                              | Key ID | Status | Key                                                              |
                              +--------+--------+------------------------------------------------------------------+
                              | HMAC   | OK     | 26E779A2876F7FC756C96B60085927EA24F7D8C57E117107BAE92C57080429E1 |
                              | AES    | OK     | 1E4FAF1129029D34C4832F6B48044C8B                                 |
                              | SERIAL | OK     | 31D46BF4ADBEF72803                                               |
                              +--------+--------+------------------------------------------------------------------+
                              
                              +------------------------------------------------------------------------------------+
                              |                      This nodes whitelist entry on other nodes                     |
                              +------------------------------------------------------------------------------------+
                              {.nodeId = <ID of this node>,.serial = {0x31,0xD4,0x6B,0xF4,0xAD,0xBE,0xF7,0x28,0x03}}
                              +------------------------------------------------------------------------------------+
                              
                              +------------------------------------------------------------------------------------+
                              |                                  WHAT TO DO NEXT?                                  |
                              +------------------------------------------------------------------------------------+
                              | This device has now been personalized. Run this sketch with its current settings   |
                              | on all the devices in your network that have security enabled.                     |
                              +------------------------------------------------------------------------------------+
                              
                              +------------------------------------------------------------------------------------+
                              |                                  Execution result                                  |
                              +------------------------------------------------------------------------------------+
                              | SUCCESS                                                                            |
                              +------------------------------------------------------------------------------------+
                              

                              Then, I watched the gateway serial output. It looks as follows:

                              NodeManager v1.5.1
                              INT1 M=255
                              INT2 M=255
                              0;255;3;0;14;Gateway startup complete.
                              0;255;0;0;18;2.2.0-beta
                              RADIO OK
                              0;255;3;0;11;Gateway
                              0;255;3;0;12;1.0
                              PRES I=200, T=23
                              0;200;0;0;23;
                              READY
                              
                              MY I=0 M=255
                              SEND D=0 I=200 C=0 T=48 S=STARTED I=0 F=0.00
                              0;200;1;0;48;STARTED
                              // stable till here - then I power up my node...
                              255;118;3;0;3;
                              RECV S=255 I=118 C=3 T=3 P=
                              255;x00
                              

                              ...and it just dies. Sometimes it prints non-printable characters to the serial output, sometimes it goes into a reset loop, and sometimes the only way to reset it is to cycle power to it.

                              However, it does appear to receive the message from the node before it dies, judging by the 255;118;3;0;3; message, which is an ID request. It looks like the gateway dies when trying to handle that incoming message maybe? If I don't power up the node, the gateway seems to appear stable, but once I power up a node, the gateway receives the ID request, and it dies.

                              Also, I don't suspect this is a hardware issue. I've used the same hardware in past experiments with MySensors (without signing and NodeManager at the time), and it has worked fine. Also, I tried all this without personalisation earlier today, and it has worked fine. I've added 47uF electrolytic caps to both radios, so power shouldn't be an issue.

                              So, the erratic behaviour of the gateway when receiving a message seems to be suggesting that we're hitting memory limits, maybe?

                              AnticimexA U 2 Replies Last reply
                              0
                              • rakeshpaiR rakeshpai

                                @Anticimex From what it looks like at the moment, it looks like I'm hitting up against the memory issue. (But I might be wrong.)

                                I don't know why the two nodes weren't communicating in my previous post. However, I decided to start from scratch, and ran the ClearEEPROM sketch on both my arduinos. Then, I disabled personalisation on both the gateway and the node sketches, flashed them to the boards, and watched ID requests come on the gateway's serial output, from the node. (I don't have a controller running.) So, everything looks good so far.

                                Then I ran the SecurityPersonalizer on the gateway, and flashed the gateway sketch to it again. This was the output of the personalizer:

                                +------------------------------------------------------------------------------------+
                                |                           MySensors security personalizer                          |
                                +------------------------------------------------------------------------------------+
                                
                                +------------------------------------------------------------------------------------+
                                |                               Configuration settings                               |
                                +------------------------------------------------------------------------------------+
                                | * Guided personalization/storage of keys in EEPROM                                 |
                                | * Software based personalization (no ATSHA204A usage whatsoever)                   |
                                | * Will not require any UART confirmations                                          |
                                | * Will store HMAC key to EEPROM                                                    |
                                | * Will store AES key to EEPROM                                                     |
                                | * Will store soft serial to EEPROM                                                 |
                                +------------------------------------------------------------------------------------+
                                
                                +------------------------------------------------------------------------------------+
                                |                           Hardware security peripherals                            |
                                +--------------+--------------+--------------+------------------------------+--------+
                                | Device       | Status       | Revision     | Serial number                | Locked |
                                +--------------+--------------+--------------+------------------------------+--------+
                                | AVR          | DETECTED     | N/A          | N/A (generation required)    | N/A    |
                                +--------------+--------------+--------------+------------------------------+--------+
                                
                                +------------------------------------------------------------------------------------+
                                |                                    Key storage                                     |
                                +--------+--------+------------------------------------------------------------------+
                                | Key ID | Status | Key                                                              |
                                +--------+--------+------------------------------------------------------------------+
                                | HMAC   | OK     | 26E779A2876F7FC756C96B60085927EA24F7D8C57E117107BAE92C57080429E1 |
                                | AES    | OK     | 1E4FAF1129029D34C4832F6B48044C8B                                 |
                                | SERIAL | OK     | BED2C17579AF3CDA6D                                               |
                                +--------+--------+------------------------------------------------------------------+
                                
                                +------------------------------------------------------------------------------------+
                                |                                       EEPROM                                       |
                                +--------+--------+------------------------------------------------------------------+
                                | Key ID | Status | Key                                                              |
                                +--------+--------+------------------------------------------------------------------+
                                | HMAC   | OK     | 26E779A2876F7FC756C96B60085927EA24F7D8C57E117107BAE92C57080429E1 |
                                | AES    | OK     | 1E4FAF1129029D34C4832F6B48044C8B                                 |
                                | SERIAL | OK     | BED2C17579AF3CDA6D                                               |
                                +--------+--------+------------------------------------------------------------------+
                                
                                +------------------------------------------------------------------------------------+
                                |                      This nodes whitelist entry on other nodes                     |
                                +------------------------------------------------------------------------------------+
                                {.nodeId = <ID of this node>,.serial = {0xBE,0xD2,0xC1,0x75,0x79,0xAF,0x3C,0xDA,0x6D}}
                                +------------------------------------------------------------------------------------+
                                
                                +------------------------------------------------------------------------------------+
                                |                                  WHAT TO DO NEXT?                                  |
                                +------------------------------------------------------------------------------------+
                                | This device has now been personalized. Run this sketch with its current settings   |
                                | on all the devices in your network that have security enabled.                     |
                                +------------------------------------------------------------------------------------+
                                
                                +------------------------------------------------------------------------------------+
                                |                                  Execution result                                  |
                                +------------------------------------------------------------------------------------+
                                | SUCCESS                                                                            |
                                +------------------------------------------------------------------------------------+
                                

                                So, that's a success. A little confusingly, the output says 'Run this sketch with its current settings on all the devices in your network that have security enabled.' However, won't the device IDs / serial key be different on every device?

                                I then flashed the SecurityPersonalizer and node sketch on the other arduino. The output of the security personalizer is as follows, notably with a different serial key:

                                +------------------------------------------------------------------------------------+
                                |                           MySensors security personalizer                          |
                                +------------------------------------------------------------------------------------+
                                
                                +------------------------------------------------------------------------------------+
                                |                               Configuration settings                               |
                                +------------------------------------------------------------------------------------+
                                | * Guided personalization/storage of keys in EEPROM                                 |
                                | * Software based personalization (no ATSHA204A usage whatsoever)                   |
                                | * Will not require any UART confirmations                                          |
                                | * Will store HMAC key to EEPROM                                                    |
                                | * Will store AES key to EEPROM                                                     |
                                | * Will store soft serial to EEPROM                                                 |
                                +------------------------------------------------------------------------------------+
                                
                                +------------------------------------------------------------------------------------+
                                |                           Hardware security peripherals                            |
                                +--------------+--------------+--------------+------------------------------+--------+
                                | Device       | Status       | Revision     | Serial number                | Locked |
                                +--------------+--------------+--------------+------------------------------+--------+
                                | AVR          | DETECTED     | N/A          | N/A (generation required)    | N/A    |
                                +--------------+--------------+--------------+------------------------------+--------+
                                
                                +------------------------------------------------------------------------------------+
                                |                                    Key storage                                     |
                                +--------+--------+------------------------------------------------------------------+
                                | Key ID | Status | Key                                                              |
                                +--------+--------+------------------------------------------------------------------+
                                | HMAC   | OK     | 26E779A2876F7FC756C96B60085927EA24F7D8C57E117107BAE92C57080429E1 |
                                | AES    | OK     | 1E4FAF1129029D34C4832F6B48044C8B                                 |
                                | SERIAL | OK     | 31D46BF4ADBEF72803                                               |
                                +--------+--------+------------------------------------------------------------------+
                                
                                +------------------------------------------------------------------------------------+
                                |                                       EEPROM                                       |
                                +--------+--------+------------------------------------------------------------------+
                                | Key ID | Status | Key                                                              |
                                +--------+--------+------------------------------------------------------------------+
                                | HMAC   | OK     | 26E779A2876F7FC756C96B60085927EA24F7D8C57E117107BAE92C57080429E1 |
                                | AES    | OK     | 1E4FAF1129029D34C4832F6B48044C8B                                 |
                                | SERIAL | OK     | 31D46BF4ADBEF72803                                               |
                                +--------+--------+------------------------------------------------------------------+
                                
                                +------------------------------------------------------------------------------------+
                                |                      This nodes whitelist entry on other nodes                     |
                                +------------------------------------------------------------------------------------+
                                {.nodeId = <ID of this node>,.serial = {0x31,0xD4,0x6B,0xF4,0xAD,0xBE,0xF7,0x28,0x03}}
                                +------------------------------------------------------------------------------------+
                                
                                +------------------------------------------------------------------------------------+
                                |                                  WHAT TO DO NEXT?                                  |
                                +------------------------------------------------------------------------------------+
                                | This device has now been personalized. Run this sketch with its current settings   |
                                | on all the devices in your network that have security enabled.                     |
                                +------------------------------------------------------------------------------------+
                                
                                +------------------------------------------------------------------------------------+
                                |                                  Execution result                                  |
                                +------------------------------------------------------------------------------------+
                                | SUCCESS                                                                            |
                                +------------------------------------------------------------------------------------+
                                

                                Then, I watched the gateway serial output. It looks as follows:

                                NodeManager v1.5.1
                                INT1 M=255
                                INT2 M=255
                                0;255;3;0;14;Gateway startup complete.
                                0;255;0;0;18;2.2.0-beta
                                RADIO OK
                                0;255;3;0;11;Gateway
                                0;255;3;0;12;1.0
                                PRES I=200, T=23
                                0;200;0;0;23;
                                READY
                                
                                MY I=0 M=255
                                SEND D=0 I=200 C=0 T=48 S=STARTED I=0 F=0.00
                                0;200;1;0;48;STARTED
                                // stable till here - then I power up my node...
                                255;118;3;0;3;
                                RECV S=255 I=118 C=3 T=3 P=
                                255;x00
                                

                                ...and it just dies. Sometimes it prints non-printable characters to the serial output, sometimes it goes into a reset loop, and sometimes the only way to reset it is to cycle power to it.

                                However, it does appear to receive the message from the node before it dies, judging by the 255;118;3;0;3; message, which is an ID request. It looks like the gateway dies when trying to handle that incoming message maybe? If I don't power up the node, the gateway seems to appear stable, but once I power up a node, the gateway receives the ID request, and it dies.

                                Also, I don't suspect this is a hardware issue. I've used the same hardware in past experiments with MySensors (without signing and NodeManager at the time), and it has worked fine. Also, I tried all this without personalisation earlier today, and it has worked fine. I've added 47uF electrolytic caps to both radios, so power shouldn't be an issue.

                                So, the erratic behaviour of the gateway when receiving a message seems to be suggesting that we're hitting memory limits, maybe?

                                AnticimexA Offline
                                AnticimexA Offline
                                Anticimex
                                Contest Winner
                                wrote on last edited by
                                #52

                                @rakeshpai yes, that is a plausible explanation. The confusing message is probably because you have opted to manually set your serial instead of having it generated. Then you have to adjust your manual setting if you want unique serials on all devices.

                                Do you feel secure today? No? Start requiring some signatures and feel better tomorrow ;)

                                1 Reply Last reply
                                0
                                • rakeshpaiR rakeshpai

                                  @Anticimex From what it looks like at the moment, it looks like I'm hitting up against the memory issue. (But I might be wrong.)

                                  I don't know why the two nodes weren't communicating in my previous post. However, I decided to start from scratch, and ran the ClearEEPROM sketch on both my arduinos. Then, I disabled personalisation on both the gateway and the node sketches, flashed them to the boards, and watched ID requests come on the gateway's serial output, from the node. (I don't have a controller running.) So, everything looks good so far.

                                  Then I ran the SecurityPersonalizer on the gateway, and flashed the gateway sketch to it again. This was the output of the personalizer:

                                  +------------------------------------------------------------------------------------+
                                  |                           MySensors security personalizer                          |
                                  +------------------------------------------------------------------------------------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                               Configuration settings                               |
                                  +------------------------------------------------------------------------------------+
                                  | * Guided personalization/storage of keys in EEPROM                                 |
                                  | * Software based personalization (no ATSHA204A usage whatsoever)                   |
                                  | * Will not require any UART confirmations                                          |
                                  | * Will store HMAC key to EEPROM                                                    |
                                  | * Will store AES key to EEPROM                                                     |
                                  | * Will store soft serial to EEPROM                                                 |
                                  +------------------------------------------------------------------------------------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                           Hardware security peripherals                            |
                                  +--------------+--------------+--------------+------------------------------+--------+
                                  | Device       | Status       | Revision     | Serial number                | Locked |
                                  +--------------+--------------+--------------+------------------------------+--------+
                                  | AVR          | DETECTED     | N/A          | N/A (generation required)    | N/A    |
                                  +--------------+--------------+--------------+------------------------------+--------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                                    Key storage                                     |
                                  +--------+--------+------------------------------------------------------------------+
                                  | Key ID | Status | Key                                                              |
                                  +--------+--------+------------------------------------------------------------------+
                                  | HMAC   | OK     | 26E779A2876F7FC756C96B60085927EA24F7D8C57E117107BAE92C57080429E1 |
                                  | AES    | OK     | 1E4FAF1129029D34C4832F6B48044C8B                                 |
                                  | SERIAL | OK     | BED2C17579AF3CDA6D                                               |
                                  +--------+--------+------------------------------------------------------------------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                                       EEPROM                                       |
                                  +--------+--------+------------------------------------------------------------------+
                                  | Key ID | Status | Key                                                              |
                                  +--------+--------+------------------------------------------------------------------+
                                  | HMAC   | OK     | 26E779A2876F7FC756C96B60085927EA24F7D8C57E117107BAE92C57080429E1 |
                                  | AES    | OK     | 1E4FAF1129029D34C4832F6B48044C8B                                 |
                                  | SERIAL | OK     | BED2C17579AF3CDA6D                                               |
                                  +--------+--------+------------------------------------------------------------------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                      This nodes whitelist entry on other nodes                     |
                                  +------------------------------------------------------------------------------------+
                                  {.nodeId = <ID of this node>,.serial = {0xBE,0xD2,0xC1,0x75,0x79,0xAF,0x3C,0xDA,0x6D}}
                                  +------------------------------------------------------------------------------------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                                  WHAT TO DO NEXT?                                  |
                                  +------------------------------------------------------------------------------------+
                                  | This device has now been personalized. Run this sketch with its current settings   |
                                  | on all the devices in your network that have security enabled.                     |
                                  +------------------------------------------------------------------------------------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                                  Execution result                                  |
                                  +------------------------------------------------------------------------------------+
                                  | SUCCESS                                                                            |
                                  +------------------------------------------------------------------------------------+
                                  

                                  So, that's a success. A little confusingly, the output says 'Run this sketch with its current settings on all the devices in your network that have security enabled.' However, won't the device IDs / serial key be different on every device?

                                  I then flashed the SecurityPersonalizer and node sketch on the other arduino. The output of the security personalizer is as follows, notably with a different serial key:

                                  +------------------------------------------------------------------------------------+
                                  |                           MySensors security personalizer                          |
                                  +------------------------------------------------------------------------------------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                               Configuration settings                               |
                                  +------------------------------------------------------------------------------------+
                                  | * Guided personalization/storage of keys in EEPROM                                 |
                                  | * Software based personalization (no ATSHA204A usage whatsoever)                   |
                                  | * Will not require any UART confirmations                                          |
                                  | * Will store HMAC key to EEPROM                                                    |
                                  | * Will store AES key to EEPROM                                                     |
                                  | * Will store soft serial to EEPROM                                                 |
                                  +------------------------------------------------------------------------------------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                           Hardware security peripherals                            |
                                  +--------------+--------------+--------------+------------------------------+--------+
                                  | Device       | Status       | Revision     | Serial number                | Locked |
                                  +--------------+--------------+--------------+------------------------------+--------+
                                  | AVR          | DETECTED     | N/A          | N/A (generation required)    | N/A    |
                                  +--------------+--------------+--------------+------------------------------+--------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                                    Key storage                                     |
                                  +--------+--------+------------------------------------------------------------------+
                                  | Key ID | Status | Key                                                              |
                                  +--------+--------+------------------------------------------------------------------+
                                  | HMAC   | OK     | 26E779A2876F7FC756C96B60085927EA24F7D8C57E117107BAE92C57080429E1 |
                                  | AES    | OK     | 1E4FAF1129029D34C4832F6B48044C8B                                 |
                                  | SERIAL | OK     | 31D46BF4ADBEF72803                                               |
                                  +--------+--------+------------------------------------------------------------------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                                       EEPROM                                       |
                                  +--------+--------+------------------------------------------------------------------+
                                  | Key ID | Status | Key                                                              |
                                  +--------+--------+------------------------------------------------------------------+
                                  | HMAC   | OK     | 26E779A2876F7FC756C96B60085927EA24F7D8C57E117107BAE92C57080429E1 |
                                  | AES    | OK     | 1E4FAF1129029D34C4832F6B48044C8B                                 |
                                  | SERIAL | OK     | 31D46BF4ADBEF72803                                               |
                                  +--------+--------+------------------------------------------------------------------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                      This nodes whitelist entry on other nodes                     |
                                  +------------------------------------------------------------------------------------+
                                  {.nodeId = <ID of this node>,.serial = {0x31,0xD4,0x6B,0xF4,0xAD,0xBE,0xF7,0x28,0x03}}
                                  +------------------------------------------------------------------------------------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                                  WHAT TO DO NEXT?                                  |
                                  +------------------------------------------------------------------------------------+
                                  | This device has now been personalized. Run this sketch with its current settings   |
                                  | on all the devices in your network that have security enabled.                     |
                                  +------------------------------------------------------------------------------------+
                                  
                                  +------------------------------------------------------------------------------------+
                                  |                                  Execution result                                  |
                                  +------------------------------------------------------------------------------------+
                                  | SUCCESS                                                                            |
                                  +------------------------------------------------------------------------------------+
                                  

                                  Then, I watched the gateway serial output. It looks as follows:

                                  NodeManager v1.5.1
                                  INT1 M=255
                                  INT2 M=255
                                  0;255;3;0;14;Gateway startup complete.
                                  0;255;0;0;18;2.2.0-beta
                                  RADIO OK
                                  0;255;3;0;11;Gateway
                                  0;255;3;0;12;1.0
                                  PRES I=200, T=23
                                  0;200;0;0;23;
                                  READY
                                  
                                  MY I=0 M=255
                                  SEND D=0 I=200 C=0 T=48 S=STARTED I=0 F=0.00
                                  0;200;1;0;48;STARTED
                                  // stable till here - then I power up my node...
                                  255;118;3;0;3;
                                  RECV S=255 I=118 C=3 T=3 P=
                                  255;x00
                                  

                                  ...and it just dies. Sometimes it prints non-printable characters to the serial output, sometimes it goes into a reset loop, and sometimes the only way to reset it is to cycle power to it.

                                  However, it does appear to receive the message from the node before it dies, judging by the 255;118;3;0;3; message, which is an ID request. It looks like the gateway dies when trying to handle that incoming message maybe? If I don't power up the node, the gateway seems to appear stable, but once I power up a node, the gateway receives the ID request, and it dies.

                                  Also, I don't suspect this is a hardware issue. I've used the same hardware in past experiments with MySensors (without signing and NodeManager at the time), and it has worked fine. Also, I tried all this without personalisation earlier today, and it has worked fine. I've added 47uF electrolytic caps to both radios, so power shouldn't be an issue.

                                  So, the erratic behaviour of the gateway when receiving a message seems to be suggesting that we're hitting memory limits, maybe?

                                  U Offline
                                  U Offline
                                  user2684
                                  Contest Winner
                                  wrote on last edited by
                                  #53

                                  @rakeshpai in your first test, I notice the gateway logs do not have the "RADIO OK" message which should show up once the radio is initialized so probably the radio was faulty or not connected correctly I may guess (a #define MY_DEBUG should tell you the truth if it will happen again).
                                  In your second test, I'd definitely say it has something to do with the memory utilization. I wonder if you can keep signing on but turn encryption off so to save a good amount of memory and see if the tests will run just fine.

                                  rakeshpaiR 1 Reply Last reply
                                  0
                                  • U user2684

                                    @rakeshpai in your first test, I notice the gateway logs do not have the "RADIO OK" message which should show up once the radio is initialized so probably the radio was faulty or not connected correctly I may guess (a #define MY_DEBUG should tell you the truth if it will happen again).
                                    In your second test, I'd definitely say it has something to do with the memory utilization. I wonder if you can keep signing on but turn encryption off so to save a good amount of memory and see if the tests will run just fine.

                                    rakeshpaiR Offline
                                    rakeshpaiR Offline
                                    rakeshpai
                                    wrote on last edited by rakeshpai
                                    #54

                                    @user2684 Sorry for the delay. I tested the whole thing as above with signing and without encryption. During compile, it did appear to reduce memory usage by about 20%. However, it didn't change the behaviour of the gateway. It still hangs after receiving the first message.

                                    Good catch about the RADIO OK message. So, I tried removing signing and encryption, just to eliminate variables, and I still didn't see the RADIO OK message. Also, I was still seeing the gateway crash even without signing and encryption. I could have sworn this used to work fine. Now I don't even know what to believe of my previous tests anymore. Did I botch them up, or mis-remember them? Is this real life? :P

                                    I guess I'll build another Arduino Pro Mini + NRF on a perfboard and try again. I'll update once I have something. Meanwhile, and I know this is a big ask, but if you could please test on hardware you have laying around that you know works well, it'll really help to know if this is somehow just my setup acting up.

                                    EDIT: I flashed the node sketch on the board I was normally using for the gateway, and I still don't see a RADIO OK message. This either means that both my boards are broken (possible but unlikely), or that the RADIO OK message isn't printed in the serial output by the sketch at all. I haven't looked at the code yet, so I'm not sure which it is. Also, worth remembering that even though the node doesn't print a RADIO OK message, the gateway definitely reacts to the node being powered on (by crashing), so there's definitely some radio communication going on.

                                    rakeshpaiR 1 Reply Last reply
                                    0
                                    • rakeshpaiR rakeshpai

                                      @user2684 Sorry for the delay. I tested the whole thing as above with signing and without encryption. During compile, it did appear to reduce memory usage by about 20%. However, it didn't change the behaviour of the gateway. It still hangs after receiving the first message.

                                      Good catch about the RADIO OK message. So, I tried removing signing and encryption, just to eliminate variables, and I still didn't see the RADIO OK message. Also, I was still seeing the gateway crash even without signing and encryption. I could have sworn this used to work fine. Now I don't even know what to believe of my previous tests anymore. Did I botch them up, or mis-remember them? Is this real life? :P

                                      I guess I'll build another Arduino Pro Mini + NRF on a perfboard and try again. I'll update once I have something. Meanwhile, and I know this is a big ask, but if you could please test on hardware you have laying around that you know works well, it'll really help to know if this is somehow just my setup acting up.

                                      EDIT: I flashed the node sketch on the board I was normally using for the gateway, and I still don't see a RADIO OK message. This either means that both my boards are broken (possible but unlikely), or that the RADIO OK message isn't printed in the serial output by the sketch at all. I haven't looked at the code yet, so I'm not sure which it is. Also, worth remembering that even though the node doesn't print a RADIO OK message, the gateway definitely reacts to the node being powered on (by crashing), so there's definitely some radio communication going on.

                                      rakeshpaiR Offline
                                      rakeshpaiR Offline
                                      rakeshpai
                                      wrote on last edited by rakeshpai
                                      #55

                                      I'm glad to report that I'm not completely losing my mind. :P

                                      The difference between the recent tests, and my initial tests (where I saw everything working), was that in my recent tests, I was using NodeManager 1.5.1 from your fork.

                                      I reverted to using the NodeManager from the MySensors repo's master branch, disabled signing and encryption, and everything worked. To push the boundaries slightly, I enabled signing, but kept encryption disabled, and flashed this on the gateway and on the node. Compiler messages:

                                      Sketch uses 22,772 bytes (74%) of program storage space. Maximum is 30,720 bytes.
                                      Global variables use 1,871 bytes (91%) of dynamic memory, leaving 177 bytes for local variables. Maximum is 2,048 bytes.
                                      Low memory available, stability problems may occur.
                                      

                                      So, not enough leg room memory-wise, but I tried running it anyway:

                                      NodeManager v1.5
                                      INT1 M=255
                                      INT2 M=255
                                      0;255;3;0;14;Gateway startup complete.
                                      0;255;0;0;18;2.2.0-beta
                                      RADIO OK
                                      0;255;3;0;11;Gateway
                                      0;255;3;0;12;1.0
                                      PRES I=200, T=23
                                      0;200;0;0;23;
                                      READY
                                      
                                      MY I=0 M=255
                                      SEND D=0 I=200 C=0 T=48 S=STARTED I=0 F=0.00
                                      0;200;1;0;48;STARTED
                                      255;40;3;0;3;
                                      RECV S=255 I=40 C=3 T=3 P=
                                      255;249;3;0;3;
                                      RECV S=255 I=249 C=3 T=3 P=
                                      255;202;3;0;3;
                                      RECV S=255 I=202 C=3 T=3 P=
                                      255;155;3;0;3;
                                      RECV S=255 I=155 C=3 T=3 P=
                                      255;86;3;0;3;
                                      RECV S=255 I=86 C=3 T=3 P=
                                      255;40;3;0;3;
                                      RECV S=255 I=40 C=3 T=3 P=
                                      255;249;3;0;3;
                                      RECV S=255 I=249 C=3 T=3 P=
                                      255;202;3;0;3;
                                      RECV S=255 I=202 C=3 T=3 P=
                                      

                                      I don't have any controller running, so nothing's handling the serial data from the gateway. However, you can see the last few lines in the paste above, where the gateway receives ID requests from the node just fine, and without a crash.

                                      Considering that the only difference between my last post and this one is the version of NodeManager I've been using, I think it's safe to conclude that the crashes I was facing was due to 1.5.1. Not sure what the fix is, but at least we've identified the issue, somewhat at least.

                                      That said, 1.5.1 was definitely the correct direction in terms of memory usage, since I could fit everything in with it. (With 1.5, I have to disable encryption to get it to fit.) So, if it is possible to identify and fix the stability issue with 1.5.1, and still be able to fit within memory limits, that'd be ideal.

                                      EDIT: I still don't see the RADIO_OK message, or anything else really, in the node's serial output, though the gateway behaves like it's receiving messages just fine. I suspect the node just isn't printing that string to the serial output.

                                      U 2 Replies Last reply
                                      0
                                      • rakeshpaiR rakeshpai

                                        I'm glad to report that I'm not completely losing my mind. :P

                                        The difference between the recent tests, and my initial tests (where I saw everything working), was that in my recent tests, I was using NodeManager 1.5.1 from your fork.

                                        I reverted to using the NodeManager from the MySensors repo's master branch, disabled signing and encryption, and everything worked. To push the boundaries slightly, I enabled signing, but kept encryption disabled, and flashed this on the gateway and on the node. Compiler messages:

                                        Sketch uses 22,772 bytes (74%) of program storage space. Maximum is 30,720 bytes.
                                        Global variables use 1,871 bytes (91%) of dynamic memory, leaving 177 bytes for local variables. Maximum is 2,048 bytes.
                                        Low memory available, stability problems may occur.
                                        

                                        So, not enough leg room memory-wise, but I tried running it anyway:

                                        NodeManager v1.5
                                        INT1 M=255
                                        INT2 M=255
                                        0;255;3;0;14;Gateway startup complete.
                                        0;255;0;0;18;2.2.0-beta
                                        RADIO OK
                                        0;255;3;0;11;Gateway
                                        0;255;3;0;12;1.0
                                        PRES I=200, T=23
                                        0;200;0;0;23;
                                        READY
                                        
                                        MY I=0 M=255
                                        SEND D=0 I=200 C=0 T=48 S=STARTED I=0 F=0.00
                                        0;200;1;0;48;STARTED
                                        255;40;3;0;3;
                                        RECV S=255 I=40 C=3 T=3 P=
                                        255;249;3;0;3;
                                        RECV S=255 I=249 C=3 T=3 P=
                                        255;202;3;0;3;
                                        RECV S=255 I=202 C=3 T=3 P=
                                        255;155;3;0;3;
                                        RECV S=255 I=155 C=3 T=3 P=
                                        255;86;3;0;3;
                                        RECV S=255 I=86 C=3 T=3 P=
                                        255;40;3;0;3;
                                        RECV S=255 I=40 C=3 T=3 P=
                                        255;249;3;0;3;
                                        RECV S=255 I=249 C=3 T=3 P=
                                        255;202;3;0;3;
                                        RECV S=255 I=202 C=3 T=3 P=
                                        

                                        I don't have any controller running, so nothing's handling the serial data from the gateway. However, you can see the last few lines in the paste above, where the gateway receives ID requests from the node just fine, and without a crash.

                                        Considering that the only difference between my last post and this one is the version of NodeManager I've been using, I think it's safe to conclude that the crashes I was facing was due to 1.5.1. Not sure what the fix is, but at least we've identified the issue, somewhat at least.

                                        That said, 1.5.1 was definitely the correct direction in terms of memory usage, since I could fit everything in with it. (With 1.5, I have to disable encryption to get it to fit.) So, if it is possible to identify and fix the stability issue with 1.5.1, and still be able to fit within memory limits, that'd be ideal.

                                        EDIT: I still don't see the RADIO_OK message, or anything else really, in the node's serial output, though the gateway behaves like it's receiving messages just fine. I suspect the node just isn't printing that string to the serial output.

                                        U Offline
                                        U Offline
                                        user2684
                                        Contest Winner
                                        wrote on last edited by
                                        #56

                                        @rakeshpai thanks for reporting back, sounds like a serious problem with 1.5.1, I'm glad I've not released it yet :) I will try to reproduce and of course fix the issue and I'll see if I'll get the same with a serial gw as well during the weekend. Sorry for making you wait for so long :-(

                                        1 Reply Last reply
                                        0
                                        • rakeshpaiR rakeshpai

                                          I'm glad to report that I'm not completely losing my mind. :P

                                          The difference between the recent tests, and my initial tests (where I saw everything working), was that in my recent tests, I was using NodeManager 1.5.1 from your fork.

                                          I reverted to using the NodeManager from the MySensors repo's master branch, disabled signing and encryption, and everything worked. To push the boundaries slightly, I enabled signing, but kept encryption disabled, and flashed this on the gateway and on the node. Compiler messages:

                                          Sketch uses 22,772 bytes (74%) of program storage space. Maximum is 30,720 bytes.
                                          Global variables use 1,871 bytes (91%) of dynamic memory, leaving 177 bytes for local variables. Maximum is 2,048 bytes.
                                          Low memory available, stability problems may occur.
                                          

                                          So, not enough leg room memory-wise, but I tried running it anyway:

                                          NodeManager v1.5
                                          INT1 M=255
                                          INT2 M=255
                                          0;255;3;0;14;Gateway startup complete.
                                          0;255;0;0;18;2.2.0-beta
                                          RADIO OK
                                          0;255;3;0;11;Gateway
                                          0;255;3;0;12;1.0
                                          PRES I=200, T=23
                                          0;200;0;0;23;
                                          READY
                                          
                                          MY I=0 M=255
                                          SEND D=0 I=200 C=0 T=48 S=STARTED I=0 F=0.00
                                          0;200;1;0;48;STARTED
                                          255;40;3;0;3;
                                          RECV S=255 I=40 C=3 T=3 P=
                                          255;249;3;0;3;
                                          RECV S=255 I=249 C=3 T=3 P=
                                          255;202;3;0;3;
                                          RECV S=255 I=202 C=3 T=3 P=
                                          255;155;3;0;3;
                                          RECV S=255 I=155 C=3 T=3 P=
                                          255;86;3;0;3;
                                          RECV S=255 I=86 C=3 T=3 P=
                                          255;40;3;0;3;
                                          RECV S=255 I=40 C=3 T=3 P=
                                          255;249;3;0;3;
                                          RECV S=255 I=249 C=3 T=3 P=
                                          255;202;3;0;3;
                                          RECV S=255 I=202 C=3 T=3 P=
                                          

                                          I don't have any controller running, so nothing's handling the serial data from the gateway. However, you can see the last few lines in the paste above, where the gateway receives ID requests from the node just fine, and without a crash.

                                          Considering that the only difference between my last post and this one is the version of NodeManager I've been using, I think it's safe to conclude that the crashes I was facing was due to 1.5.1. Not sure what the fix is, but at least we've identified the issue, somewhat at least.

                                          That said, 1.5.1 was definitely the correct direction in terms of memory usage, since I could fit everything in with it. (With 1.5, I have to disable encryption to get it to fit.) So, if it is possible to identify and fix the stability issue with 1.5.1, and still be able to fit within memory limits, that'd be ideal.

                                          EDIT: I still don't see the RADIO_OK message, or anything else really, in the node's serial output, though the gateway behaves like it's receiving messages just fine. I suspect the node just isn't printing that string to the serial output.

                                          U Offline
                                          U Offline
                                          user2684
                                          Contest Winner
                                          wrote on last edited by
                                          #57

                                          @rakeshpai unfortunately I'm struggling in reproducing the issue :-(
                                          I open up 1.5.1 and just changed the radio settings and defined MY_GATEWAY_SERIAL without touching anything else. This is my config.h file:

                                          #ifndef config_h
                                          #define config_h
                                          
                                          /**********************************
                                           * Sketch configuration
                                           */
                                          
                                          #define SKETCH_NAME "NodeManager"
                                          #define SKETCH_VERSION "1.0"
                                          
                                          /**********************************
                                           * MySensors node configuration
                                           */
                                          
                                          // General settings
                                          #define MY_BAUD_RATE 9600
                                          //#define MY_DEBUG
                                          //#define MY_NODE_ID 100
                                          
                                          // NRF24 radio settings
                                          //#define MY_RADIO_NRF24
                                          //#define MY_RF24_ENABLE_ENCRYPTION
                                          //#define MY_RF24_CHANNEL 76
                                          //#define MY_RF24_PA_LEVEL RF24_PA_HIGH
                                          //#define MY_DEBUG_VERBOSE_RF24
                                          
                                          // RFM69 radio settings
                                          #define MY_RADIO_RFM69
                                          #define MY_RFM69_FREQUENCY RFM69_868MHZ
                                          #define MY_IS_RFM69HW
                                          //#define MY_DEBUG_VERBOSE_RFM69
                                          #define MY_RFM69_NEW_DRIVER
                                          //#define MY_RFM69_ENABLE_ENCRYPTION
                                          #define MY_RFM69_NETWORKID 110
                                          //#define MY_RF69_IRQ_PIN D1
                                          //#define MY_RF69_IRQ_NUM MY_RF69_IRQ_PIN
                                          //#define MY_RF69_SPI_CS D2
                                          
                                          /**********************************
                                           * MySensors gateway configuration
                                           */
                                          // Common gateway settings
                                          //#define MY_REPEATER_FEATURE
                                          
                                          // Serial gateway settings
                                          #define MY_GATEWAY_SERIAL
                                          
                                          // Ethernet gateway settings
                                          //#define MY_GATEWAY_W5100
                                          
                                          // ESP8266 gateway settings
                                          //#define MY_GATEWAY_ESP8266
                                          //#define MY_ESP8266_SSID ""
                                          //#define MY_ESP8266_PASSWORD ""
                                          
                                          // Gateway networking settings
                                          //#define MY_IP_ADDRESS 192,168,178,87
                                          //#define MY_IP_GATEWAY_ADDRESS 192,168,178,1
                                          //#define MY_IP_SUBNET_ADDRESS 255,255,255,0
                                          //#define MY_PORT 5003
                                          //#define MY_GATEWAY_MAX_CLIENTS 2
                                          //#define MY_USE_UDP
                                          
                                          // Gateway MQTT settings
                                          //#define MY_GATEWAY_MQTT_CLIENT
                                          //#define MY_CONTROLLER_IP_ADDRESS 192, 168, 178, 68
                                          //#define MY_PORT 1883
                                          //#define MY_MQTT_USER "username"
                                          //#define MY_MQTT_PASSWORD "password"
                                          //#define MY_MQTT_CLIENT_ID "mysensors-1"
                                          //#define MY_MQTT_PUBLISH_TOPIC_PREFIX "mygateway1-out"
                                          //#define MY_MQTT_SUBSCRIBE_TOPIC_PREFIX "mygateway1-in"
                                          
                                          // Gateway inclusion mode
                                          //#define MY_INCLUSION_MODE_FEATURE
                                          //#define MY_INCLUSION_BUTTON_FEATURE
                                          //#define MY_INCLUSION_MODE_DURATION 60
                                          //#define MY_DEFAULT_LED_BLINK_PERIOD 300
                                          
                                          // Gateway Leds settings
                                          //#define MY_DEFAULT_ERR_LED_PIN 4
                                          //#define MY_DEFAULT_RX_LED_PIN  5
                                          //#define MY_DEFAULT_TX_LED_PIN  6
                                          
                                          /***********************************
                                           * NodeManager configuration
                                           */
                                          
                                          // if enabled, enable debug messages on serial port
                                          #define DEBUG 1
                                          
                                          // if enabled, enable the capability to power on sensors with the arduino's pins to save battery while sleeping
                                          #define POWER_MANAGER 1
                                          // if enabled, will load the battery manager library to allow the battery level to be reported automatically or on demand
                                          #define BATTERY_MANAGER 1
                                          // if enabled, allow modifying the configuration remotely by interacting with the configuration child id
                                          #define REMOTE_CONFIGURATION 1
                                          // if enabled, persist the remote configuration settings on EEPROM
                                          #define PERSIST 0
                                          // if enabled, a battery sensor will be created at BATTERY_CHILD_ID and will report vcc voltage together with the battery level percentage
                                          #define BATTERY_SENSOR 1
                                          // if enabled, send a SLEEPING and AWAKE service messages just before entering and just after leaving a sleep cycle and STARTED when starting/rebooting
                                          #define SERVICE_MESSAGES 0
                                          
                                          // Enable this module to use one of the following sensors: SENSOR_ANALOG_INPUT, SENSOR_LDR, SENSOR_THERMISTOR, SENSOR_MQ, SENSOR_ML8511, SENSOR_ACS712, SENSOR_RAIN_GAUGE
                                          #define MODULE_ANALOG_INPUT 1
                                          // Enable this module to use one of the following sensors: SENSOR_DIGITAL_INPUT
                                          #define MODULE_DIGITAL_INPUT 1
                                          // Enable this module to use one of the following sensors: SENSOR_DIGITAL_OUTPUT, SENSOR_RELAY, SENSOR_LATCHING_RELAY
                                          #define MODULE_DIGITAL_OUTPUT 1
                                          // Enable this module to use one of the following sensors: SENSOR_DHT11, SENSOR_DHT22
                                          #define MODULE_DHT 0
                                          // Enable this module to use one of the following sensors: SENSOR_SHT21
                                          #define MODULE_SHT21 0
                                          // Enable this module to use one of the following sensors: SENSOR_SWITCH, SENSOR_DOOR, SENSOR_MOTION
                                          #define MODULE_SWITCH 0
                                          // Enable this module to use one of the following sensors: SENSOR_DS18B20
                                          #define MODULE_DS18B20 0
                                          // Enable this module to use one of the following sensors: SENSOR_BH1750
                                          #define MODULE_BH1750 0
                                          // Enable this module to use one of the following sensors: SENSOR_MLX90614
                                          #define MODULE_MLX90614 0
                                          // Enable this module to use one of the following sensors: SENSOR_BME280
                                          #define MODULE_BME280 0
                                          // Enable this module to use one of the following sensors: SENSOR_SONOFF
                                          #define MODULE_SONOFF 0
                                          // Enable this module to use one of the following sensors: SENSOR_BMP085
                                          #define MODULE_BMP085 0
                                          // Enable this module to use one of the following sensors: SENSOR_HCSR04
                                          #define MODULE_HCSR04 0
                                          // Enable this module to use one of the following sensors: SENSOR_MCP9808
                                          #define MODULE_MCP9808 0
                                          #endif
                                          

                                          This is what I get on my serial output:

                                          NodeManager v1.5.1
                                          INT1 M=255
                                          INT2 M=255
                                          0;255;3;0;14;Gateway startup complete.
                                          0;255;0;0;18;2.2.0-beta
                                          RADIO OK
                                          0;255;3;0;11;NodeManager
                                          0;255;3;0;12;1.0
                                          PRES I=200, T=23
                                          0;200;0;0;23;
                                          PRES I=201, T=30
                                          0;201;0;0;30;
                                          BATT V=3.24 P=91
                                          SEND D=0 I=201 C=0 T=38 S= I=0 F=3.24
                                          0;201;1;0;38;3.24
                                          0;255;3;0;0;91
                                          READY
                                          
                                          MY I=0 M=1
                                          

                                          Do you see any difference between my settings and yours?
                                          Thanks

                                          rakeshpaiR 1 Reply Last reply
                                          0
                                          Reply
                                          • Reply as topic
                                          Log in to reply
                                          • Oldest to Newest
                                          • Newest to Oldest
                                          • Most Votes


                                          8

                                          Online

                                          11.7k

                                          Users

                                          11.2k

                                          Topics

                                          113.0k

                                          Posts


                                          Copyright 2019 TBD   |   Forum Guidelines   |   Privacy Policy   |   Terms of Service
                                          • Login

                                          • Don't have an account? Register

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • MySensors
                                          • OpenHardware.io
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular