💬 NodeManager

user2684

Hello all, just to let you know when compiling against the latest version of the mysensors 2.2.0-beta library, NodeManager will crash on startup, just after presenting the mysensors logo. Thanks @gohan for pointing this out! Root cause is still unknown but when MY_DEBUG is defined, the crash doesn't take place (https://github.com/mysensors/NodeManager/issues/223)

vikasjee

@user2684, Can we have a quick hot fix?

user2684

@vikasjee sure, as soon as I'll be able to understand the reason why it crashes when MY_DEBUG is not defined :-( if somebody from the core team has any hit, would be really appreciated!
Thanks

Sergio Rius

I'm afraid that issue 223 is not a NodeManager failure. I can reproduce it in a new sketch without using NodeManager.
Other issue with MySensors are, NRF24 doesn't link with controller.
If MY_DEBUG_VERBOSE_RF24 is defined, it automagically works.

vikasjee

@Sergio-Rius, with RF24_VERBOSE on, the system crashes in 40-50 seconds especially if you're connected to the serial monitor or if your serial gateway has this #DEFINE on.

Sergio Rius

Still more strange, suddenly started to work properly on 2/3 of my testing nodes. That killed me.

What about the gateways you are using? And the controllers?

FredRoot

Hi,

I'm having trouble with the motion sample code. I have changed the setSleepHours(1) command to setSleepMinutes(1). I understand that the sensor will send a hart beat every 1 min. This is working fine for the hart beat, but evertime the hart beat is send the sensor sends a Trigger messages as well even though the sensor is not triggered. I thought that maybe there is a power dip and that it might cause the PIR (RS501) to send a interrupt, but I have removed the setSleepMinutes command and replaced it with a setBatteryReportHours(1) command. This command is send correctly and the sensor only sends the trigger command when the PIR is triggered. Did anyone have a similar issue?

Attached is the sensor debug output when the setSleepMinutes command is used:

0 MCO:BGN:INIT NODE,CP=RNNNA--,VER=2.1.1
40 MCO:BGN:BFR
REG I=1 P=3 P=1 T=16
NodeManager v1.6
LIB V=2.1.1 R=N T=N A=A S=- B=-
INT P=3 M=1
INT P=2 M=255
96 TSM:INIT
176 TSF:WUR:MS=0
198 TSM:INIT:TSP OK
219 TSM:INIT:STATID=3
243 TSF:SID:OK,ID=3
264 TSM:FPAR
313 TSF:MSG:SEND,3-3-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
385 TSF:MSG:READ,0-0-3,s=255,c=3,t=8,pt=1,l=1,sg=0:0
440 TSF:MSG:FPAR OK,ID=0,D=1
2387 TSM:FPAR:OK
2404 TSM:ID
2418 TSM:ID:OK
2433 TSM:UPL
2449 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=24,pt=1,l=1,sg=0,ft=0,st=OK:1
2521 TSF:MSG:READ,0-0-3,s=255,c=3,t=25,pt=1,l=1,sg=0:1
2578 TSF:MSG:PONG RECV,HP=1
2607 TSM:UPL:OK
2625 TSM:READY:ID=3,PAR=0,DIS=1
2660 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
2734 TSF:MSG:READ,0-0-3,s=255,c=3,t=15,pt=6,l=2,sg=0:0100
2797 TSF:MSG:SEND,3-3-0-0,s=255,c=0,t=17,pt=0,l=5,sg=0,ft=0,st=OK:2.1.1
2875 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=6,pt=1,l=1,sg=0,ft=0,st=OK:0
2945 TSF:MSG:READ,0-0-3,s=255,c=3,t=6,pt=0,l=1,sg=0:M
RADIO OK
3004 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=11,pt=0,l=11,sg=0,ft=0,st=OK:NodeManager
3096 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=12,pt=0,l=3,sg=0,ft=0,st=OK:1.0
PRES I=200, T=23
3170 TSF:MSG:SEND,3-3-0-0,s=200,c=0,t=23,pt=0,l=0,sg=0,ft=0,st=OK:
PRES I=201, T=30
3260 TSF:MSG:SEND,3-3-0-0,s=201,c=0,t=30,pt=0,l=0,sg=0,ft=0,st=OK:
BATT V=3.30 P=100
SEND D=0 I=201 C=0 T=38 S= I=0 F=3.30
3418 !MCO:SND:NODE NOT REG
3506 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=0,pt=1,l=1,sg=0,ft=0,st=OK:100
PRES I=1 T=1
3579 TSF:MSG:SEND,3-3-0-0,s=1,c=0,t=1,pt=0,l=0,sg=0,ft=0,st=OK:
READY

3659 MCO:REG:REQ
3688 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=26,pt=1,l=1,sg=0,ft=0,st=OK:2
3760 TSF:MSG:READ,0-0-3,s=255,c=3,t=27,pt=1,l=1,sg=0:1
3817 MCO:PIM:NODE REG=1
3842 MCO:BGN:STP
MY I=3 M=1
3860 MCO:BGN:INIT OK,TSP=1
SLEEP 60s

3901 MCO:SLP:MS=60000,SMS=1,I1=1,M1=1,I2=255,M2=255
3971 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=22,pt=5,l=4,sg=0,ft=0,st=OK:127
4544 MCO:SLP:TPD
4560 MCO:SLP:WUP=1
INT P=3, M=1
AWAKE
SWITCH I=1 P=3 V=1
SEND D=0 I=1 C=0 T=16 S= I=1 F=0.00
4597 TSF:MSG:SEND,3-3-0-0,s=1,c=1,t=16,pt=2,l=2,sg=0,ft=0,st=OK:1
SLEEP 60s

4732 MCO:SLP:MS=60000,SMS=1,I1=1,M1=1,I2=255,M2=255
4800 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=22,pt=5,l=4,sg=0,ft=0,st=OK:956
5376 MCO:SLP:TPD
5392 MCO:SLP:WUP=1
INT P=3, M=1
AWAKE
SWITCH I=1 P=3 V=0
SEND D=0 I=1 C=1 T=16 S= I=0 F=0.00
5429 TSF:MSG:SEND,3-3-0-0,s=1,c=1,t=16,pt=2,l=2,sg=0,ft=0,st=OK:0
SLEEP 60s

5564 MCO:SLP:MS=60000,SMS=1,I1=1,M1=1,I2=255,M2=255
5632 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=22,pt=5,l=4,sg=0,ft=0,st=OK:1787
6207 MCO:SLP:TPD
6223 MCO:SLP:WUP=-1
AWAKE
SLEEP 60s

6246 MCO:SLP:MS=60000,SMS=1,I1=1,M1=1,I2=255,M2=255
6322 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=22,pt=5,l=4,sg=0,ft=0,st=OK:2478
6897 MCO:SLP:TPD
6914 MCO:SLP:WUP=1
INT P=3, M=1
AWAKE
SWITCH I=1 P=3 V=1
SEND D=0 I=1 C=1 T=16 S= I=1 F=0.00
6950 TSF:MSG:SEND,3-3-0-0,s=1,c=1,t=16,pt=2,l=2,sg=0,ft=0,st=OK:1
SLEEP 60s

7086 MCO:SLP:MS=60000,SMS=1,I1=1,M1=1,I2=255,M2=255
7153 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=22,pt=5,l=4,sg=0,ft=0,st=OK:3309
7729 MCO:SLP:TPD
7745 MCO:SLP:WUP=1
INT P=3, M=1
AWAKE
SWITCH I=1 P=3 V=0
SEND D=0 I=1 C=1 T=16 S= I=0 F=0.00
7782 TSF:MSG:SEND,3-3-0-0,s=1,c=1,t=16,pt=2,l=2,sg=0,ft=0,st=OK:0
SLEEP 60s

7917 MCO:SLP:MS=60000,SMS=1,I1=1,M1=1,I2=255,M2=255
7985 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=22,pt=5,l=4,sg=0,ft=0,st=OK:4141
8560 MCO:SLP:TPD
8577 MCO:SLP:WUP=-1
AWAKE
SLEEP 60s

8599 MCO:SLP:MS=60000,SMS=1,I1=1,M1=1,I2=255,M2=255
8675 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=22,pt=5,l=4,sg=0,ft=0,st=OK:4831
9252 MCO:SLP:TPD
9269 MCO:SLP:WUP=1
INT P=3, M=1
AWAKE
SWITCH I=1 P=3 V=1
SEND D=0 I=1 C=1 T=16 S= I=1 F=0.00
9306 TSF:MSG:SEND,3-3-0-0,s=1,c=1,t=16,pt=2,l=2,sg=0,ft=0,st=OK:1
SLEEP 60s

9441 MCO:SLP:MS=60000,SMS=1,I1=1,M1=1,I2=255,M2=255
9508 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=22,pt=5,l=4,sg=0,ft=0,st=OK:5664
10084 MCO:SLP:TPD
10102 MCO:SLP:WUP=1
INT P=3, M=1
AWAKE
SWITCH I=1 P=3 V=0
SEND D=0 I=1 C=1 T=16 S= I=0 F=0.00
10139 TSF:MSG:SEND,3-3-0-0,s=1,c=1,t=16,pt=2,l=2,sg=0,ft=0,st=OK:0
SLEEP 60s

10274 MCO:SLP:MS=60000,SMS=1,I1=1,M1=1,I2=255,M2=255
10344 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=22,pt=5,l=4,sg=0,ft=0,st=OK:6500
10921 MCO:SLP:TPD

user2684

@FredRoot this is weird, from the logs looks like there is kind of a floating interrupt (alternatively 1 and 0). The strange thing is that the interrupt is coming from the call to mysensors' sleep() function which is returning a positive number corresponding to the interrupt linked to pin 3 instead of a negative value like it should after a sleeping cycle. Anybody else experiencing this behavior with NodeManager v1.6? thanks

mvader

@user2684 can you add support for the Si7021 sensor as found in sensebender micro?
https://www.mysensors.org/hardware/micro

also, 2nd question.
how do i register a sensor that is onboard?
you ask for a child id and pin that it's on. but if it's an onboard sensor. how do you make that work? just omit the pin?

user2684

@mvader for question #1, I've added it to the existing issue https://github.com/mysensors/NodeManager/issues/221. For question #2, for sensors using i2c the pin is not required despite within the code nodemanager will set a random pin (have a look at SensorSHT21 for example)

mvader

@user2684 said in 💬 NodeManager:

@mvader for question #1, I've added it to the existing issue https://github.com/mysensors/NodeManager/issues/221. For question #2, for sensors using i2c the pin is not required despite within the code nodemanager will set a random pin (have a look at SensorSHT21 for example)

great.. thank you!

user2684

After a few months working behind the scene, NodeManager got kind of a full review in terms of architecture and usability. The code should be now simpler to understand for new users, the memory utilization has been optimized and the overall user experience has been improved.

The looooong list of things that have been changed are detailed on https://github.com/mysensors/NodeManager/pull/229.
Please note this is only available in the development branch (https://github.com/mysensors/NodeManager/tree/development) and will be generally available only when NodeManager's v1.7 will be released (it will take I guess still a few months).

Since this change was blocking other pending changes, I hope now the development will be faster and feel free to submit your PRs now that the new architecture has been finalized.
As always, any feedback is more than welcome.
Thanks

reinhold

@user2684 Thanks a lot for the architecture change, which makes a lot of sense conceptually.

Unfortunately, it appears to use more memory than before. I'm working on an air quality board with eight MQ sensors, an MH-Z19 CO2 sensor and a Plantower PMSA003 particulate matter sensor. In previous development versions (1.7-dev, before today), the eight MQ sensors and the MH-Z19 worked fine, but now after your big merge of #229 the sketch appears to run out of memory after adding the child of the sixth MQ sensor. Do you see any chance to optimize for memory?

Old sketch (using a previous development version of NodeManager, without the PMSA003; 155 bytes left after all nine sensors are fully initialized): https://github.com/open-tools/NodeManager_GasSensor/tree/master/NodeManager_GasSensor

New sketch (using the latest version of NodeManager, plus a PMSA003): https://github.com/kainhofer/NodeManager/tree/GasSensor
That sketch runs out of memory with the sixth MQ sensor (I added debug output with free memory at a lot of spots). Of course, I have commented out the PMSA003 completely to have a correct comparison.

FWIW, the sketch is for this board: https://github.com/open-tools/NodeManager_GasSensor/blob/master/images/IMAG2267.jpg

Nca78

@reinhold if you have high memory requirement, what about switching to a more modern architecture like stm32f103c8t6 on the famous "blue pill" boards ? It has 20k or RAM (10 times what you have on Arduino) and 64k of flash and it's supported by MySensors, in addition to be very cheap (2$/board). You will need voltage converter between 3.3V of the board and 5V of your sensors but that's not a big deal in exchange of the possibility to expand your board with more sensors in the future.

mfalkvidd

@reinhold not sure how much this applies to NodeManager, but there are some recomendations for reducing memory footprint at https://www.mysensors.org/apidocs-beta/group__memorysavings.html

reinhold

@nca78 Thanks for the suggestion to use the blue pill STM32 boards. I haven't thought of it (I only looked at the ESP8266, but that has only one ADC). I have a few lying around here, but never got around to trying them, as I first need to burn the bootloader. It looks quite promising, but the boards are huge, so I'll need to enlarge my PCB...
The need for a 5V->3.3V regulator is no issue, as the NRF24 already needs 3.3V. The 5V analog level of the MQ sensors will need some converter (the analog ADC inputs are marked as non-5V-tolerant...), but then in turn I can get rid of the voltage dividers for the PMSA003. The larger issue seems to be the connection of the nrf24, which takes away 4 ADC pins, so there are only 6 ADC pins left, while I have 8 analog sensors :(

Still, for now I have the PCBs with the ProMini, so I'll try to strip down the Sensor or SensorMQ classes for my own use.

reinhold

@nca78 Two more things popped up with the latest development version of NodeManager:

In NodeManager.ino: function void before(): The sample code to set the child id of a child is sht.children.get(1)->child_id = 5, but that does not work, because at that point the children are not yet even generated. They are created by the node.before(); call at the very end of the function... So, currently this will not assign any new child ID and the default IDs will be used. I have not checked in detail whether calling node.before() at the beginning of the before() function and then assigning child_id to the children will solve the issue. From a quick look at the code, it might work...
I'm trying to implement an OLED display, where the node would wake up every 5 seconds, take a measurement and display it on the OLED. But the report interval to the MySensors gateway should only be 10 minutes. I thought that
node.setSleepSeconds(5); node.setReportIntervalMinutes(10); would be the correct way to setup NodeManager to take measurements every 5 seconds, but only report it every 10 minutes to the network. Unfortunately, Sensor::loop(MyMessage* message) seems to immediately return without measurement if the reporting timer has not elapsed. Somehow I don't understand that logic. Why have different sleep and reporting intervals when measurements will only be taken at reporting intervals anyway? I would expect measurements to be taken with the sleep intervals (e.g. to calculate an average over the whole reporting interval) and report back to the network using report intervals.

user2684

@reinhold thanks for reporting this issue. One of the main objectives of the new architecture was to reduce the memory footprint so if this is not the case, there is definitely something to fix. I've noticed in your sketch you are using the remote configuration sensor, that one is huge in terms of memory, if not strictly required I'd comment it out (BTW very nice project!). In PR #229 I've also uploaded a spreadsheet with all tests I run for realizing what was using the memory the most. Anyway, there is still a lot of room for improvement looks like :)
Thanks @mfalkvidd , I'll give the directives you pointed out a try!

user2684

@reinhold you're right, assigning the child_id before calling node.before() does not work. The way to go would be to have all the methods being called before node.before() and the child_id assigned just after. Not very intuitive, I probably need to find a completely different and a better way to do so, probably by creating the Child in the constructor instead (reopening https://github.com/mysensors/NodeManager/issues/198 for this).

Regarding setSleep and setReportInterval, measurement are driven only by setReportInterval. Those are different because there are situations (e.g. relays, motion sensors) when you have nothing to report but you want the capture the heartbeat from sleep(). What you are looking for is something I never considered which is actually a pretty nice idea (taking measurements over a given time period and than reporting at the end). I've created https://github.com/mysensors/NodeManager/issues/243 for this. With the new architecture, since keeping already track of the average within Child, should not be so difficult even if would probably require an additional Timer. Thanks!

reinhold

@user2684 Thanks for the spreadsheet with your memory footprints. Are these the sketch size and the free memory that the compiler reports? Or do you measure them when running the sketch (i.e. including all dynamically allocated variables in SRAM)?

Also, one problem you don't seem to have covered is heap fragmentation. Particularly your new List class seems to be prone to heap fragmentation with a large number of sensors: each time a sensor is added, the list needs to re-allocate the memory for its pointers. The old memory will be released, but it will remain as a hole in the memory, where new variables might be allocated. However, the next time you add an element to the list, the new pointer array can probably not re-use some of those freed holes.
I'll provide a PR (https://github.com/mysensors/NodeManager/pull/247) that adds an allocateBlocks(count) method to the List, which can be used in the sketch and in the sensor's onSetup methods to pre-allocate the required sized. In turn, the _preAllocBlocks member does currently not serve any purpose, and I don't think it's really required functionality. In my eyes, it's more important to ensure that at a certain moment the required memory is allocated. It is not neccessary to keep a minimum nuber of blocks allocated at all times.
For a node with 10 physical sensors (and maybe one or more built-in sensors), this will save at least 22 bytes (1,1% of total SRAM!) and probably reduce heap fragmentation by quite a bit.

Another are where >22 bytes can be saved is the power manager. None of my nodes makes use of the power manager, yet each sensor needs to keep a (null) pointer to a power manager. Providing a compiler #define NO_MODULE_POWER_MANAGER can exclude everything PowerManager-related from compilation and save one pointer per sensor and node, i.e. with 10 sensors you'll save another 22 bytes (1,1% of total SRAM!) I'll provide a PR for this, too: https://github.com/mysensors/NodeManager/pull/248

Apart from that, I simply stripped down the SensorMQ class and removed all configurability for my own use (don't worry, I won't submit this as a pull request... it is simply meant to squeeze the sketch onto the pro mini with the boards that I have already built).
BTW, the SensorConfiguration is no problem as far as SRAM is concerned (it just adds the normal SRAM footprint of a Sensor instance): It has no class members of its own. Only the flash size is considerable, but for me this is not the issue, I'm running out of SRAM.

💬 NodeManager

16

11.7k

11.2k

113.0k