NodeManager v1.7-beta: feedback needed

[user2684]

Hi, NodeManager v1.7 is almost ready and all the major features and bug fixes for this release have been now implemented. For those willing to give it a try to identify any potential issue or just share a comment, please feel free to do so (https://github.com/mysensors/NodeManager/tree/development).

There are no compatibility issue with the MySensors library but an automatic upgrade from NodeManager v1.6 is not possible due to a deep architecture review.

Please review the README file and the release notes for a summary of the new features. For a full list of what has changed please refer to https://github.com/mysensors/NodeManager/milestone/9?closed=1.

Ideally I'd love to release it in a couple of weeks if no major issues will be reported. Thanks in advance!
Thanks!

[pragtich]

Hello,

I took some time yesterday to try out the development branch, and I must say that it seems to be a great step forward. I love how all configuration now is in one file, and that you now get direct access to the Sensors' objects. Also, the FEATURE_ constants really simplify configuration.

One thing that surprised me, which probably was the same in the previous versions, is that the interrupt driven sensors (SensorInterrupt and its derivatives), define an empty onLoop. This way, it is not possible to get them to report it's status after a given interval, despite not having received an interrupt. Something like the Force Update Timer in Conditional Report mode.
That's a feature I would like to have: a sensor that responds to interrupts (so has an immediate response to change, but sleeps a lot), but also sends a confirmation of its status every now and then.
I imagine you have thought about this yourself: how would you feel if I tried to formuate a pull request for this feature?

I suspect that this may also explain some strange behaviour that I see on SensorInterrupt sensors when FEATURE_CONDITIONAL_REPORT == 1. In that case, it seems that the SensorInterrupt never manages to send an update message. See my logfile here. This works fine with conditional reporting switched off.
Or am I just interpreting things wrong? I have to admit that I'm quite new to MySensors.

Best regards,
pragtich

[user2684]

@joris-pragt thanks for your feedback first of all! The way I'm using things here is to make the node sleeping at e.g. 1 hour intervals so it will report its heartbeat once awake, but still keeping onLoop() empty. Would it be feasible in your case as well? Ideally I'd prefer not to report the same value without an interrupt because some controller may act upon it, even if is not changed. Let me know if I've missed something.
Thanks!

[pragtich]

@user2684 , glad to help.

I guess there's several situations imagineable. I made a fork and will play around with some options. I understand your point of view also, so anything I do should eventually be configureable.

Actually, I am working with HomeAssistant, and it actively discards updates where the value did not change, so in that case it is no issue to receive multiple updates of the same value (actually, there has been much discussion on this in HomeAssistant, as this behaviour also makes it difficult to see weither or not a sensor node is still online, or that the value simply did not change).

I can explain my use case: I have a reed switch mounted to a lock, which is supposed to report the locked/unlocked state of the lock. For this application, the state switches are quite infrequent (max a few times per day), but can also be in very quick succession if someone changes their mind while opening the lock.
I want to be reasonably sure that I have the correct state in the controller. Now, we are working with battery-powered radio nodes, so there is always the chance that a message does not get transmitted or received for whatever reason. This will lead to the state in the controller being incorrect, and this incorrect state will continue until the next correct state change gets received.
I want to be sure, that after a max of n minutes, I can conclude that either the state is surely correct, or the node is apparently offline. Otherwise, the state might be incorrect (perhaps for days at a time).

Example:
In the current implementation of NodeManager, when the SensorInterrupt (I am using SensorDoor) fires twice in quick succession (open the switch, then close it before the state change message gets sent over the radio), we end up with an incorrect state. The second interrupt fires before we send the first message and therefore we do not send the second message.
This situation is totally not implausible, it happens to me sometimes.
It also ends up leading to an incorrect reading at the controller.

As I said, I'll spend a few weeks of play time trying to come up with a way to trigger regular timed updates. Of course this must be configureable, in order to be ok with your use case. I am thinking this could be achieved in several ways:

1. Add a onLoop and let it gather the data. Probably remove data gathering from the onInterrupt routine, but somehow signal to the loop that we were woken up and need a message sent.
2. Base everything on SensorDigitalInput, but add an interrupt routine to wake up and send a message.

Would love to hear your thoughts,
best regards,

pragtich

[user2684]

@pragtich thanks very clear now. I wonder if a way to achieve what you are trying to accomplish can be by leveraging https://github.com/mysensors/NodeManager/pull/312 which is basically allowing to add custom code by hooking into the different sensors' functions. You could add a callback into loop which is calling setValueInt() to the child which will eventually be reported to the gateway, provided a reporting interval is set. Would this be useful? Thanks!

[pragtich]

@user2684 FYI, I was successful in getting the periodic reporting to work. If you are interested, here's a branch of mine that has it running. I'm reasonably happy with how easy this was, the FEATURE_HOOKING turned out nice.
Thanks for your work!

pragtich

[user2684]

@pragtich cool! Glad the new features have been helpful for you and thanks for sharing your example

[Nick Willis]

Hi I've just started with NodeManager but not with MySensors. I've been writing my own sketches up to now but am impressed with NM's v1.7 as it makes so much sense and avoids so much of the repetition and debug time with manual coding.
I've got a DHT22 node up with 1.7 - nice and easy. I noticed the FEATURE_CONDITIONAL_REPORT feature and would like to know how to use it as it isn't covered in the documentation yet.
Since the sensor can often fluctuate by 0.1c (and humidity even more so) I'd like to only report for 'genuine' changes. Something like:

• If the value changes by more than 0.1c since the last value was reported, then report immediately. Otherwise don't report yet.
• Report anyway after 5 minutes if no report was made. This 5 minutes would get reset if a report was made by the first >0.1c condition.

I think that makes sense although open to ideas on the best way to keep regular but not too regular sensor reports.
Looking forward to rolling out NM across my sensor network and expanding!

[user2684]

@nick-willis thanks for the feedback first of all! You're right, there is a mistake in the documentation related to the Sensor API, I'll fix it ASAP. Anyway, once you turn FEATURE_CONDITIONAL_REPORT on, you can call the following independently for each sensor:

// [7] if true will report the measure only if different than the previous one (default: false)
    void setTrackLastValue(bool value);
    // [9] if track last value is enabled, force to send an update after the configured number of minutes
    void setForceUpdateMinutes(int value);

I wonder though if this would work as expected even if you need a single decimal comparison (e.g. since a float is stored and the float precision set is not considered when e.g. temperature changes from 20.11 to 20.19, it will get reported which is not what you are expecting). I'll check this one out too. Thanks!

[Nick Willis]

Ah great thanks that looks good! I guess you might need another parameter stating how much the value must differ from the previous to trigger the extra report. Given that the DHT22 only reports to 0.1c anyway it won't be an issue I think.
Which also reminds me - I notice the values come set to 2 d.p. but the sensor only goes to 1 d.p. so they end up coming down as : 21.60, 21.70, 21.90 etc.
Can we configure number of decimal places? Or in the case of DHT22 just keep it to 1 d.p.?

[user2684]

@nick-willis good advice, let me see if I can add this just before releasing v1.7 (https://github.com/mysensors/NodeManager/issues/334). Regarding the 21.60 etc. is because each child has a float precision set and if set to two (the default), two decimals will be always reported, regardless of the sensor. You can customize it with something like this:

dht22.children.get(1)->float_precision = 1;

Thanks

[Nick Willis]

Thanks for that - working great.
Been playing with the 1.7 branch and your new methods to do what I want and it seems a good approach.
I've been monitoring a DHT22 with settings:
dht22.children.get(1)->setValueDelta(0.1);
dht22.children.get(1)->setForceUpdateMinutes(10);
dht22.children.get(2)->setValueDelta(0.1);
dht22.children.get(2)->setForceUpdateMinutes(10);

My results are that I'm seeing a value every 10 mins but I'm never seeing the extra reports when the value delta > 0.1. My results:

Note a 0.2c change on the temperature set 22.0 to 22.2 with no extra report. Humidity shows this clearer where there is a jump from 37.9% to 37.2% over 10 mins. I would have expected to see some additional reports within that time rather than waiting for the forced report.
Anything I can do to test this further let me know!

[user2684]

@nick-willis strange, I run exactly the same test and was working fine, I wonder if there was any change in the code just after messing this up. I will test it again, meanwhile if you can collect NodeManager's logs based on this configuration and share would be great. Thanks!