Discussion: Reliable delivery
-
Very cool! Thanks a lot @hek! Can't believe I have read several hundred forum posts about the protocol design and ack problems the last two weeks without realizing that what I needed was already included in the library.
From the discussions I've read it looks like almost no-one else has understood either.
Some ideas to help people understand how to use the built-in features for reliable delivery:
- Add an example that has a complete implementation of end-to-end ack usage. This includes
- setting up timer(s) for re-sending
- storing sent messages so they can be re-sent
- determining which message was acked when an ack message is received (there might be several messages that haven't been acked yet)
- removing acked message from the sent message store and clearing the timer(s)
- re-sending when the timers expire
- Update the documentation for bool send(MyMessage &msg, bool ack); (and similar functions) to explain that the bool returned is the result of the next-hop ack.
- Rename the bool ack parameter to bool end-to-end-ack or something similar to make it clear(er) that there are two types of acks
- In the documentation for send() (and similar functions), refer to the example in (1) for information on how to use end-to-end ack.
- When the example in (1) is good enough, add some of the required code to the MySensors library so people don't need to copy-paste a lot of code into their sketches. Surround the code with ifdefs to make it optional.
(2) and (3) should be quite easy to do. Can it be done with a pull request or how are documentation improvements handled?
I'm willing to do 1 (will post the sketch in the forum for public scrutiny/feedback of course).
4 can be done when the community think the example is "good enough"
5 can wait until the example has been thoroughly vetted.
I got ready to do a PR for this:
- Update the documentation for bool send(MyMessage &msg, bool ack); (and similar functions) to explain that the bool returned is the result of the next-hop ack.
But the code for 1.5.4 already contains this:
@return true Returns true if message reached the first stop on its way to destination.so the problem seems to be the that @return statements didn't make it to http://www.mysensors.org/download/sensor_api_15
@hek: It seems like a PR won't help in this case? -
I got ready to do a PR for this:
- Update the documentation for bool send(MyMessage &msg, bool ack); (and similar functions) to explain that the bool returned is the result of the next-hop ack.
But the code for 1.5.4 already contains this:
@return true Returns true if message reached the first stop on its way to destination.so the problem seems to be the that @return statements didn't make it to http://www.mysensors.org/download/sensor_api_15
@hek: It seems like a PR won't help in this case?@mfalkvidd @return is doxygen formatting info, while the link points to documentation manually written. For 2.0 doxygen is on the agenda, and maybe the manual docs will be replaced by doxygen generated docs. Probably @hek has better insight on the subject.
http://yveaux.blogspot.nl
-
@mfalkvidd @return is doxygen formatting info, while the link points to documentation manually written. For 2.0 doxygen is on the agenda, and maybe the manual docs will be replaced by doxygen generated docs. Probably @hek has better insight on the subject.
-
It's already building, but only @Anticimex has been a good boy adding the correct doxygen comments to his code .
http://ci.mysensors.org/job/MySensorsArduino/branch/development/Doxygen_HTML/ -
-
Has there been any progress on this guys?
@Mark-Swift thanks for asking. Number 3 and 4 in this post were actually already in place in the code and documentation before this thread was started, but it seems like updating the site to reflect that is quite hard. I don't know how updating the site works unfortunately. The only advice I can give is "Don't look at the online documentation, dig through the source code instead".
number 1 is blocked by me getting enough motivation to do the work required to get the development channel working without screwing up my existing projects. There are so many changes to folder locations and similar that I am unable to compile anything if I switch to the development version and I'm not looking forward to figuring out how to handle that.
-
@Mark-Swift thanks for asking. Number 3 and 4 in this post were actually already in place in the code and documentation before this thread was started, but it seems like updating the site to reflect that is quite hard. I don't know how updating the site works unfortunately. The only advice I can give is "Don't look at the online documentation, dig through the source code instead".
number 1 is blocked by me getting enough motivation to do the work required to get the development channel working without screwing up my existing projects. There are so many changes to folder locations and similar that I am unable to compile anything if I switch to the development version and I'm not looking forward to figuring out how to handle that.
@mfalkvidd said:
Add an example that has a complete implementation of end-to-end ack usage. This includes
setting up timer(s) for re-sending
storing sent messages so they can be re-sent
determining which message was acked when an ack message is received (there might be several messages that haven't been acked yet)
removing acked message from the sent message store and clearing the timer(s)
re-sending when the timers expireAre you referring to this number 1 post?
Does this functionality need to be part of the bool send(MyMessage &msg, bool ack); function or become a layer over the send() function?
Maybe i can have a look one of these days...
-
@mfalkvidd said:
Add an example that has a complete implementation of end-to-end ack usage. This includes
setting up timer(s) for re-sending
storing sent messages so they can be re-sent
determining which message was acked when an ack message is received (there might be several messages that haven't been acked yet)
removing acked message from the sent message store and clearing the timer(s)
re-sending when the timers expireAre you referring to this number 1 post?
Does this functionality need to be part of the bool send(MyMessage &msg, bool ack); function or become a layer over the send() function?
Maybe i can have a look one of these days...
-
I'd love to see some examples of how we can implement a reliable delivery method into critical nodes, especially now 2.0.0 is out in the wild.
-
Any news on this, guys? Do we have some good examples?
-
@hek said:
No, only the findal destination node will answer with a soft ack message.
Note, the (soft) ack message has to be picked up by yourself in your incomingMessage function.
Thanks for explaining. Just checking if my understanding is correct:
bool send(MyMessage &msg, bool ack);will return the result of the hardware ack, regardless if the second parameter is true or false?
So my post above should have been like this:
Hardware ack is always on
- Hardware ack is hop-to-hop acknowledgment
- Sending is a blocking function which returns true if an hardware ack was received, and false if ~70ms passed without receiving an hardware ack.
- Sending hardware ack (when the message has reached the next-hop node) is handled automatically by the library(/radio)
- Receiving an hardware ack is handled automatically by the library, by setting the return value from the send call mentioned above
- Up to 15 tries are made automatically by the library. Further re-send needs to be handled manually in each sketch (a while-loop with a counter counting up to maxRetries and using wait() between each resend seems to be the most common solution)
Software ack is what I get when using send(msg, true) (and is also supported by some other functions like present(), sendSketchInfo() and sendBatteryLevel()
- Software ack is end-to-end acknowledgment
- Sending is a non-blocking function
- Sending ack (when the message has reached its final destination) is handled automatically by the library
- Re-send needs to be handled manually in each sketch (setting up a timer when sending the original message might be a good solution)
- Receiving an ack needs to be handled manually in each sketch (by checking msg.isAck() in incomingMessage and clearing the timer mentioned above)
@mfalkvidd I just came across this post and I am very supprised by the fact that this information is not written anywhere in the documentation ... I believe this is one of the most important stuff everyone should know to actually use MySensors in a real world application - not just diy/hobby project :/ till now I was sure that return value of send funtion (which is NOT explained in the documentation !) is the end-to-end confirmation ...
-
@mfalkvidd I just came across this post and I am very supprised by the fact that this information is not written anywhere in the documentation ... I believe this is one of the most important stuff everyone should know to actually use MySensors in a real world application - not just diy/hobby project :/ till now I was sure that return value of send funtion (which is NOT explained in the documentation !) is the end-to-end confirmation ...
-
I have to chime in and share my agreement. This seems like a fundamental piece of information, I mean, we all want reliable delivery right? For now I've copied a resend function I found on the forums (with some tweaks), but in my opinion, such code should at least me in the core - with the appropriate explanation.
-
The new and still experimental, but quite extensive Doxygen documentation contains the documentation stuff that's available in the source code. For the return value of the send function, see https://ci.mysensors.org/job/Verifiers/job/MySensors/branch/development/Doxygen_HTML/group__MySensorsCoregrp.html#gadbea3e429757e7fbc66a54776590a2e8
-
-
Returns true if message reached the first stop on its way to destination
What will be returned in the case of NRF24L01 as gateway's receiver at destination and internal nrf24 buffer is already contains 3 unread messages?
-
Returns true if message reached the first stop on its way to destination
What will be returned in the case of NRF24L01 as gateway's receiver at destination and internal nrf24 buffer is already contains 3 unread messages?
-
Hi Guyz,
I have been watching this wonderful forum for a while silently now and just started looking at few implementations myself. I am really surprised to know that so much work has been put in MySensors and this area is not given more attention and I second few opinions here that reliable delivery is one of the most important thing especially for actuators. I will try to do an example of end to end (node to gateway) reliable delivery as gateway to controller is pretty much reliable anyways. Meanwhile if anyone has any example already created, please do share over here.
-
Hi Guyz,
Not sure if anyone is interested in this but I would like to share my thinking here.- For reliable delivery, Only source and destination should be taking part, transport can do its best but as its can change we cannot expect it to behave precisely.
- For the sake of simplicity I am assuming all message to be Idempotent. (i.e if this is executed multiple times result will be same. say turn on a switch). I know about OFF ON OFF scenario but for most of our requirements and times involved, I would say that we can assume that when we are delivering one message other will not come. However if any actuator changes so frequently we should think about it.
- For Controller to to Actuator delivery, I am implementing a solution where actuator send its state back to controller on every change. When controller sends a command out to actuator it will wait for a second or two to see if the state is updated. If not it retries and it retries for a max number of times everyday which is getting logged to keep track of reliability in transport. This way if my max retry is not reached I can be sure that every message has reached actuator.
- For actuator to controller delivery I am planning to do similar where it sends until it returns true with a acknowledge requested and keep sending it until either ack has come or max retry has been reached. Also keeping track of attempts it had to make and send those as well reliably to controller for analysis. I know this will not ensure gateway to controller delivery but I will see how this one works and if needed will add that later. But not doing it now to avoid dependency of controller on my individual node.
