Discussion: Reliable delivery

[mfalkvidd]

Very cool! Thanks a lot @hek! Can't believe I have read several hundred forum posts about the protocol design and ack problems the last two weeks without realizing that what I needed was already included in the library.

From the discussions I've read it looks like almost no-one else has understood either.

Some ideas to help people understand how to use the built-in features for reliable delivery:

1. Add an example that has a complete implementation of end-to-end ack usage. This includes

• setting up timer(s) for re-sending
• storing sent messages so they can be re-sent
• determining which message was acked when an ack message is received (there might be several messages that haven't been acked yet)
• removing acked message from the sent message store and clearing the timer(s)
• re-sending when the timers expire

2. Update the documentation for bool send(MyMessage &msg, bool ack); (and similar functions) to explain that the bool returned is the result of the next-hop ack.
3. Rename the bool ack parameter to bool end-to-end-ack or something similar to make it clear(er) that there are two types of acks
4. In the documentation for send() (and similar functions), refer to the example in (1) for information on how to use end-to-end ack.
5. When the example in (1) is good enough, add some of the required code to the MySensors library so people don't need to copy-paste a lot of code into their sketches. Surround the code with ifdefs to make it optional.

(2) and (3) should be quite easy to do. Can it be done with a pull request or how are documentation improvements handled?

I'm willing to do 1 (will post the sketch in the forum for public scrutiny/feedback of course).

4 can be done when the community think the example is "good enough"

5 can wait until the example has been thoroughly vetted.

[hek]

Sounds like a plan.

The code documentation could be solved with a PR as you say. But the main site isn't available on github (a messy thing) so I have to update that part when PR arrives/has been merged.

[mfalkvidd]

ok. I can create the PR. From where can I clone the repo?

[mfalkvidd]

After a quick chat with hek I now know that I should add code comments to the relevant functions in the code and he'll manually update the API documentation. I'll post a link to the PR when I'm done.

[robosensor]

Seems like it is impossible now to distinguish for which message we have received acknowledgement. For example, when you just sent two identical messages. Message should contain message ID field (also called packet id or sequence id, unique for sending side) either in protocol headers (need to change protocol and break compatibility) or in user-defined message body (also breaks compatibility).

[mfalkvidd]

@robosensor Yes it is impossible to distinguish between acks for two identical messages. But in which use case is that a problem?

[robosensor]

@mfalkvidd for example, to know last actuator state.

Imagine that you are sending three commands:

1. ON
2. OFF
3. ON

Then you receive two confirmations: ON and OFF. For which one ON ack is received? Is current actuator state after receiving two confirmations ON or OFF? How to determine which message (first or third) is lost and what you need to resend?

[mfalkvidd]

@robosensor good example. As you said before, there is no way to solve that with the current protocol. However, the example can be made even simpler, it is enough to just send OFF then ON. In-order delivery is not guaranteed so you don't know which message was received last.

[robosensor]

@mfalkvidd yes, I understand that this feature will break everything.

Also, message id field can help to filter out duplicate messages on receiving side (due to message acknowledgement is not received by sending side) and can help to re-order incoming messages in right way using increasing message ids.

Even more, sending acknowledgement with only numeric id instead of full message data can help to remove load from MCU (need to parse full message), from RAM (need to store full message), and from 'network' (both wired and wireless, data channel will be busy less time).

[mfalkvidd]

@robosensor yes, I agree.

[mfalkvidd]

I got ready to do a PR for this:

2. Update the documentation for bool send(MyMessage &msg, bool ack); (and similar functions) to explain that the bool returned is the result of the next-hop ack.

But the code for 1.5.4 already contains this:

@return true Returns true if message reached the first stop on its way to destination.

so the problem seems to be the that @return statements didn't make it to http://www.mysensors.org/download/sensor_api_15
@hek: It seems like a PR won't help in this case?

[Yveaux]

@mfalkvidd @return is doxygen formatting info, while the link points to documentation manually written. For 2.0 doxygen is on the agenda, and maybe the manual docs will be replaced by doxygen generated docs. Probably @hek has better insight on the subject.

[mfalkvidd]

@Yveaux yes he has. As mentioned above hek said he would manually update the documentation after I had created a PR but it seems like a PR is not needed.

[hek]

It's already building, but only @Anticimex has been a good boy adding the correct doxygen comments to his code .
http://ci.mysensors.org/job/MySensorsArduino/branch/development/Doxygen_HTML/

[Mark Swift]

Has there been any progress on this guys?

[mfalkvidd]

@Mark-Swift thanks for asking. Number 3 and 4 in this post were actually already in place in the code and documentation before this thread was started, but it seems like updating the site to reflect that is quite hard. I don't know how updating the site works unfortunately. The only advice I can give is "Don't look at the online documentation, dig through the source code instead".

number 1 is blocked by me getting enough motivation to do the work required to get the development channel working without screwing up my existing projects. There are so many changes to folder locations and similar that I am unable to compile anything if I switch to the development version and I'm not looking forward to figuring out how to handle that.

[BartE]

@mfalkvidd said:

Add an example that has a complete implementation of end-to-end ack usage. This includes
setting up timer(s) for re-sending
storing sent messages so they can be re-sent
determining which message was acked when an ack message is received (there might be several messages that haven't been acked yet)
removing acked message from the sent message store and clearing the timer(s)
re-sending when the timers expire

Are you referring to this number 1 post?

Does this functionality need to be part of the bool send(MyMessage &msg, bool ack); function or become a layer over the send() function?

Maybe i can have a look one of these days...

[mfalkvidd]

@BartE I'm referring to the post that starts with "Very cool! Thanks a lot @hek!"

[Mark Swift]

I'd love to see some examples of how we can implement a reliable delivery method into critical nodes, especially now 2.0.0 is out in the wild.

[Mark Swift]

Any news on this, guys? Do we have some good examples?