Porting MySensors to work with the RadioHead library
-
It makes sense to migrate the MySensors last,sender and destination bytes to RH from,source,dest.
I do not think it makes sense to move type (V_code) or command into the RH FLAGs. Typically when such space is reserved for the protocol in the header but not yet fully used, it's subject to change as the protocal evolves. If part of the goal is to take advantage of the maintenance and ongoing development of RH, we don't want to set up a conflict - because other users are not going to use those FLAGs our way anyway.
Basically, Radio Head should be concerned with delivering a payload (set of bytes) to a node. Everything specific to just the MySensors application should be in the payload (as seen by RH). That includes:
- child id ("sensor")
- command
- V_code ("type")
- version (if needed)
- ack - is this still needed??
If we want to remain compatible with other RH users (ie: their development), rather than re-allocating bytes or bits in the RH header to our own purposes, we would be putting our MySensors' header in the RH payload, but omitting from our header the redundant fields which are handled by RH:
- last,
- sender,
- destination
The RH payload would include the reduced MySensors header plus the MySensors payload.
@Zeph said:
I do not think it makes sense to move type (V_code) or command into the RH FLAGs. Typically when such space is reserved for the protocol in the header but not yet fully used, it's subject to change as the protocal evolves. If part of the goal is to take advantage of the maintenance and ongoing development of RH, we don't want to set up a conflict - because other users are not going to use those FLAGs our way anyway.
From the RH documentation / the source code:
- for Datagram FLAGS:
A bitmask of flags. The most significant 4 bits are reserved for use by RadioHead. The least significant 4 bits are reserved for applications. - for Router/Mesh FLAGS:
Optional flags for use by subclasses or application layer
Not setting up a conflict with the RH code will be the biggest issue: the two benefits of RH are the multi radio support and the more advanced Mesh topology - which would be worth a couple of additional bytes in flash and ram - but the other reason for the big overhead is that several advanced features of the nRF24 chip are not used because they are not available in all supported chips.
Two examples:
- the multi-pipe capability of nRF24 would allow to filter traffic to only current address and broadcast and avoid that any other traffic ever reaches the MCU (only slightly less code but wondering if that helps with battery powered nodes).
- the ReliableDiagram processing in code essentially duplicates capabilities that are build into the chipset as well (auto-acknowledge / autoresubmit)
I know you are interested in getting RF69 supported but the question is how much negative impact for all the other users (with current nRF24 setup) is acceptable. That's like using the RF69 and implementing an AES encryption in code rather than using the build-in capabilities of the chip...
-
Didn't want to sound too negative yesterday...
- I'm still looking at the RH library but changes will be major and will not necessarily be in line with the RH purpose so Mike might not want to merge them into the main codebase: mainly defining what driver is used at compile time of the library (similar to MyConfig) which would then allow to e.g. "bypass" the majority of ReliableDatagram etc.
- what we should definitely do is splitting application layer and network layer: no longer inherit MySensor from RF24 but create an RF24 instance at runtime / leave application layer message handling in MySensors / move network layer message handling to a separate class (the "driver" for RF24). This will make it way easier to switch radios in the future.
-
Didn't want to sound too negative yesterday...
- I'm still looking at the RH library but changes will be major and will not necessarily be in line with the RH purpose so Mike might not want to merge them into the main codebase: mainly defining what driver is used at compile time of the library (similar to MyConfig) which would then allow to e.g. "bypass" the majority of ReliableDatagram etc.
- what we should definitely do is splitting application layer and network layer: no longer inherit MySensor from RF24 but create an RF24 instance at runtime / leave application layer message handling in MySensors / move network layer message handling to a separate class (the "driver" for RF24). This will make it way easier to switch radios in the future.
@ToSa said:
Didn't want to sound too negative yesterday...
- I'm still looking at the RH library but changes will be major and will not necessarily be in line with the RH purpose so Mike might not want to merge them into the main codebase: mainly defining what driver is used at compile time of the library (similar to MyConfig) which would then allow to e.g. "bypass" the majority of ReliableDatagram etc.
Is it not sufficient that we define the radio at compile time in the sensors and the gateway as we have done now?
-
@ToSa said:
Didn't want to sound too negative yesterday...
- I'm still looking at the RH library but changes will be major and will not necessarily be in line with the RH purpose so Mike might not want to merge them into the main codebase: mainly defining what driver is used at compile time of the library (similar to MyConfig) which would then allow to e.g. "bypass" the majority of ReliableDatagram etc.
Is it not sufficient that we define the radio at compile time in the sensors and the gateway as we have done now?
@kolaf said:
Is it not sufficient that we define the radio at compile time in the sensors and the gateway as we have done now?
I'm talking about changing how the library is compiled and essentially having a separate ReliableDiagram class that bypasses manual handling of ACKs etc. and instead leaves reliable delivery to the radio itself. Mesh and Router are derived from the ReliableDatagram class - therefore we would need to define which radio to use prior to library compilation - not just prior to sketch compilation.
-
@Zeph said:
I do not think it makes sense to move type (V_code) or command into the RH FLAGs. Typically when such space is reserved for the protocol in the header but not yet fully used, it's subject to change as the protocal evolves. If part of the goal is to take advantage of the maintenance and ongoing development of RH, we don't want to set up a conflict - because other users are not going to use those FLAGs our way anyway.
From the RH documentation / the source code:
- for Datagram FLAGS:
A bitmask of flags. The most significant 4 bits are reserved for use by RadioHead. The least significant 4 bits are reserved for applications. - for Router/Mesh FLAGS:
Optional flags for use by subclasses or application layer
Not setting up a conflict with the RH code will be the biggest issue: the two benefits of RH are the multi radio support and the more advanced Mesh topology - which would be worth a couple of additional bytes in flash and ram - but the other reason for the big overhead is that several advanced features of the nRF24 chip are not used because they are not available in all supported chips.
Two examples:
- the multi-pipe capability of nRF24 would allow to filter traffic to only current address and broadcast and avoid that any other traffic ever reaches the MCU (only slightly less code but wondering if that helps with battery powered nodes).
- the ReliableDiagram processing in code essentially duplicates capabilities that are build into the chipset as well (auto-acknowledge / autoresubmit)
I know you are interested in getting RF69 supported but the question is how much negative impact for all the other users (with current nRF24 setup) is acceptable. That's like using the RF69 and implementing an AES encryption in code rather than using the build-in capabilities of the chip...
- for Datagram FLAGS:
-
@ToSa said:
the two benefits of RH are the multi radio support and the more advanced Mesh topology
Could someone summarize the differences between the current MySensors mesh and the RadioHead mesh, and why the latter is preferable for us?
RH is a true mesh where every node can communicate with any other directly (if in range). All nodes acts as repeaters.
MySensors forms nodes forms a star network where every node has a parent and potentially a few child sensors. The gateway node is the "center". Every node always tries to find the closest (least number of hops) route to gateway.
GW / \ A D / \ B CIf B want to communicate with C messages has to pass A in the MySensors case. In RH C and B will communicate directly if the can hear each other.
RH does not use the nifty features NRF24L01 offers which offloads mcu (pipes/addresses) nor auto ack (impossible in RH setup). So nodes in range has to process every message it can "hear". And this can potentially be a lot.
-
RH is a true mesh where every node can communicate with any other directly (if in range). All nodes acts as repeaters.
MySensors forms nodes forms a star network where every node has a parent and potentially a few child sensors. The gateway node is the "center". Every node always tries to find the closest (least number of hops) route to gateway.
GW / \ A D / \ B CIf B want to communicate with C messages has to pass A in the MySensors case. In RH C and B will communicate directly if the can hear each other.
RH does not use the nifty features NRF24L01 offers which offloads mcu (pipes/addresses) nor auto ack (impossible in RH setup). So nodes in range has to process every message it can "hear". And this can potentially be a lot.
@hek
Thanks, that was very helpful, and a good explanation.That helps explain the overhead in the RH library mesh. It sounds cool for some purposes, but it also sounds like possible overkill for the wireless sensor network. Allowing B and C to communicate directly is not a primary use case. The MySensors is purpose-built for the sensor network with central gateway use case, and is pretty lean thereby. As I see it, even allowing B to send to C via A is not a primary use case, but sort of falls out "for free" given the routing approach that allows repeating nodes.
Aside - can B send a message to itself via A?
In my case, I'm mainly focusing on multi-sensor nodes, so RAM or Flash could be in short supply with a larger radio library. Worse still, I want to be able to incorporate MySensors functionality into nodes which also have non MySensors functions (eg: xmas light control also usng nRF24L01+), so space is even more at a premium.
So while the RH option is interesting, and a lighter weight RH derivative may pan out, I hope the MySensor WSN radio layer will continue to provide a "lean and mean" alternative into the future.
-
RH is a true mesh where every node can communicate with any other directly (if in range). All nodes acts as repeaters.
MySensors forms nodes forms a star network where every node has a parent and potentially a few child sensors. The gateway node is the "center". Every node always tries to find the closest (least number of hops) route to gateway.
GW / \ A D / \ B CIf B want to communicate with C messages has to pass A in the MySensors case. In RH C and B will communicate directly if the can hear each other.
RH does not use the nifty features NRF24L01 offers which offloads mcu (pipes/addresses) nor auto ack (impossible in RH setup). So nodes in range has to process every message it can "hear". And this can potentially be a lot.
@hek said:
RH does not use the nifty features NRF24L01 offers which offloads mcu (pipes/addresses) nor auto ack (impossible in RH setup). So nodes in range has to process every message it can "hear". And this can potentially be a lot.
So MySensors can use both individual node addresses as well a broadcast, while RH uses only broadcast (at the radio level)? I understand why RH needs to do this, and I wonder if the receive FIFO ever overruns with unneeded packets.
I wonder if MySensors gets much savings from using the auto-ack of ESB. If you already have network ack code anyway, how much code & ram is saved by sometimes using ESB (one hop away) and sometimes not? (I do get that a star network with no repeater nodes could use only ESB autoack and save some code).
-
@Zeph said:
So MySensors can use both individual node addresses as well a broadcast, while RH uses only broadcast (at the radio level)?
Yes.
I think end-to-end ack is very important. The inter-node ack is only used as a trigger re-routing today (if communication fails 3 times to parent node .. the node tries to find another parent).
-
Could someone please explain to me how the direct messaging of the NRF24 works? In my mind every radio network is a broadcast network on the physical layer, and it is up to the receiving radio to determine whether the packet is relevant or not. How is this different from your radios? Is it perhaps that your radios have a hardware address so that the filtering is done on the hardware level, while Radiohead uses a software address, so each packet has to be explicitly processed? I have no idea, I'm just guessing here...
-
Could someone please explain to me how the direct messaging of the NRF24 works? In my mind every radio network is a broadcast network on the physical layer, and it is up to the receiving radio to determine whether the packet is relevant or not. How is this different from your radios? Is it perhaps that your radios have a hardware address so that the filtering is done on the hardware level, while Radiohead uses a software address, so each packet has to be explicitly processed? I have no idea, I'm just guessing here...
Yes,
NRF chip offer 6 reading "pipes". In practice it acts as a hw filter for addresses between 0-255.
I use this to have one broadcast address (255) which all repeating nodes and gateway listens to and each sensor also listen to its own address.
-
Looking at the documentation for the RF69 (page 41) it talks about sync word recognition. This is a programmable word that apparently can be used as the node's hardware address, similarly to the NRF24. Perhaps it is possible to expand the Radiohead drivers to utilise this functionality where it is available for filtering incoming packets?
As for the dynamic mesh, this is a feature I find especially useful for one of my dream applications. We have horses, and it would be really fun to place a radio module with a accelerometer, and maybe GPS receiver, on each horse to track their movements. This could also be combined with a breakable wire in the harness which would let us know if the radio had been ripped off of the horse. Having a dynamic mesh will greatly increase the range of flexibility of such a monitoring network. I know that this is somewhat outside of the normal use case, but how fun wouldn't it be to build this? :-).
Personally I'm also a bit interested in direct sensor to sensor configurations to allow for tightly coupled control systems without the need for a central controller in the loop (all the time). I know, we have resource limits so that I can probably not have everything, but one can dream...
-
Nevermind the sink word recognition, this appears to function as some kind of network ID has to be the same for both the sender and receiver. However, there is also an optional address byte which is considered by the radio before the packet enters the FIFO queue. Page 55 of http://www.hoperf.com/upload/rfchip/RF69-V1.2.pdf
-
Nevermind the sink word recognition, this appears to function as some kind of network ID has to be the same for both the sender and receiver. However, there is also an optional address byte which is considered by the radio before the packet enters the FIFO queue. Page 55 of http://www.hoperf.com/upload/rfchip/RF69-V1.2.pdf
-
Looking at the documentation for the RF69 (page 41) it talks about sync word recognition. This is a programmable word that apparently can be used as the node's hardware address, similarly to the NRF24. Perhaps it is possible to expand the Radiohead drivers to utilise this functionality where it is available for filtering incoming packets?
As for the dynamic mesh, this is a feature I find especially useful for one of my dream applications. We have horses, and it would be really fun to place a radio module with a accelerometer, and maybe GPS receiver, on each horse to track their movements. This could also be combined with a breakable wire in the harness which would let us know if the radio had been ripped off of the horse. Having a dynamic mesh will greatly increase the range of flexibility of such a monitoring network. I know that this is somewhat outside of the normal use case, but how fun wouldn't it be to build this? :-).
Personally I'm also a bit interested in direct sensor to sensor configurations to allow for tightly coupled control systems without the need for a central controller in the loop (all the time). I know, we have resource limits so that I can probably not have everything, but one can dream...
@kolaf said:
As for the dynamic mesh, this is a feature I find especially useful for one of my dream applications.
Yep, both pros and cons. :)
A nice feature you can do with MySensors is to set a static parent (when calling gw.begin()). This could be useful for a presence sensor e.g. mounted on a car set statically to contact a repeater node mounted near the garage. The car sensor won't try to find or contact any other node than the garage one.
-
Looking at the documentation for the RF69 (page 41) it talks about sync word recognition. This is a programmable word that apparently can be used as the node's hardware address, similarly to the NRF24. Perhaps it is possible to expand the Radiohead drivers to utilise this functionality where it is available for filtering incoming packets?
As for the dynamic mesh, this is a feature I find especially useful for one of my dream applications. We have horses, and it would be really fun to place a radio module with a accelerometer, and maybe GPS receiver, on each horse to track their movements. This could also be combined with a breakable wire in the harness which would let us know if the radio had been ripped off of the horse. Having a dynamic mesh will greatly increase the range of flexibility of such a monitoring network. I know that this is somewhat outside of the normal use case, but how fun wouldn't it be to build this? :-).
Personally I'm also a bit interested in direct sensor to sensor configurations to allow for tightly coupled control systems without the need for a central controller in the loop (all the time). I know, we have resource limits so that I can probably not have everything, but one can dream...
@kolaf said:
As for the dynamic mesh, this is a feature I find especially useful for one of my dream applications. We have horses, and it would be really fun to place a radio module with a accelerometer, and maybe GPS receiver, on each horse to track their movements. This could also be combined with a breakable wire in the harness which would let us know if the radio had been ripped off of the horse. Having a dynamic mesh will greatly increase the range of flexibility of such a monitoring network.
Just brainstorming, but what if we consider adding the concept of a roving node to the MySensors network, without the overhead of a full mesh? Your use case doesn't require that all nodes be meshed with each other, just that a class of leaf nodes be able to move around.
If I understand, a gateway or repeater node will forward a packet from anybody, if it recognizes the destination (as a node id in its routing table). Suppose you had coverage of your corral (or whatever) via gateway and various repeaters. In theory (perhaps naive theory) the roving node could report to the gateway by broadcasting a message picked up by whatever repeater is nearest; being addressed to node 0, the repeater knows what to do with it.
There would be no network ack and no way for the gateway to send to the roving node as described so far. That is, unless this special "rover packet" caused each relay to rewrite that entry of its routing table as it was passed to the gateway, to enable an updated path back to the roving node. If your horses run around among many nodes often, EEPROM wear could be a concern, unless there was a small "routing override table" in RAM to handle roving nodes.
One part I'm not clear about is how to avoid two or more nearby repeaters both forwarding this broadcast packet from the roving node. That might be tricky if it often causes OTA collisions. It could also complicate the back route logic, IF that was implemented. If we could avoid or gracefully recover from collisions, the gateway and controller may be able to deal with multiple copies of the same packet (not unlike the MQTT level 1 "deliver at least once").
And - it might be technically infeasible to enhance MySensors networking to support roving nodes. But in that case I'm sure I'll learn something from the manner in which it's shot down :-)
-
@kolaf said:
an optional address byte
The trouble is you have to distinguish between broadcasts and direct messages, so you need to listen at 2 addresses at a time, so to say.
@Yveaux said:
@kolaf said:
an optional address byte
The trouble is you have to distinguish between broadcasts and direct messages, so you need to listen at 2 addresses at a time, so to say.
There is a separate optional broadcast address byte. I'm not sure how this is used, but I'm guessing that the radio checks against both addresses before deciding what to do with the message. That should solve the broadcast problem.
-
@kolaf said:
As for the dynamic mesh, this is a feature I find especially useful for one of my dream applications. We have horses, and it would be really fun to place a radio module with a accelerometer, and maybe GPS receiver, on each horse to track their movements. This could also be combined with a breakable wire in the harness which would let us know if the radio had been ripped off of the horse. Having a dynamic mesh will greatly increase the range of flexibility of such a monitoring network.
Just brainstorming, but what if we consider adding the concept of a roving node to the MySensors network, without the overhead of a full mesh? Your use case doesn't require that all nodes be meshed with each other, just that a class of leaf nodes be able to move around.
If I understand, a gateway or repeater node will forward a packet from anybody, if it recognizes the destination (as a node id in its routing table). Suppose you had coverage of your corral (or whatever) via gateway and various repeaters. In theory (perhaps naive theory) the roving node could report to the gateway by broadcasting a message picked up by whatever repeater is nearest; being addressed to node 0, the repeater knows what to do with it.
There would be no network ack and no way for the gateway to send to the roving node as described so far. That is, unless this special "rover packet" caused each relay to rewrite that entry of its routing table as it was passed to the gateway, to enable an updated path back to the roving node. If your horses run around among many nodes often, EEPROM wear could be a concern, unless there was a small "routing override table" in RAM to handle roving nodes.
One part I'm not clear about is how to avoid two or more nearby repeaters both forwarding this broadcast packet from the roving node. That might be tricky if it often causes OTA collisions. It could also complicate the back route logic, IF that was implemented. If we could avoid or gracefully recover from collisions, the gateway and controller may be able to deal with multiple copies of the same packet (not unlike the MQTT level 1 "deliver at least once").
And - it might be technically infeasible to enhance MySensors networking to support roving nodes. But in that case I'm sure I'll learn something from the manner in which it's shot down :-)
-
@kolaf said:
As for the dynamic mesh, this is a feature I find especially useful for one of my dream applications.
Yep, both pros and cons. :)
A nice feature you can do with MySensors is to set a static parent (when calling gw.begin()). This could be useful for a presence sensor e.g. mounted on a car set statically to contact a repeater node mounted near the garage. The car sensor won't try to find or contact any other node than the garage one.
-
@Yveaux said:
@kolaf said:
an optional address byte
The trouble is you have to distinguish between broadcasts and direct messages, so you need to listen at 2 addresses at a time, so to say.
There is a separate optional broadcast address byte. I'm not sure how this is used, but I'm guessing that the radio checks against both addresses before deciding what to do with the message. That should solve the broadcast problem.
@kolaf said:
There is a separate optional broadcast address byte. I'm not sure how this is used, but I'm guessing that the radio checks against both addresses before deciding what to do with the message. That should solve the broadcast problem.
That's good!
