Porting MySensors to work with the RadioHead library

kolaf

I am looking at the development branch of the github repository. My reference to the CRC probably comes from reading the documentation, not from reading the source code :-).

Based on your comments about the acknowledgements I assume that the code I see to handle ack messages relates to the end to end acknowledgements. In that case I will leave them in.

I definitely agree that you don't want boilerplate code. My goal is to reach a point where your radio choice is simply a parameter to the constructor. I might have to have the radio choice as a define in order to filter out unnecessary code, though.

As a disclaimer I should also mention that my C++ is a bit rusty, so I hope someone has time to review my changes at some point. Also, do not expect any significant results for many weeks to a few months.

hek

@kolaf

No worries. Don't tell me about rusty c++ knowledge. This is the first c++ program in 10+ years. But it´s slowly getting back... and why I left it for other languages long ago ;)

kolaf

Until I get everything set up and working I am committing stuff to a branch in my own repository. It is located here if anyone wants to see the progress :-).

https://github.com/kolaf/mysensors

Currently it is just a big mess...

Zeph

I've been lightly following the RF69 family of wireless sensors (jeenode (modified), moteino, anarduino) and noticed the appearance of the hardware agnostic radiohead library, so I'll be very interested in your progress.

MySensors and the underlying nRF library is fairly well optimized to take advantage of the nRF24l01+, eg: ESB. It will be interesting to see how the more hardware abstracted radiohead library works in this role.

If I understand correctly (always a question), the current MySensors library uses a hybrid between hardware link-level ack and end to end software ack. A packet from hub directly to leaf node uses ESB hardware for the ack. A packet from hub to repeater to leaf node involves "end to end" ack from the repeater to the hub, but ESB from the repeater to the leaf node. In other words, it's the next to last node in the sequence which handles network ack (and with a direct link with no relay, the "next to last node" is actually the originator). @hek and other more knowledgable folks can correct if this is wrong.

RadioHead would presumably do network acks from the leaf node back to the hub. (a more pure network ack, rather than a hybrid using ESB for the final link).

kolaf

As far as I have been able to figure out the RF69 radio uses link level acknowledgements, i.e. an acknowledgement per hop through the network. When using the RMesh manager which I am basing my port on (since this will handle everything related to routing for me) the initial send message function returns after the first of acknowledgement has been received, and the receive function of the final receiving end sends a link level acknowledgement.

When doing the port I am therefore leaving in the end-to-end acknowledgements and replacing anything related to the original RF24 acknowledgements.

Status update:
I have completed an initial rewrite of everything except the MQTTGateway code and I have reached a point where everything compiles correctly. This is with a hardcoded RF69 driver, which means that we do not get the radio agnostic version just yet. However, there is more to coding than just compiling, so I will now start the tedious task of getting this thing to actually run. I spent five minutes this morning testing it and nothing seemed to happen. I got one message from the gateway over the serial link looking something like this: "0;0;3;2;#@" where the two last symbols in the string were some weird ASCII characters. I was expecting to see some kind of initialisation message here as per the serial gateway source code.

I will continue testing and fixing over the next few days, I hope. Once I have everything running I will probably build a few sketches of my own before I start adding support for the other radio drivers.

Damme

@kolaf that sound to me as a sram overflow. There are ways to see in runtime how much free sram there is in runtime. I think there is a commented out version of it in the code..
last lines in mysensor.cpp

int MySensor::freeRam (void) {
  extern int __heap_start, *__brkval;
  int v;
  return (int) &v - (__brkval == 0 ? (int) &__heap_start : (int) __brkval);
}

kolaf

@Damme Thanks for your input. I have had a chance to debug a bit more this evening and it turned out that I was not initialising the radio correctly.

I now have a version which is working with the RF 69 radio soldered to my Arduinos. Currently it should be easy to change the radio driver by changing the driver that is initialised and changing the appropriate include files.

I have achieved a successful communication from a sensor implementation to the serial gateway implementation where it requests a node ID. This is using the RHMesh manager implementation which handles a dynamic mesh network in the background.

I have not tested return communication yet because am frankly not quite sure what the correct command would be to assign a node ID. I guess all figure this out tomorrow, unless someone cares to give me an example?

With the debug enabled the gateway and simple sensor (simply sending a message every five seconds) amounts to the following when building..

Gateway:

Sketch uses 22,674 bytes (70%) of program storage space. Maximum is 32,256 bytes.
Global variables use 1,424 bytes (69%) of dynamic memory, leaving 624 bytes for local variables. Maximum is 2,048 bytes.

Sensor:

Sketch uses 19,854 bytes (61%) of program storage space. Maximum is 32,256 bytes.
Global variables use 1,141 bytes (55%) of dynamic memory, leaving 907 bytes for local variables. Maximum is 2,048 bytes.

hek

Great, sounds like your doing real progress.

Zeph

@kolaf
How big is the flash and RAM footprint of an equivalently limited nRF24L01+ implementation? The RadioHead library sounds heavier.

As I'm understanding you, a node is using the RadioHead RF69 mesh network to send a request to the hub for a MySensors node id. Doesn't the mesh network have its own equivalent node id? How are you integrating those two? (eg: does your node use id 255 temporarily within the RadioHead addressing, then change it a hub assigned ID, or...?)

I'm not really clear enough about the logical layering of the two networks, and how you are mixing and matching the layers.

kolaf

@Damme @Zeph I took a quick stab at dumping the free run by uncommenting the serial.write command in debugPrint. I didn't get anything, so I will look into it later.

Anyway, I was finally able to send a node ID response from the serial gateway back to the sensor, and is responded with a bunch of interesting data :-). This means that every thing seems to work, and I think the implementation is in a state where I can actually use it to build sensors. The challenge now is to have the self-discipline to generalise this so that others may use it to ;).

Based on all your comments and worries about flash and RAM space, I have been thinking that maybe my approach was not the best. I have ripped out everything related to routing so that I rely completely on the RadioHead for anything related to the link and network level. This means that we get support for quite a complex dynamic sensor networks at the cost of reduced available memory for useful stuff. May be a hybrid approach would have been better, relying on the reliable datagram manager from the library and using the mysensors routing functionality. Perhaps this would yield a smaller footprint, I don't know.

As for node addressing, there is a function in the new radio library that allows me to set the node address. I can therefore use the existing node address discovery to set everything up. I've included the message sequence I get when performing this initialisation below. The first line is the node assignment sent from the gateway to the sensor. There is some extra debug garbage in the printout which I had to add when trying to get this to work.

255;255;4;0;3;1
0;0;3;9;send: 0-255 s=255,c=3,t=4,pt=0,l=1,st=302:1
0;0;3;9;sent: 0-255 s=255,c=3,t=4,pt=0,l=1,st=295:1
0;0;3;9;Available
0;0;3;9;read: 1 s=255,c=3,t=11,pt=0,l=11:Dust Sensor
1;255;3;11;Dust Sensor
0;0;3;9;Available
0;0;3;9;read: 1 s=255,c=3,t=12,pt=0,l=3:1.1
1;255;3;12;1.1
0;0;3;9;Available
0;0;3;9;read: 1 s=3,c=0,t=0,pt=0,l=15:1.4b1 (18848a2)
1;3;0;0;1.4b1 (18848a2)
0;0;3;9;Available
0;0;3;9;read: 1 s=3,c=1,t=16,pt=2,l=2:1
1;3;1;16;1
0;0;3;9;Available
0;0;3;9;read: 1 s=3,c=1,t=16,pt=2,l=2:1
1;3;1;16;1

I'm wondering since we now have a working prototype version, perhaps I could be allowed to push my branch to your repository? Of course we don't merge anything yet, but it will allow others more easy access to it for further testing. By only changing a few lines to account for different radios the current version should support most of the moteino and anarduino variations with our different radio chipsets, I think.

kolaf

I'm getting a bunch of errors when I compile the mqtt gateway source related to the initial list of constants. The error messages look like this:

D:\Home control\Arduino\libraries\MySensors\MyMQTT.cpp:14:15: error: variable 'broker' must be const in order to be put into read-only section by means of '__attribute__((progmem))'
 char broker[] PROGMEM = MQTT_BROKER_PREFIX;
               ^
D:\Home control\Arduino\libraries\MySensors\MyMQTT.cpp:16:12: error: variable 'S_0' must be const in order to be put into read-only section by means of '__attribute__((progmem))'
 char S_0[] PROGMEM = "Temperature"; //V_TEMP
            ^
D:\Home control\Arduino\libraries\MySensors\MyMQTT.cpp:17:12: error: variable 'S_1' must be const in order to be put into read-only section by means of '__attribute__((progmem))'
 char S_1[] PROGMEM = "Humidity"; //V_HUM

My current solution is to just delete these two files since I only care about the serial gateway, but it would be good to have everything working :-).

kolaf

I am also happy to report that I just tried to compile the secret knock sensor without any difficulties :-)

Sketch uses 22,074 bytes (68%) of program storage space. Maximum is 32,256 bytes.
Global variables use 1,241 bytes (60%) of dynamic memory, leaving 807 bytes for local variables. Maximum is 2,048 bytes.

Damme

@kolaf What version of arduino IDE do you use? you should really use 1.5+
(But I'll fix so it works in older version)

kolaf

@Damme said:

@kolaf What version of arduino IDE do you use? you should really use 1.5+
(But I'll fix so it works in older version)

1.5.7

kolaf

I have been thinking on how to extend the library to support different radios. As far as I can see there are two options.

Have a generic radio driver pointer (of the superclass type RHGenericDriver) and initialise this depending on an input parameter to the Gateway constructor. The trouble with this approach is that we have to static_cast every access to this variable to its appropriate subclass in order to access radio specific functions to set transmit power, et cetera. We also have to include a bunch of header files so that everything is available during runtime. This obviously increases footprint and complexity for the maintainer.
Use a define in the sensor application and Gateway application to determine which radio is to be used, e.g. "#define DRH_RF69" to select the radio I am using. With an include #ifdef directives in the header and my sensor source files to include only the necessary driver headers and choose the correct set of initialisation functions for the radio. Most of this would go into the setupRadio function.

I think that the second approach is the easiest to implement. It also requires a single addition to any client application, namely the correct define at the top of the configurations header file. We can leave this as a default radio value, and the knowledgeable user can change this if he uses a different chipset. Does this make sense?

hek

@kolaf

Tried to open a chat (see the icon at the bottom of the screen)

Yveaux

@kolaf said:

I think that the second approach is the easiest to implement

Agree. Pointers to instances makes more sense when radios have to be switched dynamically or when multiple radios have to be supported.
For this application having a #define in a configuration file makes perfect sense.

Yveaux

@kolaf said:

Sketch uses 22,074 bytes (68%) of program storage space. Maximum is 32,256 bytes.
Global variables use 1,241 bytes (60%) of dynamic memory, leaving 807 bytes for local variables. Maximum is 2,048 bytes.

These are very acceptable numbers IMHO.
No need to worry about flash/ram usage!

hek

@Yveaux said:

Pointers to instances makes more sense when radios have to be switched dynamically or when multiple radios have to be supported.

Yes, defines certainly make sense now! But it would be cool to have gateway-nodes between different radio models in the future ;)

Yveaux

@hek Maybe, but I have yet to see a valid usecase for this. You can always use 2 Arduino's running the default serial gateway and tie them together ;-)
I haven't looked into the radiohead interface yet (have to admit you guys got me curious, though) but isn't there simply one base class defining the interface and every radio derives from it? That would definately make things simple!

Porting MySensors to work with the RadioHead library

16

11.7k

11.2k

113.0k