Porting MySensors to work with the RadioHead library

[Yveaux]

@kolaf Please find a detailed explanation of the function here: http://jeelabs.org/2011/05/22/atmega-memory-use/

[kolaf]

I've turned off debug messages for the serial Gateway, and I now have 600 bytes free RAM when running. It should be even more on the sensor node, but I haven't checked. I hope this is sufficient to build relatively complex sensors without any problems.

I'm still having problems with the mesh code giving the very poor range, but I have now ruled out that this is caused by lack of memory. Maybe there is a bug in Radiohead? The saga continues...

[kolaf]

Time for a short status update:

I'm still working on trying to figure out why I get very poor radio range (10 cm) when using the automatic mesh functionality of Radiohead. I get no such issues when using simple point-to-point messages without the routing network layer.

Through trial and error I have narrowed it down to possibly a problem with the serial gateway code. I am able to communicate successfully between two implementations of a sensor using gw.send and gw.process, but as soon as I change the type from MySensors to MyGateway in one node things stop working.

I will be able to do more testing on this during the next few days. In the meantime, do you have any suggestions as to what can be different in MyGateway to stop things from working? The three things that come to my mind is its use of interrupts for inclusion mode, memory problems from increased functionality (unlikely), or possibly something to do with the serial communication.

Could you please explain to me what the purpose of inclusion mode is? I cannot seem to find any code where this mode actually does anything useful :-)

I also don't quite understand how the serial communication works. It appears to be some kind of event handler tied to the serial port, but where and how is this done? Could this have impact on the issues I'm seeing, perhaps it is blocking for some time (although I find that hard to believe)?

[hek]

@kolaf said:

Could you please explain to me what the purpose of inclusion mode is? I cannot seem to find any code where this mode actually does anything useful

The gateway is just indicating when inclusion mode is active by flashing the LEDs. You can also enable/disable this mode by sending a message to controller. Noting more is done on the gateway side.
The purpose is to prohibit unwanted device includes. You must actively enable the inclusion mode. And during this time all newly presented sensors will be added on the controller side.

It is of course optional to use this functionality from a controller implementers point of view. It was necessary to provide this functionality when implementing support for the Vera controller as it tends to restarts every time you add a new device programatically. And you wouldn't want that to happen just by presenting a new sensor in your radio network.

I doubt the interrupts would affect radio in this case. They are only triggered when pushing the inclusion button (attached at pin 3). Just check you didn't connect radio IRQ-pin to 3 :) which I did once with funny results.

I also don't quite understand how the serial communication works. It appears to be some kind of event handler tied to the serial port, but where and how is this done? Could this have impact on the issues I'm seeing, perhaps it is blocking for some time (although I find that hard to believe)?

Really doubt this is you problem.

Gateway is always assigned radio-address 0. Does this perhaps have a special meaning in RadioHead?

[kolaf]

The gateway address 0 is not a problem, when I communicate between two sensors directly they have the addresses 0 and 1 without issue.

For the moteino and anarduino the RF 69 radio is always connected to interrupt pin 2, so I agree with you that this should not be an issue.

I'm also considering that I might be doing something different in the constructor since the sensor and gateway uses different constructors, but I also find this hard to believe. All radio initialisation is done inside setUpRadio which is common for both classes.

Do you have a pointer to somewhere that explains the serial event handling?

[hek]

@kolaf said:

Do you have a pointer to somewhere that explains the serial event handling?

http://arduino.cc/en/Tutorial/SerialEvent

[kolaf]

@kolaf said:

Do you have a pointer to somewhere that explains the serial event handling?

Nevermind, I found it :-)

Regarding addresses, the only dress with a special meaning in Radiohead is 255 which is broadcast. This should not be an issue for my experiments since amusing fixed node IDs.

[kolaf]

I think I just found my problem!!

I was looking at the header file for MyGateway and I suddenly noticed that I have a bogus default value for transmit power (-14 instead of 14). This explains nicely why my range suddenly gets limited when running the Gateway. Can't wait until I get home to test this :-).

Still, experience shows it is never as easy as you think, so I won't know until tonight.

[Zeph]

OT but ... what's the point of SerialEvent()?

From the description it's called between loop() calls if there is serial input data, and you should check for more than one byte of data.

void loop() {
      // do stuff
}

void serialEvent() {
    while (Serial.available()) {
        char inChar = (char)Serial.read(); 
        // use it
    }
}

So wouldn't adding this to the end (or start) of loop do the same thing?

void loop() {
      // do stuff

      while (Serial.available()) {
         char inChar = (char)Serial.read(); 
         // use it
      }
}

Except this would work on the Leonardo etc, so there must be advantage of serialEvent that I'm missing.

[Damme]

@Zeph arduino specific.. http://www.seeedstudio.com/wiki/Where_is_Main_Function

[kolaf]

Good news everyone!

My hunch was correct and I now finally have the MySensors library working correctly through my entire house (one hop covering 20 m through two timber walls) using the mesh network manager from Radiohead. I have not tried multiple hops yet, but I have no reason to believe that that should not work since I'm using the standard Radiohead library (famous last words?).

I have pushed my current progress of the fully working version to my fork of mysensors. My ongoing task now is to merge in the current development branch and deal with all the conflicts. There are a lot of conflicts since I have changed a lot of functions signatures in my code. The next task is to try to generalise everything so that it works correctly with the radio you guys use with some decent defaults. My plan as we have discussed earlier is to have a minimal constructor and initialisation routine covers the basic functionality for every radio we want to support and exports the radio driver from the MySensors object so that power users can access this directly and tweak the settings for the radio if they want to.

I have my first my sense and node working with the new code :-). It is a simple distance sensor with a 10 cm limit which gives a binary output detected/not detected. I use this to detect a hand waving in front of the sensor and send a message to the gateway every time a hand is detected and disappears again. This is piped through a modified version of the PC mqtt Gateway I have borrowed from zephy and modified to support communication over the serial port (not just sockets). This again is connected to openhab where I use a rule based on what has been described in another thread to toggle a relay on and off and control my light :-).

When I enter my room, wave my hand, light on. When I leave my room, wave my hand, light off. Unfortunately I have to have this powered by the mains power system since I need it on all the time to have the response I want from the wave detector. If I sleep I might miss a wave, which means I will have to stand and wave for several seconds for something to happen.

[Yveaux]

@kolaf said:

My plan as we have discussed earlier is to have a minimal constructor and initialisation routine covers the basic functionality for every radio we want to support and exports the radio driver from the MySensors object so that power users can access this directly and tweak the settings for the radio if they want to.

On a second thought, I prefer to not construct the radio instance in the MySensors library and expose its ptr for 'power users', but to create the radio instance from the sketch and pass it along with the MySensors c'tor or begin-method.
This is in line with how the RadioHead library handles radios and more flexible when e.g. support for new radios is added to the RadioHead library (the MySensors library does not have to be modified then).
I also don't like all the #ifdef's in the MySensors library for all the different radio types. This just pollutes the code...

As an example I wrote down how the radio instantiation and passing it to MySensors could look like in a sketch:

#include <SPI.h>
#include <RH_NRF24.h>
#include <MySensor.h>  

RH_NRF24 nrf24;
MySensor gw;

void setup() 
{
  nrf24.init();
  // I'm a power user, so let's change the channel ;-)
  nrf24.setChannel(1);
  // Start MySensors and pass it the radio to use
  gw.begin( nrf24 );
}

I think this notation is clean and easy to understand for novice users.
How about it?

modified version of the PC mqtt Gateway I have borrowed from zephy

From Yveaux, I suppose?

[kolaf]

@Yveaux said:

I think this notation is clean and easy to understand for novice users.
How about it?

I support that wholeheartedly. The reason for my initial suggestion was the desire from the library side to be easy to initialise. Having the user initialise the radio separately removes a bunch of configuration complexity from the radio set up. The only trouble I foresee is that the typically are some standard commands required to initialise the different radios property. For instance, for my radio I have to set the frequency and transmit power. This will be identical for every sensor I will build, so it would be good to have some kind of default initialisation for the radios. I don't know what is required to set up the RF 24 radios (or was it RF 22)?

modified version of the PC mqtt Gateway I have borrowed from zephy

From Yveaux, I suppose?

Sorry, I just remembered that it was a username with a weird combination of letters ;)

[Yveaux]

@kolaf said:

a weird combination of letters

It's French :)
Kinda....

[Yveaux]

@kolaf said:

The only trouble I foresee is that the typically are some standard commands required to initialise the different radios property. For instance, for my radio I have to set the frequency and transmit power. This will be identical for every sensor I will build, so it would be

We should choose & define some sane defaults (like the current defaults).
I think most of the users will just continue using the nRF24's, so this radio can be used in the examples.

It will make the transtion rather painless, IMHO

[kolaf]

The trouble is that it cannot be solved just by using defines defaults. For my case I have to explicitly call

driver.setTXPower(14)
driver.setFrequency(868)

Every time I initialise the driver. This is a pain. Perhaps we should have a separate initialisation routine that enters default values for each driver which is called in the constructor? Or maybe we call the driver specific initialisation on the object after we have constructed it?

MyGateway gw(driver);
gw.initialise_RF69()
gw.begin();

I must admit that this does not seem very elegant...

[Yveaux]

@kolaf From my head, you're using the RF69 driver, right?
Well, this applies to any driver....
Looking in RH_RF69.cpp, RH_RF69::init(), it explicitly sets the TxPower to 13 as last statement of the init procedure.
If this default is changed into a #define which can e.g. be overwritten with your own defaults we're also done, e.g.

In e.g. RH_RF69.cpp

#ifndef RF69_DEFAULT_TXPOWER
#define RF69_DEFAULT_TXPOWER (13)
#endif

and then ofcourse in RH_RF69::init():

    setTxPower(RF69_DEFAULT_TXPOWER); 

If we now allow a file with our own MySensors defaults to be included before the default definition of RF69_DEFAULT_TXPOWER, it has precedence over the RadioHead's defaults and we're done.

This requires modifying RadioHead a little, but I can't imagine the maintainers would have a problem with implementing a mechanism for compile-time configurable default values.

[Yveaux]

I just built my first nRF24L01+ setup using the RadioHead library running the nrf24_reliable_datagram_client and nrf24_reliable_datagram_server sketches on two Uno's.

Works like a charm, after I disconnected the interrupt line to the nRF24.
Don't know why yet (possibly an interrupt handler is 'secretly' installed by the library), but it took me some time to figure out...

Also requires the driver to be constructed as RH_NRF24 driver(9, 10) when using the MySensors connection-scheme.

[kolaf]

Great to hear, I'm looking forward to seeing whether my implementation will work with your radio and whether there is any benefit from it, or just a big cost. I know that from my radio the radio library needs to know the interrupt pin, so definitely installs an interrupt handler. I shouldn't think it would do that for every driver, but I might very well be wrong. Let me know if you need any help getting my work in progress to work for you, though I suspect that you are knowledgeable enough to manage it yourself :-)

I agree that your solution to the configurations pretty elegant, and in fact for my specific case I don't think we need to change the defaults since the only reason I have to do my own initialisation is because I have the high-powered version. It makes sense to leave the RF 69 driver defaults as is, and I will just have to deal with it :-)

Still, I am a bit reluctant to make changes to the Radiohead library. The major reason is of course that it will be difficult to upgrade the library as new releases are published (which they are quite often, apparently). The new releases are only distributed using zip files as far as I can see, which means that we would need a manual merge job every time we want to upgrade the library. It would be easier if they had a git repository for the driver, then we could to a large extent handle merging in new versions automatically. I guess I could check on their Google group whether this is an option. Doing a manual merge is of course also possible, and maybe it turns out that this is the best option after all.

Another option which is sort of a compromise is to have the main constructor detect which type of driver it is supplied with. I guess it is possible to do some kind of type checking to figure out which class it is an instance of. We can then build our own default initialisation routines (by all means based on defined values) which are called by the constructor automatically without the need for user interaction. The trouble is, obviously, but this might overwrite whatever initialisation is you did yourself before instantiating the class. I guess this can be sold with an additional parameter to turn this on and off, but then it becomes messy again.

[Yveaux]

@kolaf said:

Another option which is sort of a compromise is to have the main constructor detect which type of driver it is supplied with.

I'm afraid this isn't an option as the linker will have to link code for all radios in then, got every radio configuration....