Porting MySensors to work with the RadioHead library

[ToSa]

@Yveaux said:

So, anyone still developing on RadioHead integration, or did all of you just give up?

no progress on my end - but that's rather related to the fact that this was my first week back in the office after vacation...

I think we should do one thing first: decouple the "network layer" from the "application layer" in MySensors:

• not having MySensors derived from RF24
• have a "generic" interface between MySensors and the radio

That way it would be rather easy to make use of other radios and other network topologies - could be RH or something else. I've taken a shot at that (a lot of this based on what Kolaf and Yveaux did adjusting MySensors to RH) and will provide a link once it compiles and is tested and I hope I can keep the overhead small enough to be acceptable for the gained flexibility.

[kolaf]

It is great to see that something is still happening. I wonder if I can build a packet sniffer for the RF69 radios? I guess it shouldn't be that difficult since they receive everything that is sent...

The reason for my lack of progress late the has several reasons.

1. I have a version of the library networks for my simplifications. This means that I can start building small sensors instead of spending all my time on the library
2. My kids think I spent way too much time on this project already
3. I'm waiting for some kind of consensus to emerge.

My hope has been to get this new radio library thing to a point where the community would adopt it so that I would not have to do any specific maintenance to keep my copy up to date with the official version. Based on this very long discussion thread there seems to be interest among you guys to achieve this, so I hope we will be able to get there at some point. However, we need to agree on how it is to be done, how much of the Radiohead library we want to use (or maybe even use a different library?). This obviously depends mostly on resource constraints considerations, and I'm not the best to judge this since I have not used MySensors extensively.

It is great to see effort being spent on trying to reduce the footprint, and as an experiment it makes sense to do this. Once we have seen how small it is possible to make it, then we can perhaps agree on what features we want (everything, or just the device interface, maybe with RHDatagram manager class (without acknowledgements)).

I'm holding back on developing any more this until we have reached some kind of consensus since I do not really have time to play around too much without being a relatively sure that the results will be useful.

[ToSa]

@ToSa said:

I think we should do one thing first: decouple the "network layer" from the "application layer" in MySensors:

• not having MySensors derived from RF24
• have a "generic" interface between MySensors and the radio

Initial version - tested with a simple DallasTemperatureSensor setup successfully here
Note that this fork/branch does not include the "1.4 update 1" changes for nodeID / CRLF.

Overhead is limited (~150b flash / ~20b ram):
Sketch uses 20,426 bytes (66%) of program storage space. Maximum is 30,720 bytes.
Global variables use 575 bytes (28%) of dynamic memory, leaving 1,473 bytes for local variables. Maximum is 2,048 bytes.

[kolaf]

Care to share any details on which parts you took from where? Is it basically the Radiohead drivers with the MySensors network layer? Or did you make more effort on trying to split everything up to a larger extent

[ToSa]

@kolaf it's only splitting the MySensor application layer form the underlying network layer. Still using the MySensor tree network topology and the RF24 driver. It's adding the framework to allow others to easily integrate other topologies (e.g. mesh) and other radio modules (e.g. RF69). The interface expects a reliable submission - no matter if that's achieved by software (like radiohead) or hardware (nRF24).
The MyDriver header file includes a short description of the routines and it should be fairly easy to attach it to radiohead. The main benefit is that the original MySensor tree setup still works as before if wished.

[hek]

@ToSa said:

Initial version - tested with a simple DallasTemperatureSensor setup successfully here
Note that this fork/branch does not include the "1.4 update 1" changes for nodeID / CRLF.

Looks good @ToSa!

[kolaf]

@ToSa Could you give me the link to your repository?

[ToSa]

@kolaf click the "here" in my post above - or use this:

https://github.com/ToSa27/Arduino/commit/328757d5bdb2681257326761fe1a0d90cdc3eba3

[kolaf]

@ToSa Got it. My apologies for being slow, it was getting late ;).

Itlooks like this should be quite easy to work with. For the RF69 driver, maybe I could even use the unreliable datagram service from Radiohead And Configure the Driver to Use Hardware Acknowledgements (I Believe They Exist).

I will make an effort in this direction during the next few days, I hope. I'm curious to see how much space this requires compared to the NRF24 solution

[kolaf]

@ToSa I was trying to fork your repository in order to build the driver for the RF69 based on Radiohead. It does not seem to work, maybe because I already have a fork of the main mysensors repository?

Maybe I can just check out your code, add the necessary files and be allowed to push it back to you?

[ToSa]

@kolaf said:

maybe because I already have a fork of the main mysensors repository?

yes - Git doesn't allow two forks with the same base name (Arduino) for one user. We are about to move the changes into the mysensors/development branch. That way you can pick it up from there...

[kolaf]

@ToSa Great, I will add my changes when you have done this.

[kolaf]

It has been implemented, but I won't bother testing it until your changes have been merged with development, I don't want to mess up my existing development environment. My biggest uncertainty is whether this set up will allow the Radiohead library to be installed as a regular library in the "libraries" folder or if it has to be put directly into the MySensors folder.

[kolaf]

Is there any progress of merging into the development branch? I'm currently holding off of building anything definitive for in-house deployment until we have settled on a relatively stable base for the code.

On a side note, I'm currently deploying Z wave actuators for all the lights in my house (we're refurbishing the entire electric system, anyway), and I must say that I am not very happy with the closed nature of the Z wave network management. I have gotten to the point where I believe that if I were to replace all the Z wave components with MySensors components I would get a much more stable system. Unfortunately I will get in trouble with the authorities if I try this

[Francois]

@hek It looks to me that all the activity that happen on getting the RFM69 integrated as part of the Mysensor ecosystem is dying. Can we not see if we can talk to Felix from Lowpower (https://lowpowerlab.com/) to see if he is not willing to get his Moteino integters as part of the MySensor ecosystem. Here is the tread on his forum where I asked him about this. https://lowpowerlab.com/forum/index.php/topic,542.0.html

[hek]

The integration has not stopped. It is just a bit slow as we all have lots on our plate

Felix has actually sent me a couple of Moteinos that arrived a few days ago.

But the first step is to separate radio code from the rest of the library. @ToSa has begun this work and is nearly finished. Guess he is swamped with work as I haven't heard from him the last couple of weeks.
https://github.com/mysensors/Arduino/pull/36

Then we can create a new driver for the RFM69.

[kolaf]

I am waiting for the new radio separation code to be merged into the development branch. I have a working implementation (I think) of the RadioHead RFM69 library using his new code. I don't mind the wait, though, because I have lots of other things to do in the meantime and I can still build sensors using my own version of the library

[hek]

Super @kolaf! Can't seem to get in contact with @ToSa. Maybe time to take it in (even if the config thing is unresolved) to get some progress.

[hek]

@kolaf

The driver refactoring has now been merged into development. Ready for RFM69!

[kolaf]

Working on it, fighting git about a rolled back rebase...

[kolaf]

I managed to resolve my git problem, but I'm now back to a problem we have discussed before.

The problem is with how I should include the driver in order to use it without having to modify the header of every driver file. Can't seem to find what we landed on last time...

[hek]

You mean every sketch file?

Guess you'll have to do like we did with rf24 and add it to the utility folder. Or maybe I misunderstand the question.

[kolaf]

I mean that I have to place the RadioHead library inside the main my sensors folder for everything to work. If I were to place it in the "utility" folder, I would have to change every include inside the library to use" instead of brackets.

Anyway, I have gotten it to compile, and there is some radio communication going on. Trouble is that there seems to be some issues with the buffer sizes (I guess?).

Here was what the sensor is sending:
find parent
send: 1-1-255-255 s=255,c=3,t=7,pt=0,l=0,st=bc:

And this is what the Gateway is receiving (with an additional debug message from me):
0;0;3;0;9;read: 0-0-0 s=255,c=3,t=6,pt=1,l=1:0
0;0;3;0;9;version: 0
0;0;3;0;9;version mismatch

Obviously something is getting mangled along the way...

[hek]

Good that something is transmitted.

But should you really use RadioHead? That seems a bit redundant.

Felix over at LowPowerLab directed me to: https://github.com/LowPowerLab/RFM69/

[kolaf]

I got something working with a new library. Requesting node ID, and sending all initialisation stuff seems to work correctly. I was not able to place the libraries in the correct place this time either, but perhaps you can move them and re-factor the code?

I created a pulled request so that you can look at it.

This is happening in the gateway with the new code:
0;0;3;0;14;Gateway startup complete.
0;0;3;0;9;read: 1-1-255 s=255,c=3,t=7,pt=0,l=0:
0;0;3;0;9;send: 0-0-1-1 s=255,c=3,t=8,pt=1,l=1,st=ok:0
0;0;3;0;9;read: 0-1-1 s=255,c=3,t=8,pt=1,l=1:0
0;0;3;0;9;send: 0-0-0-1 s=255,c=3,t=8,pt=1,l=1,st=fail:0
0;0;3;0;9;read: 1-1-0 s=255,c=3,t=6,pt=1,l=1:0
1;255;3;0;6;0
0;0;3;0;9;read: 1-1-0 s=255,c=3,t=11,pt=0,l=13:Wave switch 3
1;255;3;0;11;Wave switch 3
0;0;3;0;9;read: 1-1-0 s=255,c=3,t=12,pt=0,l=3:1.1
1;255;3;0;12;1.1
0;0;3;0;9;read: 1-1-0 s=1,c=0,t=3,pt=0,l=3:1.4
1;1;0;0;3;1.4
0;0;3;0;9;read: 1-1-0 s=3,c=0,t=6,pt=0,l=3:1.4
1;3;0;0;6;1.4
0;0;3;0;9;read: 1-1-0 s=2,c=2,t=2,pt=0,l=3:1.4
1;2;2;0;2;1.4
0;0;3;0;9;read: 1-1-0 s=1,c=1,t=2,pt=2,l=2:0
1;1;1;0;2;0
0;0;3;0;9;read: 1-1-0 s=3,c=1,t=0,pt=7,l=5:23.2
1;3;1;0;0;23.2
0;0;3;0;9;read: 1-1-0 s=3,c=1,t=0,pt=7,l=5:23.3
1;3;1;0;0;23.3

[hek]

Great @kolaf. I'll take a look.

[hek]

Ok merged! Seems to compile fine with the library in utility after some minimal changes. Thanks @kolaf.

[kolaf]

Excellent. I took a quick look at the changes you did to move the library, I'm a bit ashamed that I didn't manage to do that myself...

I'm very happy to have a solution in the development branch that allows the radios I bought to be brought along into the future of MySensors

[bbbio24]

Since I have four moteino , I wanted to test your code. Unfortunately, I have made ​​a mistake , because the compiler return this error :

In file included from My_Sensor.ino:2:
C:\Program Files (x86)\Arduino\libraries\MySensors/MySensor.h:32:1: warning: "debug" redefined
In file included from C:\Program Files (x86)\Arduino\libraries\MySensors/MySensor.h:16,
from My_Sensor.ino:2:
C:\Program Files (x86)\Arduino\libraries\MySensors/MyDriver.h:9:1: warning: this is the location of the previous definition
In file included from C:\Program Files (x86)\Arduino\libraries\MySensors/MyConfig.h:27,
from C:\Program Files (x86)\Arduino\libraries\MySensors/MySensor.h:17,
from My_Sensor.ino:2:
C:\Program Files (x86)\Arduino\libraries\MySensors/MyDriverRF69.h:40: error: ISO C++ forbids initialization of member 'radio'
C:\Program Files (x86)\Arduino\libraries\MySensors/MyDriverRF69.h:40: error: making 'radio' static
C:\Program Files (x86)\Arduino\libraries\MySensors/MyDriverRF69.h:40: error: invalid in-class initialization of static data member of non-integral type 'RFM69*'

[hek]

@bbbio24

IDE version?

[Francois]

This post is deleted!

[Francois]

This post is deleted!

[Francois]

Hi I now have my first MySensor RFM69 node up and running in PiDome This is great as with the NFR24L01 I battle to get coverage in my House. I put the RFm69 node in the farthest point in my house any it just work. To get the NFR24L01+ to work at the same location I have to use an repeater node. The other thing I like about the RFM69 solution like Moteino (https://lowpowerlab.com) or Anarduino (http://www.anarduino.com/miniwireless) is the size you don't have to use any wires to connected the radio and it comes fitted to the arduino board. So if you look for a small sensor I think that this is the way to go for automation.

[Anticimex]

@Francois Cool! Is this compatible with RF24, as in, you can have RF24 based GW and other sensors, and "plug in" a RFM69 unit? With its better RF properties it sounds like an excellent candidate for use as repeater node as well.
Or is it required to set up a separate GW?

[hek]

You cannot mix radios in one network.

[Anticimex]

@hek ok, så it is a different carrier frequency, or modulation, and not protocol-compatible on the "physical" layer? Pity.

[bbbio24]

@hek IDE Version : 1.0.5 r2
I tried with version 1.0.6 without success.

[ToSa]

@kolaf
I've been busy with other stuff for some time but looking at the driver separation now once again. With nRF24 working (even bootloader adjusted to pick the right header files etc.) and two Moteinos at hand (Moteino RFM69W 868/915), I compiled and flashed a serial gateway and a simple temperature sensor node. Both appear to work fine looking at the debug messages except the fact that they don't listen to eachother. Both are sending but none is receiving anything (2m distance).
I did not touch or even closely look at the RFM69 driver code yet but will try to do so over the weekend. Do you have any hint where to check first?

[ToSa]

I have the RFM69 setup working now with a couple of additional tweaks mainly for code optimization. Working fine for some time but stops after a few hours - I'll have a closer look the next couple of days.
Felix added three bytes to the payload (source address / destination address / control byte) to manage the acknowledge etc. in the lowpowerlab library. For our specific needs this is redundant as the current message format in development already includes this data. The three additional bytes are not that bad though and I'd propose to leave it like that for now and reconsider when we get to the revised message format :).

[blacey]

@ToSa

[ToSa]

RFM69 is stable now for >48h.
The OTA bootloader version for RFM69 is in progress. I'm stuck right now where the node debug messages show that it's sending a packet but the gateway doesn't receive anything... I moved the full initialization code back in that I thought would be ok to skip but still the same behavior. Only difference to the full blown driver at this point is that I don't attach the interrupt but that should not be relevant for tx as I'm polling the interrupt register for status (ready to send / send complete etc.).

[ToSa]

I had some time today to look at the bootloader code for RFM69 modules. Got the bootloader code to a point where it appears to be working fine - but the instability of the full MySensors RFM69 code is back - the serial gateway stops receiving packets after some time and it appears that the attempt to download a firmware by the bootloader just makes this problem more obvious due to the high volume of packets...
My prime suspect at the moment is the interrupt driven RFM69 library using a single packet buffer. As long is it's used sequentially everything should be fine but with e.g. submission or retrieval of ack messages it opens up the transceiver while the last packet was not fully processed yet potentially causing this... I'll go ahead and replace the routines in the library with the non-interrupt-driven version I coded for the bootloader just to see if this is the right place to dig deeper...

[CARSTEN]

@bbbio24 Hi
I am running into the same " ISO C++ forbids initialization of member 'radio'" error. Did you manage to solve this issue? am using IDE version 1.0.6. What version should I use?

Update: I managed to get past this error.
By the way I am woring with the latest develpment branch that I downloaded from here https://github.com/mysensors/Arduino/archive/development.zip.
To get past this problem I changed in MyDriverRF69.h the lne 40 to
RFM69 *radio;// = NULL;
This resolved the problem in the arduino IDE 1.0.6. Admittedly I am very new to this whole thing and am not sure what I potentially broe by changing the line bt at least I can explore further.

[CARSTEN]

I used the Mysensors 1.4.1 library and managed to get a UNO with Ethernet shield and NRF24L01+PA+LNA working as an MQTT gateway.
I then changed the library to the latest development library (downloaded on 01.12.2014) and uploaded the MQTT gateway setch to the same hardware -> Also worked fine.
I then changed MyConfig.h to use the RF69 radio instead of NRF24 and just tried to compile. I got a lot of errors:
MQTT_Gateway_Compile_errors.txt

I am using Arduino IDE 1.5.8

I am happy to help with testing, but I am afraid my programming skills are not good enough to assist with that.

[Francois]

@ToSa do you see the RFM69 only work on the serial gateway or will it also work on the MQTT gateway?

[kolaf]

@TOSA I just popped back into the forum to report on my experiences with instabilities using the current version of the RFM69 code only to find that you have already had the same experience. This makes me quite happy since it means that at least someone else have had a chance to think a bit about it. On a side note, I really wish this forum had some functionality to subscribe to topics, it is not a forum I visit regularly enough to catch everything that's happening.

I do not have any good information regarding why the microcontrollers stop responding, but I see it quite regularly in both the Gateway and in the sensors. For instance, I have a power measuring sensor that counts the blinks on my power meter and toggles a LED each time the power meter blinks. I have to reset this about every two days (sometimes more often) when I see that the LED no longer blinks. All my current sensors are running the first version of the code based on your rewritten radio obstructions.

Please let me know if there's anything I can do to help out with debugging this. The instability is starting to get quite frustrating, and I was actually considering to go back to my earlier prototype implementation based on the radiohead library. I did not experience any such instabilities using this library, so at least we know it is not a hardware problem. As such it shouldn't be too difficult to figure out. Maybe it is possible to discuss this with the lowpowerlabs guy who developed the library?

Unfortunately my experience with debugging microcontrollers is limited, but I will help out where I can.

[Francois]

@ToSa I am running this version and I don't have any problems with it so far. I only have one node up and running with a humility sensor. I order 5 more Anarduino units and hope to get them in Jan so will do more testing then. The humility node is up and running now for more than 3 weeks with out any resets.

[kolaf]

@ToSa How did you make the RF69 stable?

[nagelc]

@CARSTEN I got the same compile errors. It turns out Arduino IDE 1.5.7 and above have a new compiler which requires PROGMEM data to be const.
See this discussion: http://forum.arduino.cc/index.php?topic=277173.0

I was able to resolve the compile errors by adding const where flagged by the compiler:
In MyMQTT.h:
. . . .
const char V_0[] PROGMEM = "TEMP"; //V_TEMP
etc . ..

Then:
const char * const vType[] PROGMEM = {
V_0, V_1, . . . .

You also need to change the getType function in MQTTGateway.ino to account for the change in type of vType[]
char *getType(char *b, const char * const *index) { . . . .

Compiled with no errors for RFM69 on Arduino 1.5.8 after these changes.
I have not had a chance to upload and test on my Motineo's.

[guillaume brucker]

Hi all,

Happy new year
I've just have the same idea, I'm new to arduino since chrismas
and I want to test mysensor with nrf905, It seem to be compatible with RH librairies.

Could you just say me what i have to do ?
I'll have to include the kolaf librairies ?

Just, If you have a link to the repository in order to work on it with you

Thanks for all

[guillaume brucker]

So, I'm working on it with the developpement branch of mysensor.

I've find some nrf905 librairies arround the web, but the RadioHead seem to be more powerfull and complete.
I'll give you news soon,

First of all, i probably need help to diagnose because i've had only one raspberry, one arduino and two atmega328p chipset more

[andrewm844]

Did anyone ever solve the this compile error?

In file included from /home/andrew/sketchbook/libraries/MySensors/MyConfig.h:27,
from /home/andrew/sketchbook/libraries/MySensors/MySensor.h:17,
from SerialGateway.ino:27:
/home/andrew/sketchbook/libraries/MySensors/MyDriverRF69.h:40: error: ISO C++ forbids initialization of member ‘radio’
/home/andrew/sketchbook/libraries/MySensors/MyDriverRF69.h:40: error: making ‘radio’ static
/home/andrew/sketchbook/libraries/MySensors/MyDriverRF69.h:40: error: invalid in-class initialization of static data member of non-integral type ‘RFM69*’

I have tried with IDEs 1.04, 1.05 and 1.06

[guillaume brucker]

@andrewm844
Hi,
I'm working on 905, so I don't have to check the 69, but I've the same compile error
I just replace
// RFM69 *radio = NULL;
RFM69 *radio;
in the MyDriverRF69.h file

[BenjaminK87]

Hi everyone,
i am new to MySensors. I already made sime Arduino pro mini running with a NRF24L01.
But i also bought some Anarduino with the RF69 installed. Can anybody tell me how i can tell the library to use the RF69 instead of the NRF24L01?
I used the Developement version of the MySensor library. I tried to find the point in MyConfig.h but i could only find the NRF24L01 Driver Defaults part.

Maybe someone could help me.

Thanks in advance.
Benjamin

[lafleur]

I'm interested in the progress of getting RadioHead working with MySensor....

what the current status??

where is the current code for this effort?

Thanks

[hek]

@lafleur

In the development branch on github we have support for RF69. We have not incorporated the full RadioHead library.

[lafleur]

Is there support for the MoteinoMega? with the rfm69? can you point me to a code base to start with??

I have used the MoteinoMega in other project, so I know a bit on how they work

Thanks

[lafleur]

@ToSa do you have a sample project (serial Gateway or ?) that I could start from that uses the RF69W radio?? Want to develop with Moteino-Mega

thanks

[hek]

You only need to initiate the MySensor class with the rfm69 class like this in the gateway (and sensors):

https://github.com/mysensors/Arduino/blob/development/libraries/MySensors/examples/SerialGateway/SerialGateway.ino#L44

MyTransportRF69 radio;  

[lafleur]

I had done that, getting error...

Some questions:

I notice that the MyTransportNRF24 had definition for the pins in the function, RM69, has none...
I take it, its defaulted to the Moteino standards pinout?

The file MyHwATMega328.cpp is for the 328, i assume that I would need to make one for the ATmega1284P chip? if there are any changes needed.

Thanks I will work on these error, if you have any insight on cause, please let me know...

(sure wish the IDE was better.....)

---

Arduino: 1.6.1 (Mac OS X), Board: "MoteinoMEGA"

SerialGateway.ino:56:26: error: no matching function for call to 'MySensor::MySensor(MyTransportRF69 (&)(), MySigningNone&)'
SerialGateway.ino:56:26: note: candidates are:
In file included from SerialGateway.ino:34:0:
/Users/lafleur/Desktop/MySensor-Arduino-development/libraries/MySensors/MySensor.h:78:2: note: MySensor::MySensor(MyTransport&, MyHw&, MySigning&)
MySensor(MyTransport &radio =*new MyTransportNRF24(), MyHw &hw=*new MyHwDriver()
^
/Users/lafleur/Desktop/MySensor-Arduino-development/libraries/MySensors/MySensor.h:78:2: note: no known conversion for argument 1 from 'MyTransportRF69()' to 'MyTransport&'
/Users/lafleur/Desktop/MySensor-Arduino-development/libraries/MySensors/MySensor.h:69:7: note: MySensor::MySensor(const MySensor&)
class MySensor
^
/Users/lafleur/Desktop/MySensor-Arduino-development/libraries/MySensors/MySensor.h:69:7: note: candidate expects 1 argument, 2 provided
Error compiling.

[hek]

@lafleur

Yeah, the work on the TransportRF69 need some polishing. It defaults to monteino pins right now. If you have the time add configuration to the driver, please help out.

I don't have any MonteinoMega so I really don't know how it differs regarding hw. Hopefully not too much.

[lafleur]

I get the same error's if I select the standard Moteino and the RF24 radio???

so, I must have a version skew or IDE issue...

I'm using the source code form the development branch....

Arduino: 1.6.1 (Mac OS X), Board: "Moteino"

SerialGateway.ino:57:26: error: no matching function for call to 'MySensor::MySensor(MyTransportNRF24&, MySigningNone&)'
SerialGateway.ino:57:26: note: candidates are:
In file included from SerialGateway.ino:34:0:
/Users/lafleur/Desktop/MySensor-Arduino-development/libraries/MySensors/MySensor.h:78:2: note: MySensor::MySensor(MyTransport&, MyHw&, MySigning&)
MySensor(MyTransport &radio =*new MyTransportNRF24(), MyHw &hw=*new MyHwDriver()
^
/Users/lafleur/Desktop/MySensor-Arduino-development/libraries/MySensors/MySensor.h:78:2: note: no known conversion for argument 2 from 'MySigningNone' to 'MyHw&'
/Users/lafleur/Desktop/MySensor-Arduino-development/libraries/MySensors/MySensor.h:69:7: note: MySensor::MySensor(const MySensor&)
class MySensor
^
/Users/lafleur/Desktop/MySensor-Arduino-development/libraries/MySensors/MySensor.h:69:7: note: candidate expects 1 argument, 2 provided
Error compiling.

This report would have more information with
"Show verbose output during compilation"
enabled in File > Preferences.

[A Former User]

@lafleur:
I got the same error, also 1.6.1 (Mac OSX) board Uno, and Nano. My solution was to remove the signing object in the initialization:

// Construct MySensors library
MySensor gw(radio);

Then it compiles without issues. I have other issues though, but they might be related to the radios.

[lafleur]

removing the Construct object, will cause other error, as the gw is not defined at this point...

SerialGateway.ino: In function 'void setup()':
SerialGateway.ino:67:3: error: 'gw' was not declared in this scope
SerialGateway.ino: In function 'void loop()':
SerialGateway.ino:85:3: error: 'gw' was not declared in this scope

I'm new to Arduino and this IDE, so it painful trying to work my way into this code...

[kolaf]

Hi guys,

I lost some of my inspiration when I couldn't get the RF69 implementation to be stable for more than a few days and TOSA said he could before he disappeared without telling me how :-(. However, Easter is upon us and I thought I would give it another try.

I updated to the latest version of the development branch, and things seem to have changed a bit since I was here last time. Is there anything that needs to be updated on the RF69 side? At the very least it seems to be missing a definition for IS_RF69HW, as well as probably some useful defaults.

[kolaf]

I also note that there now is an include for a specific radio in the serial Gateway example. This is different from the way it was earlier where the radio type was defined in MyConfig. Wasn't this a better solution?

Anyway, is it sufficient simply to change the included radio and then expect the serial Gateway code to work correctly on a moteino?

[hek]

@kolaf

The defines in MyConfig started to become a big mess. Especially if we would want to have a Moteino variant of the gateways downloadable using codebender.

Ok, just updated gateway classes to include the available options like this:

// NRFRF24L01 radio driver (set low transmit power by default) 
MyTransportNRF24 transport(RF24_CE_PIN, RF24_CS_PIN, RF24_PA_LEVEL_GW);  
//MyTransportRF69 transport;

// Message signing driver (signer needed if MY_SIGNING_FEATURE is turned on in MyConfig.h)
//MySigningNone signer;
//MySigningAtsha204Soft signer;
//MySigningAtsha204 signer;

// Hardware profile 
MyHwATMega328 hw;

// Construct MySensors library (signer needed if MY_SIGNING_FEATURE is turned on in MyConfig.h)
MySensor gw(transport, hw /*, signer*/);

So you should only have to enable the one you want to use. Do the same in the sensor/actuator sketch (haven't updated all of them yet).

[dzairo]

.. and hardware distance test? rfm69hw have +20dB .. pls make test in building .. or make compare test with nRF24L01+ ..
we waiting for this ..
best regards..

[hek]

@dzairo said:

we waiting for this

I'm waiting also. Please do the test @dzairo!

[dzairo]

sorry .. is possible make test with this module if you have it ?
I can compare only nrf24l01+ with cc1101 868mhz ..
in free area is possible with standard nrf24l01+ with good antenna from wifi get 200m .. pcb antenna give less then 30m . rfm69hw is good because is possible set lowe transfer speed less then in nrf24l01+ ..
if make test with CC1101 868mhz version with +14dB output power then no problem get distance in my house 3 floor ..

best regards

[A Former User]

@lafleur:
I only removed the "signer" out of the initialization and changed the line to the code above. I did not comment the whole line.

@kolaf:
To make the library work for the RFM69HW for me, I needed to change the radio pointer to an object amongst other things.., which combination did eventually do the trick, I do not know for sure.
I will summarize the changes I did to get connection between two nodes using the RFM69HWs.

The problem I had was that the radio would not start up for me. It was stuck at the RFM69.cpp initialization at:

do writeReg(REG_SYNCVALUE1, 0xAA); while (readReg(REG_SYNCVALUE1) != 0xAA);
do writeReg(REG_SYNCVALUE1, 0x55); while (readReg(REG_SYNCVALUE1) != 0x55);

So the changes I did were:

In MyTransportRF69.h:

@@ -37,7 +37,7 @@ public:
        uint8_t receive(void* data);
        void powerDown();
 private:
-       RFM69 *radio;
+       RFM69 radio;
        uint8_t _address;
 };

and in MyTransportRF69.cpp:

 MyTransportRF69::MyTransportRF69()
        :
        MyTransport(),
-       radio()
+       radio(10,2,true,0)  //I set the Pins manually (SPI SS 10, InterruptPin 2, IsHW = true, and InternalInterrupt 0)
 {
 }
 void MyTransportRF69::init() {
        // Start up the radio library
-    radio->initialize(FREQUENCY,_address,NETWORKID);
+    radio.initialize(FREQUENCY,2,NETWORKID); //I changed the node_ids manually for each node, see below for comment
+    radio.rcCalibration();
+    Serial.println("Radio init done");
 #ifdef IS_RFM69HW
-    radio->setHighPower(); //uncomment only for RFM69HW!
+    radio.setHighPower(); //uncomment only for RFM69HW!
 #endif
-    // radio->encrypt(ENCRYPTKEY);
+    radio.setPowerLevel(20); //I decreased the powerlevel a bit, since I had some troubles transmitting between two nodes on my desk before when using the full power.

of course change all the following "radio->" to "radio." with string replace.

Also I realized that the NODEID is not set properly using the _address argument, I had a serial println there and it always returned "0 (zero)", which should not be correct. So I set them manually for each node.
I also copied the latest version of Felix LowPowerRFM library into utils from here:
https://github.com/LowPowerLab/RFM69
(I do not know if they differ much, the diff is not very conclusive due to different formatting)

I am using an Arduino Buono UNO clone set with a switch to run at 3.3V with the RFM69HW connected via breadboard. With the changes above, all was compiled and started up.

But I had a major problem with the Dev branch that in general that no ACKs at all were send or at least received, even when using the NRF2401+. So that is why I did not continue with the RFM69HW at the moment. The master branch on the very same setup (using the standard radio) did work without any troubles.

I managed to get connection between the nodes (e.g. in the serial output of the gateway the request for parent was received, but all messages had a "failed" status). I last checked the dev-branch last week.
I am away over Easter, but am willing to try a bit more afterwards, if I get the Dev branch running in general with ACKs...

I got inspired by this project:
http://www.instructables.com/id/Uber-Home-Automation-w-Arduino-Pi/?ALLSTEPS
There is also a forum here:
http://homeautomation.proboards.com/board/2/openhab-rfm69-based-arduino
but eventually I turned here, since mysensors seems to have more active support and a already a bigger community and codebase. However there they have a RFM69 gateway running with nodes. So it might be worth to check it out how they did it (that is also how I got the RFM69 running in the first place).

Sorry for the long post..

Cheers!

[hek]

@daenny

Thanks for the analysis. The RFM69 code really need a loving hand after being almost untouched for months since ToSa disappeared. There shouldn't be any changes done to the LowPowerLab code (but it hasn't been updated either).

The question is if hardware acks i coming in but is just picked up wrongly from LowPowerLab-library?

I should really setup a test rig with those radios... But my time is limited.. If you're interested in helping out finding the root cause for the RFM69 ack-problems I would be super grateful.

The ack-problems you're seeing on NRF24L01 will investigate by me.

[kolaf]

@daenny I've started playing with it now, and the first fix for your pointer problem is to initialise the radio pointer correctly which is not done in the current implementation. I simply added the following line to the beginning of init:

radio=new RFM69();

No further changes were then necessary to the pointers.

[kolaf]

I have two devices talking to each other, but I'm a bit rusty, so can't immediately see what's happening from the output. This is the sensor side:

req id
send: 3-3-0-0 s=255,c=3,t=3,pt=0,l=0,sg=0,st=fail:
send: 3-3-0-0 s=255,c=0,t=17,pt=0,l=5,sg=0,st=fail:1.4.1
send: 3-3-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=fail:0
sensor started, id=3, parent=0, distance=1
send: 3-3-0-0 s=255,c=3,t=11,pt=0,l=12,sg=0,st=fail:Power sensor
send: 3-3-0-0 s=255,c=3,t=12,pt=0,l=3,sg=0,st=fail:1.0
send: 3-3-0-0 s=1,c=0,t=13,pt=0,l=5,sg=0,st=fail:1.4.1
send: 3-3-0-0 s=2,c=0,t=6,pt=0,l=5,sg=0,st=fail:1.4.1
find parent
send: 3-3-255-255 s=255,c=3,t=7,pt=0,l=0,sg=0,st=bc:

And this is the gateway side:

0;0;3;0;9;gateway started, id=255, parent=0, distance=1
0;0;3;0;14;Gateway startup complete.
0;0;3;0;9;read: 3-3-255 s=255,c=3,t=7,pt=0,l=0,sg=0:
3;255;3;0;7;
0;0;3;0;9;read: 3-3-255 s=255,c=3,t=7,pt=0,l=0,sg=0:
3;255;3;0;7;
0;0;3;0;9;read: 3-3-255 s=255,c=3,t=7,pt=0,l=0,sg=0:
3;255;3;0;7;

Can anyone tell me what is happening here? I'm using a sensor I configured a long time ago, so it is already received its ID from the gateway previously.

[kolaf]

Okay, I have done some digging.

The gist of my last post was that it received no response for any of the messages sent directly to the Gateway, they never even arrived at the gateway. The only message that arrived was the broadcast message.

The first hints for me was the initialisation string from the gateway. It indicates that it is initialised with the address 255. This is a bit weird, since the Gateway source code provides a value of 0 to gw.begin(). Looking into the source for this it turns out that the value provided here is never stored in the node configuration, and thus is never used as the address for the node. I made some changes to store the address in the node configuration, and suddenly the Gateway started to respond to the messages from the sensor.

As for the acknowledgements, it turns out that ACKRequested never seems to return true even if we use sendWithRetries. If we assume this to be true and transmit acknowledgements regardless (which is correct since we always use send with retries) the status of all the send messages from the sensor revert to "ok".

I guess that this fixed to the acknowledgements issue is good enough for now, but I am definitely not sure about the issues with the Gateway node id. Any thoughts on this would be very welcome.

diff --git a/libraries/MySensors/MySensor.cpp b/libraries/MySensors/MySensor.cpp
index 91644e0..d5badb3 100755
--- a/libraries/MySensors/MySensor.cpp
+++ b/libraries/MySensors/MySensor.cpp
@@ -142,6 +142,13 @@ void MySensor::begin(void (*_msgCallback)(const MyMessage &), uint8_t _nodeId, b
 			// Try to fetch node-id from gateway
 			requestNodeId();
 		}
+	} else {
+				if (_nodeId != AUTO) {
+			// Set static id
+			nc.nodeId = _nodeId;
+			// Save static id in eeprom
+			hw_writeConfig(EEPROM_NODE_ID_ADDRESS, _nodeId);
+			}
 	}
 
 	setupNode();
diff --git a/libraries/MySensors/MyTransportRF69.cpp b/libraries/MySensors/MyTransportRF69.cpp
index b55ce14..3874773 100644
--- a/libraries/MySensors/MyTransportRF69.cpp
+++ b/libraries/MySensors/MyTransportRF69.cpp
@@ -29,7 +29,7 @@ uint8_t MyTransportRF69::getAddress() {
 
 bool MyTransportRF69::send(uint8_t to, const void* data, uint8_t len) {
 	// Make sure radio has powered up
-	return radio->sendWithRetry(to,data,len);
+	return radio->sendWithRetry(to,data,len,5,200);
 }
 
 bool MyTransportRF69::available(uint8_t *to) {
@@ -45,9 +45,13 @@ uint8_t MyTransportRF69::receive(void* data) {
 		// data[i]= (void)radio->DATA[i];
 	// }
 	memcpy(data,(const void *)radio->DATA, radio->DATALEN);
-	if (radio->ACKRequested())
+	if (radio->ACKRequested() || 1)
     {
+      Serial.println("Acknowledgement requested");
       radio->sendACK();
+      Serial.println("Sent acknowledgement");
+    } else {
+    	Serial.println("Not wanting acknowledgement");
     }
 	return radio->DATALEN;
 }

[hek]

@kolaf

Thanks! I will have a look at the id=0 problem tonight.

[A Former User]

@kolaf
Nice work, I am new completely new to the MySensors code, so I could not dig into it deep enough to see how everything is build up.
I completely overlooked that the pointer is not initialized properly, but your fix is definitively easier :).

@hek
My guess is that the problem with the ACKs and Gateway ID could be the same for the nrf radio, which might explain the troubles I had with the dev branch?

[kolaf]

@hek
Great. In the meantime I'm doing some stability testing of the current code base.

[hek]

@kolaf

[hek]

Ok, tried to clean up some of the mess in RFM69 (and gw node id).

I have a hard time to do any verification tonight here (the family sleeping too near the "lab room"). Would be great to get some feedback if you have your gears setup and ready for tests.

[reddy11]

@hek
Thank you for taking a look at the RFM69 code. I downloaded the development branch. While I was able to get it to compile, it does not seem to work, but Im note sure if it is something I am doing.
Background:
I had the RFM69 radio working with code from the development branch I downloaded in early Jan. I have a few sensors that and a serial gateway using the Jan code.

After downloading the latest development branch I tried to update my serial gateway, assuming it would still be able to read the sensors with the older code.

Observations:
Currently I cannot see any of the older nodes that I have. I have not tried to update any of the nodes yet, so I don't know if my problem is with the current code or not.
Currently there is not a myconfig.h option to set the RFM69HW option

I have been stumbing around the code trying to get a feel for it, and I am not and expert but am wondering if Line 137 of Mysensor.h has anything to do with the problem?

	MySensor(MyTransport &radio =*new MyTransportNRF24(), MyHw &hw=*new MyHwDriver()

Should there be something that would define MyTransportRFM69 instead?

Thoughts?

Would love to get this working and ultimately figure out the MQTT server options.

Thanks!

[hek]

Yes, it differs a bit. You can now initialise the transport using RFM69 like this (see constructor default values in https://github.com/mysensors/Arduino/blob/development/libraries/MySensors/MyTransportRFM69.h.
5:th argument is the one you're looking for. I Might add a default for this in MyConfig as well if this is something people change often.

#include <MyTransportRFM69.h>
#include <MySensor.h>

MyTransportRFM69 transport(freqBand, networkId,  slaveSelectPin, interruptPin, isRFM69HW, interruptNum);

MySensors gw(transport);

[kolaf]

@hek I updated with your latest changes and refreshed my node and gateway (which I am happy to report has run stable since yesterday). Things seem to work out of the box now which is good. There is a small issue, I think, where you could use the GATEWAY_ADDRESS a few more places for consistency, but apart from that I'm quite happy :-).

diff --git a/libraries/MySensors/MySensor.cpp b/libraries/MySensors/MySensor.cpp
index 70d3680..4a87e33 100755
--- a/libraries/MySensors/MySensor.cpp
+++ b/libraries/MySensors/MySensor.cpp
@@ -101,9 +101,9 @@ void MySensor::begin(void (*_msgCallback)(const MyMessage &), uint8_t _nodeId, b
 
 	if (isGateway) {
 		// Set configuration for gateway
-		nc.parentNodeId = 0;
+		nc.parentNodeId = GATEWAY_ADDRESS;
 		nc.distance = 0;
-		nc.nodeId = 0;
+		nc.nodeId = GATEWAY_ADDRESS;
 	} else {
 		// Read settings from eeprom
 		hw_readConfigBlock((void*)&nc, (void*)EEPROM_NODE_ID_ADDRESS, sizeof(NodeConfig));

[hek]

@kolaf
Sound good.

[A Former User]

@kolaf and @hek
Thanks for the great work. I will try to setup a test node and gateway by tomorrow and let it run over the Easter weekend. Hopefully it still runs when I am back

[kolaf]

@hek I tried setting up a second sensor (temperature and power supply voltage) with the updated library. I wiped eeprom, uploaded the sketch, and fired it up.

It requested a node ID from the gateway, which it received, and everything seemed to go smoothly until it had transmitted the first data point. After that it immediately enters a request for a new node ID, which it receives, and things go in an endless loop. Here is the log:

find parent
send: 255-255-255-255 s=255,c=3,t=7,pt=0,l=0,sg=0,st=bc:
read: 0-0-255 s=255,c=3,t=8,pt=1,l=1,sg=0:0
parent=0, d=1
req id
send: 255-255-0-0 s=255,c=3,t=3,pt=0,l=0,sg=0,st=ok:
read: 0-0-255 s=255,c=3,t=4,pt=0,l=1,sg=0:1
send: 1-1-0-0 s=255,c=0,t=17,pt=0,l=5,sg=0,st=ok:1.4.1
send: 1-1-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=ok:0
id=1
send: 1-1-0-0 s=255,c=0,t=17,pt=0,l=5,sg=0,st=ok:1.4.1
send: 1-1-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=ok:0
sensor started, id=1, parent=0, distance=1
send: 1-1-0-0 s=255,c=3,t=11,pt=0,l=20,sg=0,st=ok:Temperature basement
send: 1-1-0-0 s=255,c=3,t=12,pt=0,l=3,sg=0,st=ok:1.0
send: 1-1-0-0 s=1,c=0,t=6,pt=0,l=5,sg=0,st=ok:1.4.1
send: 1-1-0-0 s=2,c=0,t=13,pt=0,l=5,sg=0,st=ok:1.4.1
1023send: 1-1-0-0 s=2,c=1,t=38,pt=7,l=5,sg=0,st=ok:5.4
send: 1-1-0-0 s=255,c=3,t=0,pt=1,l=1,sg=0,st=ok:102
send: 1-1-0-0 s=1,c=1,t=0,pt=7,l=5,sg=0,st=ok:20.6
req id
send: 1-1-0-0 s=255,c=3,t=3,pt=0,l=0,sg=0,st=ok:
read: 0-0-1 s=255,c=3,t=4,pt=0,l=1,sg=0:3
send: 1-1-0-0 s=255,c=0,t=17,pt=0,l=5,sg=0,st=ok:1.4.1
send: 1-1-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=ok:0
sensor started, id=1, parent=0, distance=1
send: 1-1-0-0 s=255,c=3,t=11,pt=0,l=20,sg=0,st=ok:Temperature basement
send: 1-1-0-0 s=255,c=3,t=12,pt=0,l=3,sg=0,st=ok:1.0
send: 1-1-0-0 s=1,c=0,t=6,pt=0,l=5,sg=0,st=ok:1.4.1
send: 1-1-0-0 s=2,c=0,t=13,pt=0,l=5,sg=0,st=ok:1.4.1
955send: 1-1-0-0 s=2,c=1,t=38,pt=7,l=5,sg=0,st=ok:5.0
send: 1-1-0-0 s=255,c=3,t=0,pt=1,l=1,sg=0,st=ok:95
send: 1-1-0-0 s=1,c=1,t=0,pt=7,l=5,sg=0,st=ok:20.7
req id
send: 1-1-0-0 s=255,c=3,t=3,pt=0,l=0,sg=0,st=ok:
read: 0-0-1 s=255,c=3,t=4,pt=0,l=1,sg=0:3
send: 1-1-0-0 s=255,c=0,t=17,pt=0,l=5,sg=0,st=ok:1.4.1
send: 1-1-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=ok:0
sensor started, id=1, parent=0, distance=1
send: 1-1-0-0 s=255,c=3,t=11,pt=0,l=20,sg=0,st=ok:Temperature basement
send: 1-1-0-0 s=255,c=3,t=12,pt=0,l=3,sg=0,st=ok:1.0
send: 1-1-0-0 s=1,c=0,t=6,pt=0,l=5,sg=0,st=ok:1.4.1
send: 1-1-0-0 s=2,c=0,t=13,pt=0,l=5,sg=0,st=ok:1.4.1
956send: 1-1-0-0 s=2,c=1,t=38,pt=7,l=5,sg=0,st=ok:5.0
send: 1-1-0-0 s=255,c=3,t=0,pt=1,l=1,sg=0,st=ok:95
send: 1-1-0-0 s=1,c=1,t=0,pt=7,l=5,sg=0,st=ok:20.7

As far as I can see this should be caused by the node ID still being AUTO, but this shouldn't be the case since it appears to be correctly stored in ROM?

[kolaf]

Please disregard my previous "analysis"/guess. It turns out that the repeating behaviour that was caused by a program crash. The trigger for this crash is MyHwATMega328::sleep(ms) . I'm running a RFM69HW radio on a anarduino board, which should be using the ATMega 328 chip, same as the moteino.

By replacing the call to this function in MySensor::sleep(unsigned long ms) with a simple delay(ms), the crashes no longer occur.

[kolaf]

My current guess is that the watchdog timer resets the chip after the time period instead of throwing an interrupt to wake the system...?

[kolaf]

I narrowed it down even further. It appears that there is no appropriate interrupt and present to handle when the watchdog fires. I have no idea why this is the case, and I do not know if this fix is correct, but by a placing this code somewhere in MySensors.cpp everything works nicely:

ISR( WDT_vect ) {
    /* dummy */
}

[hek]

@kolaf

Ok, strange. Would be nice to know why this fixes the problem,

It should probably be placed here if it's atmega328 related (and only need to be called once):
https://github.com/mysensors/Arduino/blob/development/libraries/MySensors/MyHwATMega328.h#L22

[kolaf]

@hek
As far as I know the function is the interrupt routine that is called by the watchdog when the time expires. I guess it doesn't matter where it is defined so it should be fine to move it. It is not called explicitly anywhere in the code, but by the watchdog. As such, I don't think it should be defined as is done in the header file for the other functions.

My question is why is it not working for me without this code, I assume it has been working for everyone else?

[hek]

Ah.. doh..

Moved parts of the low-power library into MyHwATMega328 and forgot the ISR-thingie.

https://github.com/mysensors/Arduino/blob/master/libraries/MySensors/utility/LowPower.cpp#L850

Good catch.

[kolaf]

Glad I could help :-). I guess my prize will be that you patch this so that I do not have to create a pull request

[kolaf]

I might as well continue while I'm at it

Another weird issue I'm having is that every node requests a new ID whenever it starts, but it does not switch to the new ID it receives. It just continues with the old ID it already had.

When I wipe the ROM it requests a new ID as expected, and this time it is saved and used for subsequent communication. Why is it requesting a new ID when it boots, when it already has one?

[hek]

@kolaf

Should not happen. Will look at it later tonight.

[hek]

Ok, finally got the thumbs out and actually tested the RFM69 code using a couple of Moteinos. They seem to communicate just fine.

Tested with SerialGateway<->RelayActuator-Example.

[reddy11]

I can also report success and things look pretty good. Had to change the line:

//MyTransportRFM69 transport;
MyTransportRFM69 transport(RFM69_FREQUENCY, RFM69_NETWORKID, RF69_SPI_CS, RF69_IRQ_PIN, true, RF69_IRQ_NUM);

So that the rfm69HW could be supported.

Now, on to see if I can get the MQTT working.

[kolaf]

@reddy11 I had to do the same, although I did the change in the default parameters for the function definition since all my radios are HW :-).

I'm not using the MQTT gateway, I'm using the serial gateway together with a Perl script someone here made which functions as a MQTT gateway, and which I modified to handle node ID assignments.

[kolaf]

@hek You didn't see any extra node ID requests when the sensor booted?

[tekka]

@kolaf are you using the OTA bootloader?