I had some time today to look at the bootloader code for RFM69 modules. Got the bootloader code to a point where it appears to be working fine - but the instability of the full MySensors RFM69 code is back - the serial gateway stops receiving packets after some time and it appears that the attempt to download a firmware by the bootloader just makes this problem more obvious due to the high volume of packets...
My prime suspect at the moment is the interrupt driven RFM69 library using a single packet buffer. As long is it's used sequentially everything should be fine but with e.g. submission or retrieval of ack messages it opens up the transceiver while the last packet was not fully processed yet potentially causing this... I'll go ahead and replace the routines in the library with the non-interrupt-driven version I coded for the bootloader just to see if this is the right place to dig deeper...

RFM69 is stable now for >48h.
The OTA bootloader version for RFM69 is in progress. I'm stuck right now where the node debug messages show that it's sending a packet but the gateway doesn't receive anything... I moved the full initialization code back in that I thought would be ok to skip but still the same behavior. Only difference to the full blown driver at this point is that I don't attach the interrupt but that should not be relevant for tx as I'm polling the interrupt register for status (ready to send / send complete etc.).

I have the RFM69 setup working now with a couple of additional tweaks mainly for code optimization. Working fine for some time but stops after a few hours - I'll have a closer look the next couple of days.
Felix added three bytes to the payload (source address / destination address / control byte) to manage the acknowledge etc. in the lowpowerlab library. For our specific needs this is redundant as the current message format in development already includes this data. The three additional bytes are not that bad though and I'd propose to leave it like that for now and reconsider when we get to the revised message format :).

@tekka
That sounds great! Can you post your code (or share a pull request)? This would either allow us to free up the remaining space or to add encryption :)
The only neck-breaker would be if any of the size reduction increases the risk for a bricked node that needs manual intervention (e.g. reset / power cycle etc.).

@lunarok said:

And last point, there is actually a internal message type Reboot. This one is indicated only working for OTA bootloader. Is that true ?

Reset and OTA bootloader are independent - but you need to set the fuse bits of the MCU to enable the watchdog for the software reset to work. This is done automatically when flashing the OTA bootloader but you can do that as well in a separate step keeping the existing bootloader (e.g. optiboot).

@kolaf
I've been busy with other stuff for some time but looking at the driver separation now once again. With nRF24 working (even bootloader adjusted to pick the right header files etc.) and two Moteinos at hand (Moteino RFM69W 868/915), I compiled and flashed a serial gateway and a simple temperature sensor node. Both appear to work fine looking at the debug messages except the fact that they don't listen to eachother. Both are sending but none is receiving anything (2m distance).
I did not touch or even closely look at the RFM69 driver code yet but will try to do so over the weekend. Do you have any hint where to check first?

@gregl the fuses determine how much space is reserved for the bootloader. I don't have the datasheet at hand but I think it varies from .5k to 4k. Optiboot is one of the smallest out there and the OTA bootloader consumes the full space because it needt to includs the (shrinked) wireless driver.

@marceltrapman I'm running almost all of my nodes (+gateway) using custom ATmega328 setup vs. using an Arduino (quite a few of them just on a breadboard for now). I did not try without a crystal. At the end you need to look at which operation is most time critical. Usually these are communication but SPI is not that time critical with the clock signal being generated by the MCU itself. I don't know for sure about e.g. one-wire if you want to use a DS18B20 or so. If you want a time aware sensor you probably should go with a crystal to get at least a little accuracy :)
For power consumption you should carefully check. I guess the ATmega based on the fuses turns on or off the internal clock - you might lose some of the lower power consumption again because now the ATmega itself consumes more...

@kolaf said:

maybe because I already have a fork of the main mysensors repository?

yes - Git doesn't allow two forks with the same base name (Arduino) for one user. We are about to move the changes into the mysensors/development branch. That way you can pick it up from there...

or a WLAN Gateway :)

@hek @John
using the "time.h" library on the node the time will continue to run. It will not be very accurate (as accurate as the "millis" in arduino) which should be sufficient to turn off the water after 15 minutes (no matter if that's a few seconds off) and good enough to let the clock tick further unless the next successful submission is received from the controller.

@larky said:

connection error - trying to reconnect, I think that is a good error!

Yes!!! Glad it was that easy :)

All - help me understand the use-case for a RTC on a sensor please.

From the discussion above I get that we might have two different needs:

• really high accuracy
• a second (or few seconds) deviation are acceptable

We lose accuracy in RF transmission already, therefore fetching the time on the controller or gateway and sending it to the nodes is not appropriate for the high accuracy need. A RTC would not help in this case because you need to tell the RTC the current time to start with and if that's already inaccurate, you won't get the accuracy needed. The only way to get real accuracy would be to fetch it on the node directly e.g. with a DCF77 receiver.
For the "less accuracy needed" scenario why would you add hardware to the node if receiving a time broadcast from the controller every x minutes is good enough to get the required (lower) accuracy "in software" without the need for hardware?

@larky
The package downloaded in this step:

wget http://nodejs.org/dist/v0.10.28/node-v0.10.28-linux-arm-pi.tar.gz

includes not just the "node" executable but the "npm" executable as well. Putting the uncompressed content in the right spot and setting the path appropriately afterwards lets you execute npm without the need for any additional installation step (same as for node). As far as I know the npm version that comes with the download in the installation instructions is v1.4.9.

@larky
I'm in a rush and can't look at the details but npm should not be missing. The installation of node the way it's described in the instructions wouold install npm as well. If you followed the instructions to add the path and the "node -v" command shows correct results, the same should be true for npm (I think "npm -v" would be an appropriate check for that). Once you changed /etc/profile to include the path you need to at least logout and re-login to ensure the changes are "active". When you do "sudo apt-get npm" you will get a very outdated version of npm (which is why I fetched node and npm from the nodejs.org homepage directly in the instructions).

From the error message above:

npm ERR! node -v v0.6.19
npm ERR! npm -v 1.1.4

The instructions are for node 0.10.28 and nodejs modules are version specific in a lot of cases (e.g. only valid from 0.8 onwards). When npm tries to download a module that doesn't exist for the node version you are using that would probably cause an error like this...

Happy to help you resolve this further over the weekend - maybe the hints above will already help you to fix it :)

@sundberg84
Based on what you describe I would propose that you make use of the MyGateway code. MyGateway essentially offers a way to communicate between the sensor network and the "outside world". With the three gateways available in the examples section that's either via serial (RS232) or ethernet (simple socket communication) or via MQTT (again ethernet but MQTT standard communication). You probably will end up combining the MyGateway class (or one of the gateway examples) with your code talking to nimbits and create the MySensors "NimbitsGateway" :)

@kolaf click the "here" in my post above - or use this:

https://github.com/ToSa27/Arduino/commit/328757d5bdb2681257326761fe1a0d90cdc3eba3

@kolaf it's only splitting the MySensor application layer form the underlying network layer. Still using the MySensor tree network topology and the RF24 driver. It's adding the framework to allow others to easily integrate other topologies (e.g. mesh) and other radio modules (e.g. RF69). The interface expects a reliable submission - no matter if that's achieved by software (like radiohead) or hardware (nRF24).
The MyDriver header file includes a short description of the routines and it should be fairly easy to attach it to radiohead. The main benefit is that the original MySensor tree setup still works as before if wished.

@ToSa said:

I think we should do one thing first: decouple the "network layer" from the "application layer" in MySensors:

• not having MySensors derived from RF24
• have a "generic" interface between MySensors and the radio

Initial version - tested with a simple DallasTemperatureSensor setup successfully here
Note that this fork/branch does not include the "1.4 update 1" changes for nodeID / CRLF.

Overhead is limited (~150b flash / ~20b ram):
Sketch uses 20,426 bytes (66%) of program storage space. Maximum is 30,720 bytes.
Global variables use 575 bytes (28%) of dynamic memory, leaving 1,473 bytes for local variables. Maximum is 2,048 bytes.

@Yveaux said:

So, anyone still developing on RadioHead integration, or did all of you just give up? ;-)

no progress on my end - but that's rather related to the fact that this was my first week back in the office after vacation...

I think we should do one thing first: decouple the "network layer" from the "application layer" in MySensors:

• not having MySensors derived from RF24
• have a "generic" interface between MySensors and the radio

That way it would be rather easy to make use of other radios and other network topologies - could be RH or something else. I've taken a shot at that (a lot of this based on what Kolaf and Yveaux did adjusting MySensors to RH) and will provide a link once it compiles and is tested and I hope I can keep the overhead small enough to be acceptable for the gained flexibility.