It appears this project is not really active anymore, but if anyone is interested here is a node red flow you can use to allow MYSController to use MQTT - with OTA updates (MYSBootloader)

In MYSController setup to use TCP on port 8164 (or choose another in Node red).

Then click "Discover" and everything should then start streaming in.

[{"id":"05daaa30f133dea6","type":"tab","label":"MYSController Bridge","disabled":false,"info":"","env":[]},{"id":"0c03db96217a8d15","type":"tcp in","z":"05daaa30f133dea6","name":"","server":"server","host":"","port":"8164","datamode":"stream","datatype":"utf8","newline":"","topic":"","trim":false,"base64":false,"tls":"","x":120,"y":160,"wires":[["207a2bef4ba58466"]]},{"id":"b216b0fc146f38d6","type":"mqtt out","z":"05daaa30f133dea6","name":"To mygateway1-in","topic":"","qos":"","retain":"","respTopic":"","contentType":"","userProps":"","correl":"","expiry":"","broker":"6da0af37c3ba3336","x":670,"y":160,"wires":[]},{"id":"207a2bef4ba58466","type":"function","z":"05daaa30f133dea6","name":"function 1","func":"flow.set("tcp_session",msg._session);\nflow.set("tcp_ip",msg._session);\nflow.set("tcp_port",msg._session);\nvar parts = msg.payload.split(";");\nmsg.topic = "mygateway1-in" + '/' + parts[0] + '/' + parts[1] + '/' + parts[2] + '/' + parts[3] + '/' + parts[4];\nmsg.payload = parts[5];\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":380,"y":160,"wires":[["b216b0fc146f38d6"]]},{"id":"e97f3f15b6b5c04f","type":"mqtt in","z":"05daaa30f133dea6","name":"From myqateway1-out","topic":"mygateway1-out/#","qos":"2","datatype":"utf8","broker":"6da0af37c3ba3336","nl":false,"rap":true,"rh":0,"inputs":0,"x":160,"y":240,"wires":[["aa75b302916ca68b"]]},{"id":"7f06f63ff068402c","type":"tcp out","z":"05daaa30f133dea6","name":"","host":"","port":"8164","beserver":"reply","base64":false,"end":false,"tls":"","x":640,"y":240,"wires":[]},{"id":"aa75b302916ca68b","type":"function","z":"05daaa30f133dea6","name":"function 2","func":"msg._session = flow.get("tcp_session");\nmsg.ip = flow.get("tcp_ip");\nmsg.port = flow.get("tcp_port");\nif (msg._session == undefined)\n return null;\nvar parts = msg.topic.split("/");\nmsg.payload = parts[1] + ';' + parts[2] + ';' + parts[3] + ';' + parts[4] + ';' + parts[5] + ';' + msg.payload + "\n";\nreturn msg;\n","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":400,"y":240,"wires":[["7f06f63ff068402c"]]},{"id":"6da0af37c3ba3336","type":"mqtt-broker","name":"OLD MYC PI","broker":"192.168.1.6","port":"1883","clientid":"","autoConnect":true,"usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"autoUnsubscribe":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closeRetain":"false","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willRetain":"false","willPayload":"","willMsg":{},"userProps":"","sessionExpiry":""}]

@zboblamont Are you "ghost levels" not perhaps echoes bouncing around the tank and nothing in the tank to absorb the sound quickly. If so perhaps increasing time between pings might help to let the previous ping dissipate.

@zboblamont Yes - that spec sheet does say "1,10uS above the TTL pulse". That is not a great english translation !

Chinglish translations sometimes might miss some important words - here is the spec translated by google from Chinese stating "above" 10us

Also, there are various people reporting issues at using 10 though. Here is something mentioned in the NewPing lib issues queue : https://bitbucket.org/teckel12/arduino-new-ping/issues/41/jsn-sr04t-20-needs-to-have-longer-high

@Timbergetter Isee you are still on 10us delay. I've read somewhere that 10us delay specified for the JSN is too short and causes failed readings in the JSN and I if I recall 15 or 20 us makes it more reliable. @zboblamont already mentioned this, so perhaps you already tried to increase the delay ???

Also, something else someone mentioned elsewhere was that metal casing caused some erratic readings on the JSN and grounding the case to the circuit's gnd helped.

@Timbergetter Can you can post the sketch you are using?

@zboblamont Oh - and its not really winter anymore here - only goes to last month in the screenshots previously shown. Not sure where you're from - but temp inside tanks here range from 0.69 C to 42.7 C - min / max for last year

Here is the last years graph for interest sake !

Here is my first prototype version installed on the rain tank :

@zboblamont I bought this one https://www.hobbytronics.co.za/p/1119/waterproof-ultrasonic-distance-transducer

Looks like its supposed to be the 5V version from those specs.

@Timbergetter I also tried the JSN SR04T sensor and had problems with it. Ended up using the cheap HC-SR04 sensors on my tanks. Have 3 different tank sensors - borehole water tank up the hill, rainwater tank at house and another reverse osmosis filtered water tank. All three going for just over a year now.

Condensation issues / waterproof I resolved by quickly spraying some clear laq spray paint over the sensors. On one tank I also added little standoffs so the sensor is not sealed onto the lid of the tank but rather 5 to 10mm above the lid to let the ambient air pass over and prevent high humidity inside the tank condensing on the sensor.

And then for the code ... I added LM35 temperature sensor and added temp compsensation. Also I do a few readings and average them out.

All-in-all I now get only one or two weird readings on rare occasions - could be a spider !

Here is the last day data of the tank I have up the hill. Its showing pump cycles as well to keep tank topped up - its a weak borehole so i can only pump a little at a time. I have a separate node with relay to turn on/off pump as well as float switch down the hole to detect low water levels in the borehole.

Here is a one week graph of the same tank : temp and distance only shown

Here is one month data

We are just coming out of winter here (in South Africa!) - so temp swings of the air inside the tanks are quite radical between day and night (except the last 2 days its been getting rather warm here !)

Basically - just wanted to let you know/see that the cheap HC-SR04 sensors can do the job - and no more hiking up hills to check tank levels - and also the pumping can be automated too by the controller

@mfalkvidd Thanks - and yes - I did make the assumption that send with ack = true means do a hardware ack. I was unaware that a "software ack" also exists. I read your comment here : https://forum.mysensors.org/topic/3346/discussion-reliable-delivery/17 and what you mention there needs to be made loud and clear in the docs - perhaps some mention in both the message send function and message receive function,

@mfalkvidd

For some reason I am now unable to reproduce the issue on the standard RelayActuator.ino example. It was definitely sending ACK requests when I was testing a few days ago. I have subsequently updated MySensors library and also MyController to their latest Snapshot - so perhaps the issue comes up under one of those scenarios.

The issue around this though is that the example does not report the current status of the relay in the loop. My own sketch was doing so. I guess many people take an example and modify it like I do. Therefore copying the example and adding in the code to send the current relay status periodically means the receive function will not work properly due to the Ack messages received,

I would then suggest adding a note to the receive function of the relay example sketch to say that "if your node sends messages then you need to check for Ack and discard those messages" - or something along those lines. This will help !

There is a problem with the example code for RelayActuator.ino It is not checking for ACK messages. See the other example called SecureActuator.ino that does this. Without checking for ACK messages my relay gets an ON signal and turns on then immediately thereafter gets an ACK signal for V_STATUS command which is assumed in this example to be a control and the value is "0" so it turns off the relay.

@jkandasa Ok. I will try it out and let you know. Thanks for all the help. Its greatly appreciated.

@jkandasa There are 2 problems I have moving to MQTT. Firstly MYSController only works on Ethernet and Serial. Secondly, in MyController.org I lose all my sensor data as it will create new nodes for each of the sensors it picks up on a new gateway.

@jkandasa I only have an ESP8266 gateway working over ethernet. I do not have an MQTT or serial gateway. Should I rather be using one of those?

@jkandasa Not sure why the "Upload file" for the forum is not working ! Sent via email !

@jkandasa Sorry - i misread ... I do have the inbuilt MQTT disabled in MyController already

@jkandasa mycontroller.log file

I am not using MQTT for MySensors. I just installed it to try out at some stage. I am using ethernet connection for MySensors from MyController.

@jkandasa Hi. Its not too much inconvenience It would just be nice to get this working as MyController seems to work rather well for me otherwise.

I am using Version 0.0.3.Final-SNAPSHOT of MyController.org

The CHIP i installed Oracle JDK and then set it up to start on reboot by reading through this http://forum.mycontroller.org/topic/122/launch-start-sh-on-startup-or-like-a-service/2.

I also have mosquitto, node and node-red setup on the CHIP.

Hi

I am having various issues with MyController and OTA updates.

1. If I compare MYSController OTA vs MyController.org OTA, the MYSController takes about 10% of the time to do an update. I am running MyController.org on a C.H.I.P, so is it possible that its not fast enough to keep up with the Firmware Requests.

2. Sometimes when the OTA fails for whatever reason like I manually power down the node being updated or battery runs flat or perhaps RF connection is lost, then MyController seems to crash my ESP8266 Ethernet Gateway. Trying to reconnect i get "Connection Refused". I manually power down gateway and even router and still can't connect to the gateway anymore. Then I notice MyController crashed and needs to be started again. Eventually everything starts working again.

3. If my C.H.I.P is unexpectedly powered down or no.2 above occurs then the database gets corrupt and I need to delete the database files in /conf and launch MyController, then do a restore, then wait a bit, then start up mycontroller again. With No.2 above seeming to happen quite often in trying out OTA, this means my DB is getting corrupt fairly often.

4. There seems to be no way to remove Firmware from a Node once its assigned. If I continue to experience issues with MyController.org's OTA then I would like to remove the firmware assigned to the node and rather use MySController's firmware update for now. Is it possible to add an option to select "No Firmware"

Thanks for the assistance

@gohan The regulator is an AMS1117 on the one I got here. I will try them again as they are useful for prototyping. It was working fine for a regular sensor sending updates, but got into the issues when using OTA and trying to get some decent speed of updates going. I also thought perhaps it was a dud unit - so i tried a few others and all gave same results. So perhaps a bigger capacitor will do the trick.

@gohan I soldered a 4.7uF tantulum capacitor directly onto the nrf24L01. Didn't help !

@gohan The manufacturer on the bottom of the board says robotdyn.com - lookinf at their website this is the exact one : http://robotdyn.com/catalog/adaptors/socket_adapter_for_nrf24l01_with_regulator_3_3v/

@jkandasa I tried the modified antenna out and also removed the adapter board for the nrf and soldered a header in close to the arduino pro mini to insert the nrf24l01. And after doing, with all other sensor nodes turned off I can now get around 5 seconds for blink, about 30 seconds for TimeReporter and about 60 seconds for the 60kb custom sketch.
So thank you for the tip on antenna mod. That does help. The major issue I think though was the adapter board for the nrf24l01 - for some reason that was causing lots of pauses in the OTA updates.

The tests were all using MYSController - the windows app.

I am having a few strange issues MYController.org. It appears to be getting stuck when doing a few updates in a row and also I can't seem to roll back to a previous version.

@jkandasa Thank you very much ! That is very helpful. It seems like you are getting excellent update speed.

For my testing of OTA the test sensor node is about 8m away from Gateway.

I am also using MyController.org, Gateway is ESP8266 (Ethernet), Radio NRF24L01+ (PA+LNA 1.1km range like the one shown here)

Current "live" sensor nodes are ...
Gateway (with PA+LNB) <------ 100 to 120m -----> Repeater Node / Pump Control Sensor (with PA+LNB) <-------- 120m to 150m -------> Water Tank Level Sensor (PCB antenna nrf24l01)

To get that range it needs good line of sight which luckily I have.

I will look into the antenna mod as well though, so thank you for the tips. I would be very happy if I can get about 1 to 2 minutes for updates on 16Mhz sensors.

@jkandasa Using the bootloader in your zip file link I get the same result - around 15 minutes for a 60KB hex sketch file. I used MyController - not MYSController.
Can you please tell me how long it takes you to do an OTA update - the time + hex file size?

@jkandasa I added some basic shielding around NRF's and now getting the 60Kb hex file transferred in around 15 minutes (was 2 hours!), and a 30Kb hex file transferred in about 7 minutes (prev was about 1 hour). So its a lot better, but still rather slow ! I am not sure if that is a good transfer rate for OTA and if what I am now getting can still be improved. I cannot find any information on what transfer speeds one should expect with OTA updates.
I am also using MyController, but using the 5V / 16Mhz MySBootloader file in development branch. So I am not sure if my issue is same as yours.

@maghac In your one configured as "MYSBootloader 1.3 beta DEV" which uses the MYSBootloader_16MHz.hex there is also a difference in upload.speed.

@jkandasa Thanks for the help, but the problem I am having cannot be in the sketch. The firmware update does gets started after a reboot. So that part is all working fine. Once the firmware is updating then no sketch code runs until it completes. The firmware updating is what is very very slow

@gohan not overlooked All seems to be working - range is good, MYSBootloader and OTA updates work. It just incredibly slow. It seems to do some message updates then pause for maybe 5 seconds or so, then carry on going. So when there are 1000 messages - these 5 sec pauses probably add up to what is making it take hours instead of minutes

@gohan The weather station does not sleep - it uses wait() and does so for 5 seconds every loop. The test node first waits for 5 seconds then sleeps for 55 seconds. Both these node seem to update at the same rate. However, it also seems MYSBootloader does the firmware update then launches the sketch code, so the wait / sleep cycle is irrelevant as it first updates the bootloader then runs the sketch

I am getting good range here. I have a repeater node (not using MYSBootloader yet) that is 100m away and works fine and another node 200 to 300m away (hence the repeater at 100m). The far node was dropping some messages but that is not too critical as its only a water tank level sensor. I plan to put in a long range PA+SMA type nrf24L01 on that one. The gateway and repeater already have the long range nrf's. I am now testing OTA to a weather station node that is just outside - around 10m from gateway. And I have a test node on my desk around 8m from the gateway. Both the weather station and the test node on my desk give same results for OTA - around 30min or so for every 20k of hex file.

@gohan Seems to be taking the same time at 2 meters apart. Can you tell me how long it should take to do an OTA update? You say "much less time" .. what should it take for a 64k update?

@gohan I have tried 2 different nodes in 2 different locations. The one node is about 8m away from the gateway. The other node is around 12m away.

Can someone please tell me how long it is supposed to take to upload a hex file with MYSBootloader. I am trying out OTA Updates with the latest dev branch of MYSBootloader and it takes almost 2 hours to upload a 64k hex file. Is this normal or should it be a lot quicker?

@maghac I copied and tried your settings for

proMYSBL16.name=ATmega328 16Mhz MYSBootloader 1.3 beta DEV

and I get errors like this uploading via FTDI cable ...
avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_getsync() attempt 1 of 10: not in sync: resp=0x2c
etc

If you want to try my settings then change the 16Mhz settings to this -

#ATmega328 16Mhz MYSBootloader 1.3 beta DEV
proMYSBL16.name=ATmega328 16Mhz MYSBootloader 1.3 beta DEV

proMYSBL16.upload.tool=avrdude
proMYSBL16.upload.protocol=arduino
proMYSBL16.upload.maximum_size=30720
proMYSBL16.upload.maximum_data_size=2048
proMYSBL16.upload.speed=115200

proMYSBL16.bootloader.tool=avrdude
proMYSBL16.bootloader.low_fuses=0xF7
proMYSBL16.bootloader.high_fuses=0xDA
proMYSBL16.bootloader.extended_fuses=0x06
proMYSBL16.bootloader.unlock_bits=0x3F
proMYSBL16.bootloader.lock_bits=0x0F
proMYSBL16.bootloader.file=MySensors/MYSBootloader_16MHz.hex

proMYSBL16.build.mcu=atmega328p
proMYSBL16.build.f_cpu=16000000L
proMYSBL16.build.board=AVR_UNO
proMYSBL16.build.core=arduino
proMYSBL16.build.variant=standard

And rem Arduino IDE needs to be closed and restarted after changes

@maghac I tested a bit now again and see there is still an issue when uploading sketch using USB Asp cable. I am getting this error :

avrdude: Yikes! Invalid device signature.
Double check connections and try again, or use -F to override
this check.

However, that is non-critical for me as I only want to flash the bootloader using USB Asp cable and then I use the FTDI Cable for sketch uploads

@maghac
The previous version (bundled with MYSController) I was using would not allow me to upload sketches using FTDI cable after flashing the MYSBootloader. It also had the node reboot issue.

The latest dev branch from github link does allow me to now and the node reboot works too !

I copied all 3 hex files into a MySensors folder (C:\Program Files (x86)\Arduino\hardware\arduino\avr\bootloaders\MySensors)

They only had one board in the boards.txt - I only have 5V 16MHz Pro Mini's so changed link to hex file in the sample boards.txt they gave and added that to bottom of boards.txt file in Arduino install dir. boards.txt content I copied is ..

MYSBootloader

proMYSBL.name=ATmega328 16Mhz MYSBootloader

proMYSBL.upload.tool=avrdude
proMYSBL.upload.protocol=arduino
proMYSBL.upload.maximum_size=30720
proMYSBL.upload.maximum_data_size=2048
proMYSBL.upload.speed=115200

proMYSBL.bootloader.tool=avrdude
proMYSBL.bootloader.low_fuses=0xF7
proMYSBL.bootloader.high_fuses=0xDA
proMYSBL.bootloader.extended_fuses=0x06
proMYSBL.bootloader.unlock_bits=0x3F
proMYSBL.bootloader.lock_bits=0x0F
proMYSBL.bootloader.file=MySensors/MYSBootloader_16MHz.hex

proMYSBL.build.mcu=atmega328p
proMYSBL.build.f_cpu=16000000L
proMYSBL.build.board=AVR_UNO
proMYSBL.build.core=arduino
proMYSBL.build.variant=standard

@maghac I was still having issues with MYSBootloader. This was using the version bundled with MYSController - in a folder called "MYSController_1_0_0beta".
I found one on github here https://github.com/mysensors/MySensorsBootloaderRF24
It seems to be a newer version.
I downloaded that one and setup the 16MHz from their boards.txt file and so far this seems to work.

After some looking around i found this ..
https://arduino.stackexchange.com/questions/2922/watchdog-timer-stuck-in-restart-loop-green-led-flashing

I flashed Optiboot bootloader and that worked.

I flashed MYSBootloader again and now it does work - I am fairly certain this bootloader was causing the issue before ?!? Oh well - it works now !

I flashed with Arduino Pro Mini 5V 16Mhz and that caused the reset hang

So it seems to be an issue with the default Arduino Pro Mini bootloader that comes with Arduino IDE.

Hope this helps anyone else experiencing same issue.

One more thing that may be helpful ... when the node "hangs" the status LED is blinking rapidly.

If by "board definitions" you mean the detail in the boards.txt file - here is what I have for Arduino Pro Mini / 5V 16Mhz ...

pro.name=Arduino Pro or Pro Mini

pro.upload.tool=avrdude
pro.upload.protocol=arduino

pro.bootloader.tool=avrdude
pro.bootloader.unlock_bits=0x3F
pro.bootloader.lock_bits=0x0F

pro.build.board=AVR_PRO
pro.build.core=arduino
pro.build.variant=eightanaloginputs

Arduino Pro or Pro Mini (5V, 16 MHz) w/ ATmega328

-------------------------------------------------

pro.menu.cpu.16MHzatmega328=ATmega328 (5V, 16 MHz)

pro.menu.cpu.16MHzatmega328.upload.maximum_size=30720
pro.menu.cpu.16MHzatmega328.upload.maximum_data_size=2048
pro.menu.cpu.16MHzatmega328.upload.speed=57600

pro.menu.cpu.16MHzatmega328.bootloader.low_fuses=0xFF
pro.menu.cpu.16MHzatmega328.bootloader.high_fuses=0xDA
pro.menu.cpu.16MHzatmega328.bootloader.extended_fuses=0xFD
pro.menu.cpu.16MHzatmega328.bootloader.file=atmega/ATmegaBOOT_168_atmega328.hex

pro.menu.cpu.16MHzatmega328.build.mcu=atmega328p
pro.menu.cpu.16MHzatmega328.build.f_cpu=16000000L

I have updated Arduino IDE o 1.8.2 recently which I guess that also updated the boards.txt file. Both the previous version and the latest version of Arduino have the same reboot issue.

Arduino IDE : 1.8.2

For standard installed bootloader : Arduino Pro or Pro Mini with Atmega328 (5V, 16Mhz)
For MySBootloader : MySBootloader AtMega328 external 16Mhz

If I try to reboot a node, from either MySController or MyController, the node hangs. Once it hangs from the reboot command, manually pressing the reset button on the Arduino Pro mini will not restart the arduino. It only starts again if power is removed and then added again.
I have the same issue using both the standard arduino bootloader and with MySBootloader.
Can anyone guide me to what may cause this?