Over the air (OTA) bootloading update tutorial?
-
** I think it can also accept serial protocol - or not?**
I believe it doesn't. It would be nice if it can accept serial upload. I would then be called Dual MysBootloader :)
-
@Zeph said:
With the bootloaders, once they begin to program the rest of flash, they wipe out the old compiled sketch - the chip can't in general boot into the sketch again until all of the new sketch has been successfully programmed. With a serial bootloader, this is usually pretty straightforward and reliable. But flashing a page at a time via an OTA bootloader like MysBootloader is a little more dicey in that it's easy to miss packets and leave the chip in a "broken" state - with no workable sketch, only the bootloader itself. The DualOptiboot approach is perhaps a little safer, in that the current sketch in on-chip flash is not overwritten until there is a validated copy of the new code in an external flash; then copying from external to internal flash by DualOptiboot is again pretty reliable and fast.
Maybe it's worthwhile noting here: if you upload a faulty sketch (e.g. failing to call gw.process() in regular intervals or hanging after startup) there is no way to recover OTA using dualoptiboot. This is the strength of MYSBootloader; you can basically develop sketches, transmit OTA and test - if they fail, power-cycle (or take advantage of the watchdog) and upload a new/updated sketch.
Usually I get pretty stable and non-faulty transmissions, about 40secs to upload an average-sized sketch - besides that, MYSBootloader re-requests corrupt or incomplete FW blocks.
I believe that at this time, MysBootloader only works with the nRF24L01+. It basically needs a subset of the nRF's library in the bootloader section (which grows the bootloader from 1/2 KiB to 2KiB I think). Porting it to use the RFM69 would be a substantial rewrite I suspect.
Yep, and including RFM69 compatibility should be relatively straightforward.
@tekka said:
Maybe it's worthwhile noting here: if you upload a faulty sketch (e.g. failing to call gw.process() in regular intervals or hanging after startup) there is no way to recover OTA using dualoptiboot.
Ah. In theory, a DualOptiboot system would have a checksum of the new firmware written to external flash, and would not copy from external flash to internal flash until that was valid. I gather from what you say that this is not currently implemented?
This is the strength of MYSBootloader; you can basically develop sketches, transmit OTA and test - if they fail, power-cycle (or take advantage of the watchdog) and upload a new/updated sketch.
Usually I get pretty stable and non-faulty transmissions, about 40secs to upload an average-sized sketch - besides that, MYSBootloader re-requests corrupt or incomplete FW blocks.
Cool. If MYSBootloader is reliable in loading new firmware OTA, I'm not sure what point there would be to the external flash method (DualOptiboot), at least within the MySensors ecosystem. Perhaps bootloader size?
If there was some way to embed a full library (ie: all functions needed by our sketches) for the nRF into the bootloader for OTA flashing, but also make it accessible it from sketches rather than loading a second copy of a nRF library, that could be a low cost variant (in terms of overall uC Flash).
-
@Zeph said:
With the bootloaders, once they begin to program the rest of flash, they wipe out the old compiled sketch - the chip can't in general boot into the sketch again until all of the new sketch has been successfully programmed. With a serial bootloader, this is usually pretty straightforward and reliable. But flashing a page at a time via an OTA bootloader like MysBootloader is a little more dicey in that it's easy to miss packets and leave the chip in a "broken" state - with no workable sketch, only the bootloader itself. The DualOptiboot approach is perhaps a little safer, in that the current sketch in on-chip flash is not overwritten until there is a validated copy of the new code in an external flash; then copying from external to internal flash by DualOptiboot is again pretty reliable and fast.
Maybe it's worthwhile noting here: if you upload a faulty sketch (e.g. failing to call gw.process() in regular intervals or hanging after startup) there is no way to recover OTA using dualoptiboot. This is the strength of MYSBootloader; you can basically develop sketches, transmit OTA and test - if they fail, power-cycle (or take advantage of the watchdog) and upload a new/updated sketch.
Usually I get pretty stable and non-faulty transmissions, about 40secs to upload an average-sized sketch - besides that, MYSBootloader re-requests corrupt or incomplete FW blocks.
I believe that at this time, MysBootloader only works with the nRF24L01+. It basically needs a subset of the nRF's library in the bootloader section (which grows the bootloader from 1/2 KiB to 2KiB I think). Porting it to use the RFM69 would be a substantial rewrite I suspect.
Yep, and including RFM69 compatibility should be relatively straightforward.
@tekka said:
This is the strength of MYSBootloader; you can basically develop sketches, transmit OTA and test - if they fail, power-cycle (or take advantage of the watchdog) and upload a new/updated sketch.
MysBootloader sounds very promising! In relation to the OP, who asked for a tutorial, is there a tutorial for it?
-
A thing to remember is that while MYSBootloader does not require an external flash to intermediately store the new firmware, this means it cannot support authenticating new firmwares as well (at least not currently from what I gather).
DualOptiboot on the other hand can support this since the checksumming of the updated firmware can be signed and verified before rebooting (and overwriting the existing firmware from external flash).
If signing is enabled and required and the checksum is not properly signed, it will be rejected and the bootloader won't be "informed" to do the update. This is at least in theory. I have not yet had the time to look into OTA security. But from the little I have learned, I would recommend DualOptiboot for the paranoid for this reason. Using MYSBootloader is easy, but it is also easy for an attacker to knock out security by remotely replacing the node firmwares. -
another little difference between the two otas is with Dualoptiboot you can still use ftdi to upload sketch (dual!), could be useful sometimes. with MYSbootloader you can't upload with ftdi. but of course if there is ota, it should not be a problem!
-
A thing to remember is that while MYSBootloader does not require an external flash to intermediately store the new firmware, this means it cannot support authenticating new firmwares as well (at least not currently from what I gather).
DualOptiboot on the other hand can support this since the checksumming of the updated firmware can be signed and verified before rebooting (and overwriting the existing firmware from external flash).
If signing is enabled and required and the checksum is not properly signed, it will be rejected and the bootloader won't be "informed" to do the update. This is at least in theory. I have not yet had the time to look into OTA security. But from the little I have learned, I would recommend DualOptiboot for the paranoid for this reason. Using MYSBootloader is easy, but it is also easy for an attacker to knock out security by remotely replacing the node firmwares.@Anticimex said:
A thing to remember is that while MYSBootloader does not require an external flash to intermediately store the new firmware, this means it cannot support authenticating new firmwares as well (at least not currently from what I gather).
DualOptiboot on the other hand can support this since the checksumming of the updated firmware can be signed and verified before rebooting (and overwriting the existing firmware from external flash).
If signing is enabled and required and the checksum is not properly signed, it will be rejected and the bootloader won't be "informed" to do the update. This is at least in theory. I have not yet had the time to look into OTA security. But from the little I have learned, I would recommend DualOptiboot for the paranoid for this reason. Using MYSBootloader is easy, but it is also easy for an attacker to knock out security by remotely replacing the node firmwares.That's correct for the 1.1 release.
However, 1.5 (various improvements) and 2.0 (signing) are under development. Stay tuned :) -
@Anticimex said:
A thing to remember is that while MYSBootloader does not require an external flash to intermediately store the new firmware, this means it cannot support authenticating new firmwares as well (at least not currently from what I gather).
DualOptiboot on the other hand can support this since the checksumming of the updated firmware can be signed and verified before rebooting (and overwriting the existing firmware from external flash).
If signing is enabled and required and the checksum is not properly signed, it will be rejected and the bootloader won't be "informed" to do the update. This is at least in theory. I have not yet had the time to look into OTA security. But from the little I have learned, I would recommend DualOptiboot for the paranoid for this reason. Using MYSBootloader is easy, but it is also easy for an attacker to knock out security by remotely replacing the node firmwares.That's correct for the 1.1 release.
However, 1.5 (various improvements) and 2.0 (signing) are under development. Stay tuned :) -
@Anticimex I'm testing different scenarios, one using a signed checksum to initiate the OTA - this would be compatible with the current framework. The challenges are obviously size limitations and the lack of a RNG...
-
@Anticimex I'm testing different scenarios, one using a signed checksum to initiate the OTA - this would be compatible with the current framework. The challenges are obviously size limitations and the lack of a RNG...
-
@Anticimex I'm testing different scenarios, one using a signed checksum to initiate the OTA - this would be compatible with the current framework. The challenges are obviously size limitations and the lack of a RNG...
@tekka a signed initiation I believe would come for free unless the initiation message is explicitly ignored when deciding to sign a message in the library. It is if/when the bootloader has to do signing or verification things get tricky. A signed initiation could be hacked though. An attacker could detect am initiated OTA transfer (target reboots in bootloader/OTA mode) and then (if the bootloader is unable to process signed messages) anyone could feed anything to it. If the bootloader could verify messages though, there would be no issue of course. The payload could be sent unsigned (for efficiency) and last a checksum is sent signed (which is validated both for signature and checksum on received payload) before bootloader marks flashed software as valid.
Do you feel secure today? No? Start requiring some signatures and feel better tomorrow ;)
-
@tekka a signed initiation I believe would come for free unless the initiation message is explicitly ignored when deciding to sign a message in the library. It is if/when the bootloader has to do signing or verification things get tricky. A signed initiation could be hacked though. An attacker could detect am initiated OTA transfer (target reboots in bootloader/OTA mode) and then (if the bootloader is unable to process signed messages) anyone could feed anything to it. If the bootloader could verify messages though, there would be no issue of course. The payload could be sent unsigned (for efficiency) and last a checksum is sent signed (which is validated both for signature and checksum on received payload) before bootloader marks flashed software as valid.
@Anticimex the bootloader will only go to update mode if the initiation msg is signed and valid. The initiation msg is nothing but a signed crc+type+version fw_config message. This message is validated against the current fw crc and the type/version settings stored in the eeprom. If the message is invalid, the bootloader will proceed with the normal startup procedure (=validate fw crc against eeprom settings and start if ok).
Now, an attacker could indeed interfere with unsigned fw payload blocks - but at the end, the crc from the newly transmitted fw will not match, and thus, the eeprom settings are not updated. At this point you are left with a non-functional node and a bootloader which requests a new (valid) fw. -
@Anticimex the bootloader will only go to update mode if the initiation msg is signed and valid. The initiation msg is nothing but a signed crc+type+version fw_config message. This message is validated against the current fw crc and the type/version settings stored in the eeprom. If the message is invalid, the bootloader will proceed with the normal startup procedure (=validate fw crc against eeprom settings and start if ok).
Now, an attacker could indeed interfere with unsigned fw payload blocks - but at the end, the crc from the newly transmitted fw will not match, and thus, the eeprom settings are not updated. At this point you are left with a non-functional node and a bootloader which requests a new (valid) fw.@tekka that sounds good. But this initiation message, it is sent during "normal" MySensors operation, right? If so, then you won't need to dabble with signing support in the bootloader. If the initiation message contain a crc of the "pending" FW, and this crc is protected by signature, the node will fully validate it, identify it as a OTA startup, store the crc in eeprom and reboot into FW upgrade mode. The bootloader just does it normal thing, receiving the new FW, and matching the calculated FW crc against the eeprom crc and reject the image if crc does not match. That's perfectly adequate for signed OTA.
-
I should add that this signing goes right under the "normal" signing configuration. The receiving node has to be configured to require signed messages (and be able to process them) , and the GW has to be able to calculate signatures.
-
Hey guys,
So, I am new to MySensors community and I am very interested in OTA. I've read this thread and I am wondering if there is some tutorial teaching how we can transfer from controller/gateway to the nodes the sketch. -
This post is deleted!
-
@Oitzu could you share how you are building the messages and other things to send the new hex?
-
As examples i recommend the nodejs controller: https://github.com/mysensors/Arduino/blob/development/NodeJsController/NodeJsController.js
Also the bootloader source:
https://github.com/mysensors/Arduino/tree/master/Bootloader -
@KaaIoT What is the webinar exactly about? I spend the last 2-3 evenings rebuilding the OTA functionallity in a custom controller and compiling the bootloader for my 8mhz 3.3v nodes.
Maybe i can learn something to improve it?@Oitzu said:
@KaaIoT What is the webinar exactly about? I spend the last 2-3 evenings rebuilding the OTA functionallity in a custom controller and compiling the bootloader for my 8mhz 3.3v nodes.
Maybe i can learn something to improve it?I think maybe it was spam or something. The post is gone.
