Over the air (OTA) bootloading update tutorial?
-
Has anyone done a write up that explains what the requirements are for using OTA bootloading updates? I'm closing in on a custom PCB board design and may want the option to support that in the future. I've looked at some existing boards people are designing and most seem to add a flash memory chip to support OTA but I haven't found much information beyond that. Some questions:
- What are the software and hardware requirements in the gateway for OTA?
- What are the software and hardware requirements in a sensor for OTA?
- Is a custom bootloader needed in the sensor?
- I assume that firmware downloads are targeted at a specific sensor ID right?
- How is an OTA update initiated? By the controller? If so, which controllers support it?
Thanks in advance.
TD
-
I think only their custom bootloader is needed, but I am not sure either. I love the main pages tutorials and this topic is missing. OTA is such a great feature, everybody should be able to use it!
-
I'll try to answer your questions.
- Both serial/ethernet gateway supports OTA.
- Either external flash + DualOptiboot or MysBootloader
- Yes, a special bootloader is required to handle the board programming in-place or by moving it from flash to program memory during boot phase when this is accessible.
- Yes
- Yes, it is initated by controller. This is currently only supported by MYSController and the new kid on the block MyController.
-
Thanks @hek. One clarification to the answer to 2) above. It sounds like you're saying that to support OTA, the sensor needs either:
- external flash chip plus DualOptiboot
- MysBootloader
Is that right? Or is the external flash required for either bootloader (that sounds more correct to me)? If that is right, can the second option just overwrite the code without needing any place to store the code?
-
Yes, MYSBootloader overwrites firmware directly in place and node has no way of recovering if things fail (more that retry the firmware update over and over).
The DualOptiboot + flash is a bit safer as it can always boot from the old firmware if checksum fails for the new firmware.
-
I have made a french tuto for jeedom. it is about Dualoptiboot OTA (with external eeprom). I will try to translate it in english when I will have time (maybe this week-end I will see). or maybe if someone does it in the mean time...
-
I have a small nit-picky request - could we say "OTA Bootloading" when we mean "Over the Air Bootloading"?
We have also discussed things like the OTA payload format, which means the bytes actually sent over the air (versus over a serial link, for example). We can also talk about the OTA speed vs the SPI speed.
OTA just means "Over the Air", which can apply to several things, not just bootloading. If we keep this clear now, it won't be a perpetual confusion, when a newbie comes across discussion about OTA protocols and assumes it MUST mean bootloading protocols because OTA==bootloading.
And if this is too much trouble to keep clear, never mind. Not a deal breaker.
-
@Zeph I've updated the thread title just for you
-
OK, here's some background, for some eventual Wiki article or such.
So there are 4 main ways to get a program into the uC:
- In System Programming (ISP) - uses SPI pins + reset, can program entire flash as well as fuses. This is how we get bootloaders into protected section of Flash. (There are also parallel programmers, and sometimes even additional methods like DebugWire or JTAG).
Using these methods is more cumbersome, so the Arduino folks chose to use "bootloaders" - a small program programmed (via ISP) into a protected section of on-chip Flash, and which can in turn communicate with the outside world and write (program) a compiled sketch into the rest of on-chip flash. The bootloaders are invoked by rebooting (or upon powerup) the uC.
-
Serial bootloader using Arduino bootloader protocol (subset of Atmel's STK500 protocol). A program in a protected section of uC onboard flash that can write compiled sketches into the rest of flash. Optiboot is the name of a particular serial bootloader used in many ATMega328p Arduino derivatives. (Not always used in APM (Arduino Pro Micros) but can be).
-
MysBootloader - similar to the serial bootloader, is a small program in a protected section of on-chip flash which can write compiled sketches into the rest of the flash, but it gets the data to write from special Over The Air (OTA) bootloader packets rather than via the serial port.
-
DualOptiboot - this is another OTA bootloader, but the idea is that you write the new compiled sketch (in binary) into an external flash chip, and then only after you have received and validated the entire new binary do you reboot and let the bootloader copy from external flash to on-chip flash. Called DualOptiboot because it's derived from OptiBoot and still supports serial bootloading as well.
With the bootloaders, once they begin to program the rest of flash, they wipe out the old compiled sketch - the chip can't in general boot into a sketch again until all of the new sketch has been successfully programmed. With a serial bootloader, this is usually pretty straightforward and reliable. But flashing a page at a time via an OTA bootloader like MysBootloader is a little more dicey in that it's easy to miss packets and leave the chip in a "broken" state - with no workable sketch, only the OTA bootloader. The DualOptiboot approach is perhaps in some ways a little safer, in that the current sketch in on-chip flash is not overwritten until there is a validated copy of the new code in an external flash; then copying from external to internal flash by DualOptiboot is again pretty reliable and fast. Also, if all else fails, you can still open up the node and user serial bootloading. However, DualOptiboot only works when you have an appropriate off-chip flash memory connected, which limits your hardware.
I believe that at this time, MysBootloader only works with the nRF24L01+. It basically needs a subset of the nRF's library in the bootloader section (which grows the bootloader from 1/2 KiB to 2KiB I think). Porting it to use the RFM69 would involve using a subset of the RFM69 library in the bootloader, and had not yet been done when this was written.
If OTA bootloading fails, you can try again (you may need to power cycle the node). You can reduce the distance between source and destination. You can open the node and fall back to normal serial bootloading (at least with DualOptiboot, maybe with future versions of MysBootloader). And of course there is the fallback of using ISP (assuming the PCB doesn't do something to the reset & SPI lines that breaks that option).
Since MysBootloader keeps requesting replacements for missing or corrupted OTA bootloading packets, it usually won't fail unless the node is really out of range or there is terrible interference.
Tekka points out below a different failure mode - suppose the bootloading was successful, but the sketch itself is faulty (bugs). A serial bootloader like OptiBoot takes control upon reset (which in modern Arduinos can usually be triggered via the serial connection using an additional wire), and goes into bootloading mode if it receives proper data on the serial port - or if not, starts the currently loaded sketch. This may also happen on power up. For an OTA bootloader there's no additional wire to trigger reset on the target uC. Tekka says that MysBootloader can recover and load another sketch OTA while for some bugs in a sketch, DualOptiboot cannot recover (without a manual reset).
-
Thanks @Zeph - that's a great write up..
-
@Zeph said:
With the bootloaders, once they begin to program the rest of flash, they wipe out the old compiled sketch - the chip can't in general boot into the sketch again until all of the new sketch has been successfully programmed. With a serial bootloader, this is usually pretty straightforward and reliable. But flashing a page at a time via an OTA bootloader like MysBootloader is a little more dicey in that it's easy to miss packets and leave the chip in a "broken" state - with no workable sketch, only the bootloader itself. The DualOptiboot approach is perhaps a little safer, in that the current sketch in on-chip flash is not overwritten until there is a validated copy of the new code in an external flash; then copying from external to internal flash by DualOptiboot is again pretty reliable and fast.
Maybe it's worthwhile noting here: if you upload a faulty sketch (e.g. failing to call gw.process() in regular intervals or hanging after startup) there is no way to recover OTA using dualoptiboot. This is the strength of MYSBootloader; you can basically develop sketches, transmit OTA and test - if they fail, power-cycle (or take advantage of the watchdog) and upload a new/updated sketch.
Usually I get pretty stable and non-faulty transmissions, about 40secs to upload an average-sized sketch - besides that, MYSBootloader re-requests corrupt or incomplete FW blocks.
I believe that at this time, MysBootloader only works with the nRF24L01+. It basically needs a subset of the nRF's library in the bootloader section (which grows the bootloader from 1/2 KiB to 2KiB I think). Porting it to use the RFM69 would be a substantial rewrite I suspect.
Yep, and including RFM69 compatibility should be relatively straightforward.
-
So the MYSBootloader doesn't support RFM69 radios right now. Does the DualOptiboot support RFM69?
-
-
** I think it can also accept serial protocol - or not?**
I believe it doesn't. It would be nice if it can accept serial upload. I would then be called Dual MysBootloader
-
@tekka said:
Maybe it's worthwhile noting here: if you upload a faulty sketch (e.g. failing to call gw.process() in regular intervals or hanging after startup) there is no way to recover OTA using dualoptiboot.
Ah. In theory, a DualOptiboot system would have a checksum of the new firmware written to external flash, and would not copy from external flash to internal flash until that was valid. I gather from what you say that this is not currently implemented?
This is the strength of MYSBootloader; you can basically develop sketches, transmit OTA and test - if they fail, power-cycle (or take advantage of the watchdog) and upload a new/updated sketch.
Usually I get pretty stable and non-faulty transmissions, about 40secs to upload an average-sized sketch - besides that, MYSBootloader re-requests corrupt or incomplete FW blocks.
Cool. If MYSBootloader is reliable in loading new firmware OTA, I'm not sure what point there would be to the external flash method (DualOptiboot), at least within the MySensors ecosystem. Perhaps bootloader size?
If there was some way to embed a full library (ie: all functions needed by our sketches) for the nRF into the bootloader for OTA flashing, but also make it accessible it from sketches rather than loading a second copy of a nRF library, that could be a low cost variant (in terms of overall uC Flash).
-
@tekka said:
This is the strength of MYSBootloader; you can basically develop sketches, transmit OTA and test - if they fail, power-cycle (or take advantage of the watchdog) and upload a new/updated sketch.
MysBootloader sounds very promising! In relation to the OP, who asked for a tutorial, is there a tutorial for it?
-
A thing to remember is that while MYSBootloader does not require an external flash to intermediately store the new firmware, this means it cannot support authenticating new firmwares as well (at least not currently from what I gather).
DualOptiboot on the other hand can support this since the checksumming of the updated firmware can be signed and verified before rebooting (and overwriting the existing firmware from external flash).
If signing is enabled and required and the checksum is not properly signed, it will be rejected and the bootloader won't be "informed" to do the update. This is at least in theory. I have not yet had the time to look into OTA security. But from the little I have learned, I would recommend DualOptiboot for the paranoid for this reason. Using MYSBootloader is easy, but it is also easy for an attacker to knock out security by remotely replacing the node firmwares.
-
another little difference between the two otas is with Dualoptiboot you can still use ftdi to upload sketch (dual!), could be useful sometimes. with MYSbootloader you can't upload with ftdi. but of course if there is ota, it should not be a problem!
-
@Anticimex said:
A thing to remember is that while MYSBootloader does not require an external flash to intermediately store the new firmware, this means it cannot support authenticating new firmwares as well (at least not currently from what I gather).
DualOptiboot on the other hand can support this since the checksumming of the updated firmware can be signed and verified before rebooting (and overwriting the existing firmware from external flash).
If signing is enabled and required and the checksum is not properly signed, it will be rejected and the bootloader won't be "informed" to do the update. This is at least in theory. I have not yet had the time to look into OTA security. But from the little I have learned, I would recommend DualOptiboot for the paranoid for this reason. Using MYSBootloader is easy, but it is also easy for an attacker to knock out security by remotely replacing the node firmwares.That's correct for the 1.1 release.
However, 1.5 (various improvements) and 2.0 (signing) are under development. Stay tuned
-
@tekka that sounds good. For signing, exactly how are you planning to solve that?
-
@tekka, please add OTA Serial monitor into the Santa Klaus list
-
@Anticimex I'm testing different scenarios, one using a signed checksum to initiate the OTA - this would be compatible with the current framework. The challenges are obviously size limitations and the lack of a RNG...
-
@tekka yes if you plan to have a signer/verifier implemented in the bootloader size will definitely a challenge. About rng, yes well there is the arduino pseudo rng in SW. The atsha204 provide one in hw.
-
@tekka a signed initiation I believe would come for free unless the initiation message is explicitly ignored when deciding to sign a message in the library. It is if/when the bootloader has to do signing or verification things get tricky. A signed initiation could be hacked though. An attacker could detect am initiated OTA transfer (target reboots in bootloader/OTA mode) and then (if the bootloader is unable to process signed messages) anyone could feed anything to it. If the bootloader could verify messages though, there would be no issue of course. The payload could be sent unsigned (for efficiency) and last a checksum is sent signed (which is validated both for signature and checksum on received payload) before bootloader marks flashed software as valid.
-
@Anticimex the bootloader will only go to update mode if the initiation msg is signed and valid. The initiation msg is nothing but a signed crc+type+version fw_config message. This message is validated against the current fw crc and the type/version settings stored in the eeprom. If the message is invalid, the bootloader will proceed with the normal startup procedure (=validate fw crc against eeprom settings and start if ok).
Now, an attacker could indeed interfere with unsigned fw payload blocks - but at the end, the crc from the newly transmitted fw will not match, and thus, the eeprom settings are not updated. At this point you are left with a non-functional node and a bootloader which requests a new (valid) fw.
-
@tekka that sounds good. But this initiation message, it is sent during "normal" MySensors operation, right? If so, then you won't need to dabble with signing support in the bootloader. If the initiation message contain a crc of the "pending" FW, and this crc is protected by signature, the node will fully validate it, identify it as a OTA startup, store the crc in eeprom and reboot into FW upgrade mode. The bootloader just does it normal thing, receiving the new FW, and matching the calculated FW crc against the eeprom crc and reject the image if crc does not match. That's perfectly adequate for signed OTA.
-
I should add that this signing goes right under the "normal" signing configuration. The receiving node has to be configured to require signed messages (and be able to process them) , and the GW has to be able to calculate signatures.
-
Hey guys,
So, I am new to MySensors community and I am very interested in OTA. I've read this thread and I am wondering if there is some tutorial teaching how we can transfer from controller/gateway to the nodes the sketch.
-
This post is deleted!
-
@KaaIoT What is the webinar exactly about? I spend the last 2-3 evenings rebuilding the OTA functionallity in a custom controller and compiling the bootloader for my 8mhz 3.3v nodes.
Maybe i can learn something to improve it?
-
@Oitzu could you share how you are building the messages and other things to send the new hex?
-
As examples i recommend the nodejs controller: https://github.com/mysensors/Arduino/blob/development/NodeJsController/NodeJsController.js
Also the bootloader source:
https://github.com/mysensors/Arduino/tree/master/Bootloader
-
@Oitzu said:
@KaaIoT What is the webinar exactly about? I spend the last 2-3 evenings rebuilding the OTA functionallity in a custom controller and compiling the bootloader for my 8mhz 3.3v nodes.
Maybe i can learn something to improve it?I think maybe it was spam or something. The post is gone.
-
@NeverDie said:
I think maybe it was spam or something. The post is gone.
Yes, it was classified as spam by the moderators.
-
I am starting to play with OTA updates and I'm interested in testing the DualOptiBoot method. I have been searching the forum but I haven't been able to find a recommendation for what external flash memory to get. Does anyone have any suggestions? An eBay link would be fantastic if possible.
-
I have a mouser link to a memory that should work: http://eu.mouser.com/Search/m_ProductDetail.aspx?R=AT25DF512C-SSHN-Bvirtualkey58070000virtualkey988-AT25DF512CSSHN-B
-
@petewill
please note that these external flashes are typically only rated for 3.3V.. (I can't remember if I have seen a 5V external flash actually).
-
@Anticimex Thanks! Maybe I'll just order from Mouser or Digi-Key. I need to get the ATSHA204A also and I can't find much on eBay.
@tbowmo Good to know, thanks! I'm sure I'll have more questions on wiring when I finally get to that point...
-
Does a signed crc really provide sufficient security? Modifying a firmware in a way that results in the same crc should be fairly easy, since crc is very predictable.
-
@mfalkvidd if so, the choice of crc algorithm is bad. And a hash should be used instead.
-
Yes. A cryptographic hash function to be specific.
-
Well, obviously. We already have sha256 capability. But not publicly available.
-
@mfalkvidd even if crc can be predictable, the signing mechanism is not. So let's assume you can fabricate a firmware with a desirable crc, you still need to provide a valid signature for that crc. And that would not be so easy given the use of a random nonce and a PSK.
-
Good point. Using a nonce should be enough even if a predictable compression function is used. The signing would then verify the entire conversation, not just the binary blob.
-
The use of random nonce ensures (at least to a significant extent) that two signatures will never look the same even with the same payload. So replaying signed messages won't work. Based on that, it won't be possible for an attacker to provide a trusted crc of any form after it has sent the forged FW that yield the same crc as a valid firmware would.
The only way I see that this could be exploited is if the attacker managed to predict the resulting crc and black out the valid FW as it is sent OTA and instead inject the forged FW. And then it let the valid senders signed crc pass though.
But that require the attacker to know the resulting crc of the real FW. And if the OTA solution include a random component with the firmware that is covered by crc that also becomes a tricky task. @tekka might be interested in that.
-
In it's current stage, the OTA FW update is initiated by a FIRMWARE_CONFIG_RESPONSE message consisting of FW type, FW version, and FW CRC. If any of these parameters mismatches, the node will request a new FW. The CRC is validated at the end of the OTA update process against the transmitted FW and written in the EEPROM. This opens ways to forge the OTA update process, as described by @Anticimex
In order to make the OTA update process more secure, adding a random byte to every FW block transmitted and computing the CRC over all sent bytes makes the process more secure and the CRC less predictable. This also implies that the signed CRC is transmitted at the end of the update process and validated against the received FW + random bytes. If any component of the transmitted FW is altered, the CRC will fail and the new FW discarded.
-
I'm not sure a random byte would be sufficient. How do we verify that the correct random byte is used? If the attacker can choose the random number in their firmware, getting a crc that matches the original firmware is trivial. Or do you suggest that a nonce is used for each FW packet? If so, how is that nonce verified?
-
Why would a random byte not be sufficient? The only problem to solve is to make the crc unpredictable. Also, making sure a OTA process is started and finalized by a signed message, and those messages are a function of the OTA firmware, I don't see any security implications.
-
My point is that a crc is, by definition, never unpredictable. If an attacker records one firmware update, the attacker can easily replace the firmware and adjust the random bytes to arrive at the same crc. Then the attacker can simply re-use the signature, since it will still be valid. Or am I missing something?
-
@mfalkvidd how can an attacker reuse a signature? Nonce used is discarded when a message is signed/verified. If the crc is sent last and based on a unpredictable blob, crc is also unpredictable and only the true source can put a valid signature on a crc that will unlikely be the same two times in a row. On top of that, throw in some AES encryption and I'd say the attacker would be better off with doing a smash & grab on the node to do what he wants.
-
I agree. I guess what's confusing me is that you're talking about validating the conversation, while tekka is talking about validating the crc. Just validating the crc will be insufficient, but that's not what you're talking about.
-
@mfalkvidd well, that is what I'm talking about. My point being, that you can't forge a valid signature. So you can't record a signed message of a crc, make your own firmware that happens to result in the same crc, transmit that, and then send the same signed crc. The receiver won't accept it since a new nonce is used every time. And the PSK is needed to calculate a new signature with a new nonce.
-
Agreed.
Will adding a random number to each FW packet (as suggested by tekka) add any security? A single nonce somewhere in the conversation should be sufficient I think. Adding random bytes to each packet would add complexity but not security, as the protection would come from the nonce anyway, right?
-
@mfalkvidd Both result in the same - making the CRC "unpredictable", which is what we need. You can see the added random bytes as a nonce streching over the entire OTA update process.
-
Nonce provide security for a single message. The signed message is secure in the sense that it is authenticated. This means that the signed message could only be made from a trusted sender. Securing OTA is therefore a matter of crafting a solution where the signed messages cover data that is a function of the OTA firmware. Preferably a function which is difficult to reverse-engineer (like a cryptographic hash function). So, if there is a signed message with some form of checksum of the firmware that is enough to guarantee authenticity of the firmware. In this case, assuming the signing paradigm is ideally secure, the signature is good enough but the security will always be limited to the strength of the checksum. Injecting signed messages in the OTA flow at some intervals to continuously "monitor" the procedure could also help increasing the security.
-
and yes, I agree with @tekka, in that making an "unpredictable" crc and signing that crc would make it really hard for an attacker to temporarily block the senders firmware and inject a bad firmware and then let through the senders signed crc since the attacker would not know in "runtime" what the crc will be. Especially if the last random bit is provided in the signed crc message. That makes a secure chain where the last link is signed, and unpredictable.
-
Looks like I'll need to make some drawings to understand what is signed. Adding random bytes to the FW packets will not make the crc unpredictable, since the attacker can calcullate the real crc continuously and just needs to modify the last bytes to get the same crc.
I'll try to make some drawings of my impression of what you are saying when I get to a computer (probably tomorrow). From there you'll probably be able to tell me where I have gotten things wrong.
-
Alright. I've spent some time to draw my understanding of what Anticimex is suggesting, and described two attack scenarios.
I've probably misunderstood something along the way, so feel free to correct me if the method is wrong.Attack scenario 2 gives immense power to the attacker. I'm not sure if that is a realistic attack scenario, as Anticimex mentioned before it might be easier to do a smash and grab to get the physical sensor. I'm fine with that, if someone is able to argue that it is a scenario we don't want to protect from.
Text in purple is authenticated. Text in black is not.
I'm attaching the xml file from https://www.draw.io/ - feel free to correct if I have misunderstood something. (Or use whatever neat tool Anticimex used when describing the message signing process.)
MySensors OTA.xmlBase fact: A CRC is, by definition, redundant. That means that the CRC does not add any information to the conversation. The CRC can be computed on-the-fly by anyone. To get a selected CRC, an attacker can flip the corresponding bit(s) anywhere in the message (mod 16 bits).
Attack scenario 1: The attacker is able to read anything and send their own packets, but is unable to block or modify legitimate traffic.
In this case, the attacker is unable to create a valid signature since a new nonce will be generated by the node for every firmware update, and the life time of the nonce is limited (just as described by Anticimex in the signing thread. The only viable attack is for the attacker to ask the Node for new nonces repeatedly until a nonce is re-used.Attack scenario 2: The attacker is able to modify, block and inject traffic in an ongoing conversation.
In this case, if the above OTA method is used, the attacker can simply replace the firmware on-the-fly, calculate the CRC on-the-fly and replace the last two bytes (assuming a CRC16) to arrive at the original CRC. The signature created by the Controller will still be valid.
-
I'm not sure I understand the illustration. It seem to suggest that a nonce exchange is performed first, then firmware is transferred and last a signature is verified.
But nonce exchange occurs behind the curtains as soon as a signed message is to be transmitted, so there is nothing sent in between a nonce exchange and the signed message the nonce belongs to.
I still maintain that a solution where a crc is calculated across the firmware blocks and finalized with a signed message where a random component is included in the signed message and the final resulting crc is also included and calculated with that random last part would be enough to guarantee the transmitted firmware authenticity as an attacker would not know what the final crc will be and can't tamper with the last part.
But I do welcome any attempt to attack the security so we can put this to the test. @tekka is working with the OTA solution for 2.0.0 where this scheme is supposed to be implemented. Perhaps you would like to volunteer to set up a attack environment so we can see if crc used in this way can be tricked and a bad firmware can be injected OTA?
I should mention that in theory, you could block the firmware transfer and on the fly generate a custom firmware that matches the crc up to the point where the final signed crc is sent, but the true sender won't wait it will send the final signed message immediately after the firmware, so there won't be a window for the bad firmware to reach the node and the signing backend has a timer. So if the receiver receives a nonce request and no signed message is received within a certain timeout, nonce is discarded and signing session is terminated.
So there is literally very limited time to perform all these operations to interdict the OTA session and still allow the real signed message to arrive without the receiver getting suspicious. And if the signed message does get through in the middle of some rogue FW transfer I suppose the node would reboot with a partially flashed firmware (@tekka might be able to confirm / deny this).
-
I assume that the firmware will be large enough to require multiple messages. Is that correct?
@Anticimex are you saying that each FW_part in my illustration should be a signed message?
-
The firmware require a lot of messages. And none of them are signed.
But nonce and signed message is exchanged immediately. I interpret your illustration as if nonce is exchange first, then firmware is transferred, and finally a signed message using the previous nonce is sent. This is not how signing works.
-
Thanks for clarifying. Is this a correct representation?
-
From a signing perspective, yes. From current OTA perspective, probably (I'm not sure). From future OTA perspective, no (crc will be sent last and signed with a unknown beforehand component of the final crc)
-
Alright, I guess I'll just have to wait until there is any documentation to discuss
As for the offer to set up an attack environment, I'm afraid I know way too little about radio communication to cause the interference required.
-
Well, I agree that crc can be spoofed. So I follow you on that. But I don't agree that sending a signed message to conclude the OTA session with a crc that needs to match previously sent firmware can be hijacked. This because transmissions are driven by the sender, also for signed messages, and signed messages cannot be delayed. We gave timers in place to detect that. So an attacker cannot delay the final signed message in order to prepare a different firmware that matches the crc. The sender will not wait with sending the final crc. And each firmware blob will contain a random part to scramble crc so a previous OTA session can not be used for reference.
This would in my opinion secure OTA. But it is worth stressing that this is not how OTA works today, and hence my big disclaimer on my signing post on OTA. Tekka is working on this and might be able to provide you the documentation you seek. For signing, I refer to my post on that (and the doxygen link if you want to look at the bleeding edge development branch).
-
I think there are some missing information here and I will provide some further detail to answer some of the open questions regarding the OTA process in 2.0.0.
As for the CRC: the CRC used in the OTA update process is a function of the transmitted firmware AND a random element (added by the controller). In the final "validation" message, which is signed of course, the CRC and the last random element is transmitted and the node validates it to authorize the newly transmitted FW.
Due to the random element present in the validation message, the final CRC cannot be predicted before it is actually send and not be forged due to the signature.
-
@tekka said:
Due to the random element present in the validation message, the final CRC cannot be predicted before it is actually send and not be forged due to the signature.
That statement is incorrect. The crc is available to the attacker the instant the last byte of the firmware message is sent.
If the attacker blocks the original firmware messages and sends their own firmware instead, all that's required after the last byte of the original firmware message is sent is to send one more fake firmware message with 2 bytes.
If the signing timer is short enough to prevent a 2 byte message, the attacker will indeed be prevented from sending a fake firmware with the same crc as the original firmware. But is the timer really that short?
-
@mfalkvidd I'm not sure I understand your point. The last random element necessary to compute the CRC is transmitted together with the CRC and that message is signed, hence the CRC is not predictable until it is send.
Please elaborate your scenario with the 2 byte message and how this results in a valid CRC.
-
I will provide a more elaborate explanation of the signing timer since my point is being missed. But I don't have the time right now so it will be later today. But in short, we have safeguards to ensure there is no way of delaying signing handshaking so if the signed crc is sent immediately after the FW, if anyone tried to delay that (and they would need to in order to inject a new FW) either the sender or receiver will detect this.
-
Alright. I now only talk about 2.0.0 OTA, just to be clear.
Sender will push OTA FW "unconditionally". That is, it will not use ACK handshaking, just duke out the packets and when done send the final signed crc.
In this scenario, an attacker wants to replace the FW with a custom FW that will result in a crc that "belong" to the signed crc. So the attacker listens in to the transmitted packets to determine how crc "evolves" but also has to prevent receiver from receiving these packets. When the signed crc arrives, the attacker needs to delay the reception og this message.
There are two options for delaying a signed message.- Delay reception of nonce request.
- Delay reception of signed message.
For 1. As soon as sender transmits nonce request, it waits for the timer (5s) and if timed out, session terminates.
For 2. As soon as nonce leaves the receiver, it starts the timer, and if expired (5s) the nonce is purged from memory and session therefore terminates.
So, assuming no handshaking is done during firmware transfer, and attacker has to delay the signed message long enough to transfer the bad FW to the receiver before the signed message can be let through. And I don't think 5s is enough for flashing a custom firmware, and if it is, the delay is a public config variable which can be decreased as user see fit.
And again, it's important that the sender forces out the FW packets and the final signed message as fast as possible without hesitation to give an attacker minimum time for preparing mischief. I'm not sure how long a typical OTA session lasts, but the signing timer should be set to below that so no bad FW can be snuck in while a signing session is active.
-
Thanks for clarifying Anticimex.
What you are describing seems reasonable, as long as the attacker is unable to send the fake firmware at the same time as the original firmware is sent by the controller. Sending at the same time (or with some milliseconds delay to allow for adaptation of the firmware to match the original crc) would give the attacker plenty of time to submit their firmware.
In a wired network, blocking original traffic while sending fake traffic is a valid attack scenario. Does such a scenario not apply to radio?
-
As both sender and receiver are on the same channel, blocking out the receiver from hearing the sender also means the receiver does not hear anything else, so from rf knowledge I have, it ought not be possible to jam a transmitter without also jamming a receiver in this case. Someone might perhaps correct me on this though, but I find it an unlikely care. Also, the jammer has to both jam the sender and at the same time listen in on the sender to determine the crc to fake and that ought also be a tricky problem.
-
ok. That means that my attack scenario 2 is invalid, which is fine. Thanks a lot for bearing with me.
If we are sure that the attacker is unable to send while the controller is transmitting, why do we bother with signing the CRC? We are saying that the attacker will be unable to transmit any firmware in the short time frame available during the signing. Why add the complexity of signing the CRC if it doesn't add to security?
Edit: I'm not arguing that we should skip the signed message, but we should keep the signed message as standard as a regular signed message and not extend it by including CRC.
-
It is still a good idea to validate that the received firmware is not corrupted in "any" way during transfer before it is accepted and booted. CRC offer a reasonable validation in this sense.
-
I agree that it is a good idea to try to detect unintentional errors when transmitting the firmware. That doesn't mean the CRC should be signed.
Signing the CRC might fool someone that the CRC can be trusted, leading to unnecessary mistakes in the future. Sending the CRC unsigned makes it clear that the CRC can only be used to detect unintentional errors.
-
Let's agree to disagree
anyway, the OTA security for 2.0.0 is currently under discussion and it might well be that we go for sha256 checksumming. That remains to be seen.
-
It's been decided that we do go for sha256. It will be hashed by the controller and the receiver will calculate the hash of the received payload. The controller will then send the calculated hash of the true firmware in a separate message (that will be signed if enabled) or possibly several (if we want to use the full 32 byte hash in the verification which we probably want). @tekka and me will communicate on the matter and security related activities can be tracked on the GitHub issue list.
-
@mfalkvidd thanks for challenging OTA security. Although I maintain that crc with signature is safe, we never picked up the topic of repeaters where the ability to inject bad stuff is a very real scenario. But sha256 will put a definite end to any questions about OTA FW authentication. (together with signing, because the hash means nothing if it is not trusted)
-
Thanks for spending time to elaborate on the plan @Anticimex
Using a cryptographic hash instead of a crc sounds great.
-
Please, can anybody write the tutorial how to bootloading OTA? Many of people (and I) can't do this thing without any guide, because don't anderstand what to do=(
-
Can I use i2c EEPROM for OTA bootloading with DualOptiboot or it has to be Serial SPI. I was not able to find any documentation on it. I brought a breakout board and it seems to be i2c, and i2c chips are easily available where I live.
-
Request, if some one help me with this question. Thanks
-
Dualoptiboot use spi protocol for eeprom. So you can't. you would have to rewrite the code to use i2c protocol.
I don't know if there are cheap spi alternative and I have not looked at ali, I buy mine at mouser.
-
@scalz Thanks buddy
. Probably I would have to keep hunting for SPI EEPROMs iguess.
-
@Suresh-Mali I have the same problem. Only I2C IC is available in my country
Looks like we have to hack the code. I will inform you when I reach this phase in my project.
-
@ahmedadelhosni, @Suresh-Mali : I don't know if you have a soldering iron ?? But you could use this ref : w25x40clsnig . This is the one used on moteino from lowpowerlab (they created Dualoptiboot).
You can find this flash ic on aliexpress. It is cheap but ic only, no eval board..
Then you can use this :
https://www.openhardware.io/view/18/OTA-and-Authentication-Evaluation-BoardI hope it helps. I have not tried these aliexpress flash but it should work. I hope
I will order some to see too.Few months ago, I made this french tut about OTA and dualoptiboot for jeedom users. I never had time to translate it. I will see when I will be able, but in case it can help, with some google translate, there are some pics too so maybe...
https://www.jeedom.com/forum/viewtopic.php?f=35&t=9539
-
@ahmedadelhosni Great, do let us know once you have something.
-
@ahmedadelhosni Hey, I found this article. Seems like the guy has used i2c for remote flash. I was not able to extactly understand whats going on, maybe you can help.
http://www.rotwang.co.uk/projects/bootloader.html
-
nice catch
yes it should work with some tests and code changes in mysensors lib to write into the i2c flash instead of spi flash...
-
@scalz , Thanks
I am not a Device Programmer, I am counting on some one to hack and make it possible. Hope somebody puts in efforts that will help everybody like me.
-
@Suresh-Mali : yep, I understand
unfortunately, I have no time, even if it is maybe not the biggest thing, tests take time..there are so much things I would like to do! Do you have a soldering iron??? What I have showed to you in previous post don't need big soldering skills...
-
@scalz Oh yes, soldering is not an issue. I have rework station and I am proficient with soldering Through Hole and SMD components.
Its about getting the EEPROMs.Buying from Ali would be my last resort. I am hunting for this chip in other parts of India. I should have an answer by day end, If I dont get them then I'll order from Ali. Waiting is the worst part of ordering from Ali. I had Order ATSHA204 on 12th Dec and still waiting.
-
Hey Guys, I managed to find a EEPROM chip from Bangalore, India. Its 25LC512, priced at Rs.65(~$1). I hope it works. I should get it next 2days. It has the same footprint as one used in SenseBenderMicro. Will try once I have them and update.
-
nice catch for the I2C bootloader.
Keep us updated please of your progress. I can't actually start this phase now as I am working on other things at the moment.
Good luck.
-
This post is deleted!
-
@Suresh-Mali please do not post the same question in more than one thread. Answer is available in http://forum.mysensors.org/topic/3060/home-made-ota-and-authentication-breakout-board/2
-
There is several alternatives in aliexpress to the memory @Anticimex posted above, SPI interface, same pin distribution, only change the memory capacity and voltage input. (and good prices)
