Over the air (OTA) bootloading update tutorial?
-
@KaaIoT What is the webinar exactly about? I spend the last 2-3 evenings rebuilding the OTA functionallity in a custom controller and compiling the bootloader for my 8mhz 3.3v nodes.
Maybe i can learn something to improve it?
-
@Oitzu could you share how you are building the messages and other things to send the new hex?
-
As examples i recommend the nodejs controller: https://github.com/mysensors/Arduino/blob/development/NodeJsController/NodeJsController.js
Also the bootloader source:
https://github.com/mysensors/Arduino/tree/master/Bootloader
-
@Oitzu said:
@KaaIoT What is the webinar exactly about? I spend the last 2-3 evenings rebuilding the OTA functionallity in a custom controller and compiling the bootloader for my 8mhz 3.3v nodes.
Maybe i can learn something to improve it?I think maybe it was spam or something. The post is gone.
-
@NeverDie said:
I think maybe it was spam or something. The post is gone.
Yes, it was classified as spam by the moderators.
-
I am starting to play with OTA updates and I'm interested in testing the DualOptiBoot method. I have been searching the forum but I haven't been able to find a recommendation for what external flash memory to get. Does anyone have any suggestions? An eBay link would be fantastic if possible.
-
I have a mouser link to a memory that should work: http://eu.mouser.com/Search/m_ProductDetail.aspx?R=AT25DF512C-SSHN-Bvirtualkey58070000virtualkey988-AT25DF512CSSHN-B
-
@petewill
please note that these external flashes are typically only rated for 3.3V.. (I can't remember if I have seen a 5V external flash actually).
-
@Anticimex Thanks! Maybe I'll just order from Mouser or Digi-Key. I need to get the ATSHA204A also and I can't find much on eBay.
@tbowmo Good to know, thanks! I'm sure I'll have more questions on wiring when I finally get to that point...
-
Does a signed crc really provide sufficient security? Modifying a firmware in a way that results in the same crc should be fairly easy, since crc is very predictable.
-
@mfalkvidd if so, the choice of crc algorithm is bad. And a hash should be used instead.
-
Yes. A cryptographic hash function to be specific.
-
Well, obviously. We already have sha256 capability. But not publicly available.
-
@mfalkvidd even if crc can be predictable, the signing mechanism is not. So let's assume you can fabricate a firmware with a desirable crc, you still need to provide a valid signature for that crc. And that would not be so easy given the use of a random nonce and a PSK.
-
Good point. Using a nonce should be enough even if a predictable compression function is used. The signing would then verify the entire conversation, not just the binary blob.
-
The use of random nonce ensures (at least to a significant extent) that two signatures will never look the same even with the same payload. So replaying signed messages won't work. Based on that, it won't be possible for an attacker to provide a trusted crc of any form after it has sent the forged FW that yield the same crc as a valid firmware would.
The only way I see that this could be exploited is if the attacker managed to predict the resulting crc and black out the valid FW as it is sent OTA and instead inject the forged FW. And then it let the valid senders signed crc pass though.
But that require the attacker to know the resulting crc of the real FW. And if the OTA solution include a random component with the firmware that is covered by crc that also becomes a tricky task. @tekka might be interested in that.
-
In it's current stage, the OTA FW update is initiated by a FIRMWARE_CONFIG_RESPONSE message consisting of FW type, FW version, and FW CRC. If any of these parameters mismatches, the node will request a new FW. The CRC is validated at the end of the OTA update process against the transmitted FW and written in the EEPROM. This opens ways to forge the OTA update process, as described by @Anticimex
In order to make the OTA update process more secure, adding a random byte to every FW block transmitted and computing the CRC over all sent bytes makes the process more secure and the CRC less predictable. This also implies that the signed CRC is transmitted at the end of the update process and validated against the received FW + random bytes. If any component of the transmitted FW is altered, the CRC will fail and the new FW discarded.
-
I'm not sure a random byte would be sufficient. How do we verify that the correct random byte is used? If the attacker can choose the random number in their firmware, getting a crc that matches the original firmware is trivial. Or do you suggest that a nonce is used for each FW packet? If so, how is that nonce verified?
-
Why would a random byte not be sufficient? The only problem to solve is to make the crc unpredictable. Also, making sure a OTA process is started and finalized by a signed message, and those messages are a function of the OTA firmware, I don't see any security implications.
-
My point is that a crc is, by definition, never unpredictable. If an attacker records one firmware update, the attacker can easily replace the firmware and adjust the random bytes to arrive at the same crc. Then the attacker can simply re-use the signature, since it will still be valid. Or am I missing something?
-
@mfalkvidd how can an attacker reuse a signature? Nonce used is discarded when a message is signed/verified. If the crc is sent last and based on a unpredictable blob, crc is also unpredictable and only the true source can put a valid signature on a crc that will unlikely be the same two times in a row. On top of that, throw in some AES encryption and I'd say the attacker would be better off with doing a smash & grab on the node to do what he wants.
-
I agree. I guess what's confusing me is that you're talking about validating the conversation, while tekka is talking about validating the crc. Just validating the crc will be insufficient, but that's not what you're talking about.
-
@mfalkvidd well, that is what I'm talking about. My point being, that you can't forge a valid signature. So you can't record a signed message of a crc, make your own firmware that happens to result in the same crc, transmit that, and then send the same signed crc. The receiver won't accept it since a new nonce is used every time. And the PSK is needed to calculate a new signature with a new nonce.
-
Agreed.
Will adding a random number to each FW packet (as suggested by tekka) add any security? A single nonce somewhere in the conversation should be sufficient I think. Adding random bytes to each packet would add complexity but not security, as the protection would come from the nonce anyway, right?
-
@mfalkvidd Both result in the same - making the CRC "unpredictable", which is what we need. You can see the added random bytes as a nonce streching over the entire OTA update process.
-
Nonce provide security for a single message. The signed message is secure in the sense that it is authenticated. This means that the signed message could only be made from a trusted sender. Securing OTA is therefore a matter of crafting a solution where the signed messages cover data that is a function of the OTA firmware. Preferably a function which is difficult to reverse-engineer (like a cryptographic hash function). So, if there is a signed message with some form of checksum of the firmware that is enough to guarantee authenticity of the firmware. In this case, assuming the signing paradigm is ideally secure, the signature is good enough but the security will always be limited to the strength of the checksum. Injecting signed messages in the OTA flow at some intervals to continuously "monitor" the procedure could also help increasing the security.
-
and yes, I agree with @tekka, in that making an "unpredictable" crc and signing that crc would make it really hard for an attacker to temporarily block the senders firmware and inject a bad firmware and then let through the senders signed crc since the attacker would not know in "runtime" what the crc will be. Especially if the last random bit is provided in the signed crc message. That makes a secure chain where the last link is signed, and unpredictable.
-
Looks like I'll need to make some drawings to understand what is signed. Adding random bytes to the FW packets will not make the crc unpredictable, since the attacker can calcullate the real crc continuously and just needs to modify the last bytes to get the same crc.
I'll try to make some drawings of my impression of what you are saying when I get to a computer (probably tomorrow). From there you'll probably be able to tell me where I have gotten things wrong.
-
Alright. I've spent some time to draw my understanding of what Anticimex is suggesting, and described two attack scenarios.
I've probably misunderstood something along the way, so feel free to correct me if the method is wrong.Attack scenario 2 gives immense power to the attacker. I'm not sure if that is a realistic attack scenario, as Anticimex mentioned before it might be easier to do a smash and grab to get the physical sensor. I'm fine with that, if someone is able to argue that it is a scenario we don't want to protect from.
Text in purple is authenticated. Text in black is not.
I'm attaching the xml file from https://www.draw.io/ - feel free to correct if I have misunderstood something. (Or use whatever neat tool Anticimex used when describing the message signing process.)
MySensors OTA.xmlBase fact: A CRC is, by definition, redundant. That means that the CRC does not add any information to the conversation. The CRC can be computed on-the-fly by anyone. To get a selected CRC, an attacker can flip the corresponding bit(s) anywhere in the message (mod 16 bits).
Attack scenario 1: The attacker is able to read anything and send their own packets, but is unable to block or modify legitimate traffic.
In this case, the attacker is unable to create a valid signature since a new nonce will be generated by the node for every firmware update, and the life time of the nonce is limited (just as described by Anticimex in the signing thread. The only viable attack is for the attacker to ask the Node for new nonces repeatedly until a nonce is re-used.Attack scenario 2: The attacker is able to modify, block and inject traffic in an ongoing conversation.
In this case, if the above OTA method is used, the attacker can simply replace the firmware on-the-fly, calculate the CRC on-the-fly and replace the last two bytes (assuming a CRC16) to arrive at the original CRC. The signature created by the Controller will still be valid.
-
I'm not sure I understand the illustration. It seem to suggest that a nonce exchange is performed first, then firmware is transferred and last a signature is verified.
But nonce exchange occurs behind the curtains as soon as a signed message is to be transmitted, so there is nothing sent in between a nonce exchange and the signed message the nonce belongs to.
I still maintain that a solution where a crc is calculated across the firmware blocks and finalized with a signed message where a random component is included in the signed message and the final resulting crc is also included and calculated with that random last part would be enough to guarantee the transmitted firmware authenticity as an attacker would not know what the final crc will be and can't tamper with the last part.
But I do welcome any attempt to attack the security so we can put this to the test. @tekka is working with the OTA solution for 2.0.0 where this scheme is supposed to be implemented. Perhaps you would like to volunteer to set up a attack environment so we can see if crc used in this way can be tricked and a bad firmware can be injected OTA?
I should mention that in theory, you could block the firmware transfer and on the fly generate a custom firmware that matches the crc up to the point where the final signed crc is sent, but the true sender won't wait it will send the final signed message immediately after the firmware, so there won't be a window for the bad firmware to reach the node and the signing backend has a timer. So if the receiver receives a nonce request and no signed message is received within a certain timeout, nonce is discarded and signing session is terminated.
So there is literally very limited time to perform all these operations to interdict the OTA session and still allow the real signed message to arrive without the receiver getting suspicious. And if the signed message does get through in the middle of some rogue FW transfer I suppose the node would reboot with a partially flashed firmware (@tekka might be able to confirm / deny this).
-
I assume that the firmware will be large enough to require multiple messages. Is that correct?
@Anticimex are you saying that each FW_part in my illustration should be a signed message?
-
The firmware require a lot of messages. And none of them are signed.
But nonce and signed message is exchanged immediately. I interpret your illustration as if nonce is exchange first, then firmware is transferred, and finally a signed message using the previous nonce is sent. This is not how signing works.
-
Thanks for clarifying. Is this a correct representation?
-
From a signing perspective, yes. From current OTA perspective, probably (I'm not sure). From future OTA perspective, no (crc will be sent last and signed with a unknown beforehand component of the final crc)
-
Alright, I guess I'll just have to wait until there is any documentation to discuss
As for the offer to set up an attack environment, I'm afraid I know way too little about radio communication to cause the interference required.
-
Well, I agree that crc can be spoofed. So I follow you on that. But I don't agree that sending a signed message to conclude the OTA session with a crc that needs to match previously sent firmware can be hijacked. This because transmissions are driven by the sender, also for signed messages, and signed messages cannot be delayed. We gave timers in place to detect that. So an attacker cannot delay the final signed message in order to prepare a different firmware that matches the crc. The sender will not wait with sending the final crc. And each firmware blob will contain a random part to scramble crc so a previous OTA session can not be used for reference.
This would in my opinion secure OTA. But it is worth stressing that this is not how OTA works today, and hence my big disclaimer on my signing post on OTA. Tekka is working on this and might be able to provide you the documentation you seek. For signing, I refer to my post on that (and the doxygen link if you want to look at the bleeding edge development branch).
-
I think there are some missing information here and I will provide some further detail to answer some of the open questions regarding the OTA process in 2.0.0.
As for the CRC: the CRC used in the OTA update process is a function of the transmitted firmware AND a random element (added by the controller). In the final "validation" message, which is signed of course, the CRC and the last random element is transmitted and the node validates it to authorize the newly transmitted FW.
Due to the random element present in the validation message, the final CRC cannot be predicted before it is actually send and not be forged due to the signature.
-
@tekka said:
Due to the random element present in the validation message, the final CRC cannot be predicted before it is actually send and not be forged due to the signature.
That statement is incorrect. The crc is available to the attacker the instant the last byte of the firmware message is sent.
If the attacker blocks the original firmware messages and sends their own firmware instead, all that's required after the last byte of the original firmware message is sent is to send one more fake firmware message with 2 bytes.
If the signing timer is short enough to prevent a 2 byte message, the attacker will indeed be prevented from sending a fake firmware with the same crc as the original firmware. But is the timer really that short?
-
@mfalkvidd I'm not sure I understand your point. The last random element necessary to compute the CRC is transmitted together with the CRC and that message is signed, hence the CRC is not predictable until it is send.
Please elaborate your scenario with the 2 byte message and how this results in a valid CRC.
-
I will provide a more elaborate explanation of the signing timer since my point is being missed. But I don't have the time right now so it will be later today. But in short, we have safeguards to ensure there is no way of delaying signing handshaking so if the signed crc is sent immediately after the FW, if anyone tried to delay that (and they would need to in order to inject a new FW) either the sender or receiver will detect this.
-
Alright. I now only talk about 2.0.0 OTA, just to be clear.
Sender will push OTA FW "unconditionally". That is, it will not use ACK handshaking, just duke out the packets and when done send the final signed crc.
In this scenario, an attacker wants to replace the FW with a custom FW that will result in a crc that "belong" to the signed crc. So the attacker listens in to the transmitted packets to determine how crc "evolves" but also has to prevent receiver from receiving these packets. When the signed crc arrives, the attacker needs to delay the reception og this message.
There are two options for delaying a signed message.- Delay reception of nonce request.
- Delay reception of signed message.
For 1. As soon as sender transmits nonce request, it waits for the timer (5s) and if timed out, session terminates.
For 2. As soon as nonce leaves the receiver, it starts the timer, and if expired (5s) the nonce is purged from memory and session therefore terminates.
So, assuming no handshaking is done during firmware transfer, and attacker has to delay the signed message long enough to transfer the bad FW to the receiver before the signed message can be let through. And I don't think 5s is enough for flashing a custom firmware, and if it is, the delay is a public config variable which can be decreased as user see fit.
And again, it's important that the sender forces out the FW packets and the final signed message as fast as possible without hesitation to give an attacker minimum time for preparing mischief. I'm not sure how long a typical OTA session lasts, but the signing timer should be set to below that so no bad FW can be snuck in while a signing session is active.
-
Thanks for clarifying Anticimex.
What you are describing seems reasonable, as long as the attacker is unable to send the fake firmware at the same time as the original firmware is sent by the controller. Sending at the same time (or with some milliseconds delay to allow for adaptation of the firmware to match the original crc) would give the attacker plenty of time to submit their firmware.
In a wired network, blocking original traffic while sending fake traffic is a valid attack scenario. Does such a scenario not apply to radio?
-
As both sender and receiver are on the same channel, blocking out the receiver from hearing the sender also means the receiver does not hear anything else, so from rf knowledge I have, it ought not be possible to jam a transmitter without also jamming a receiver in this case. Someone might perhaps correct me on this though, but I find it an unlikely care. Also, the jammer has to both jam the sender and at the same time listen in on the sender to determine the crc to fake and that ought also be a tricky problem.
-
ok. That means that my attack scenario 2 is invalid, which is fine. Thanks a lot for bearing with me.
If we are sure that the attacker is unable to send while the controller is transmitting, why do we bother with signing the CRC? We are saying that the attacker will be unable to transmit any firmware in the short time frame available during the signing. Why add the complexity of signing the CRC if it doesn't add to security?
Edit: I'm not arguing that we should skip the signed message, but we should keep the signed message as standard as a regular signed message and not extend it by including CRC.
-
It is still a good idea to validate that the received firmware is not corrupted in "any" way during transfer before it is accepted and booted. CRC offer a reasonable validation in this sense.
-
I agree that it is a good idea to try to detect unintentional errors when transmitting the firmware. That doesn't mean the CRC should be signed.
Signing the CRC might fool someone that the CRC can be trusted, leading to unnecessary mistakes in the future. Sending the CRC unsigned makes it clear that the CRC can only be used to detect unintentional errors.
-
Let's agree to disagree
anyway, the OTA security for 2.0.0 is currently under discussion and it might well be that we go for sha256 checksumming. That remains to be seen.
-
It's been decided that we do go for sha256. It will be hashed by the controller and the receiver will calculate the hash of the received payload. The controller will then send the calculated hash of the true firmware in a separate message (that will be signed if enabled) or possibly several (if we want to use the full 32 byte hash in the verification which we probably want). @tekka and me will communicate on the matter and security related activities can be tracked on the GitHub issue list.
-
@mfalkvidd thanks for challenging OTA security. Although I maintain that crc with signature is safe, we never picked up the topic of repeaters where the ability to inject bad stuff is a very real scenario. But sha256 will put a definite end to any questions about OTA FW authentication. (together with signing, because the hash means nothing if it is not trusted)
-
Thanks for spending time to elaborate on the plan @Anticimex
Using a cryptographic hash instead of a crc sounds great.
-
Please, can anybody write the tutorial how to bootloading OTA? Many of people (and I) can't do this thing without any guide, because don't anderstand what to do=(
-
Can I use i2c EEPROM for OTA bootloading with DualOptiboot or it has to be Serial SPI. I was not able to find any documentation on it. I brought a breakout board and it seems to be i2c, and i2c chips are easily available where I live.
-
Request, if some one help me with this question. Thanks
-
Dualoptiboot use spi protocol for eeprom. So you can't. you would have to rewrite the code to use i2c protocol.
I don't know if there are cheap spi alternative and I have not looked at ali, I buy mine at mouser.
-
@scalz Thanks buddy
. Probably I would have to keep hunting for SPI EEPROMs iguess.
-
@Suresh-Mali I have the same problem. Only I2C IC is available in my country
Looks like we have to hack the code. I will inform you when I reach this phase in my project.
-
@ahmedadelhosni, @Suresh-Mali : I don't know if you have a soldering iron ?? But you could use this ref : w25x40clsnig . This is the one used on moteino from lowpowerlab (they created Dualoptiboot).
You can find this flash ic on aliexpress. It is cheap but ic only, no eval board..
Then you can use this :
https://www.openhardware.io/view/18/OTA-and-Authentication-Evaluation-BoardI hope it helps. I have not tried these aliexpress flash but it should work. I hope
I will order some to see too.Few months ago, I made this french tut about OTA and dualoptiboot for jeedom users. I never had time to translate it. I will see when I will be able, but in case it can help, with some google translate, there are some pics too so maybe...
https://www.jeedom.com/forum/viewtopic.php?f=35&t=9539
-
@ahmedadelhosni Great, do let us know once you have something.
-
@ahmedadelhosni Hey, I found this article. Seems like the guy has used i2c for remote flash. I was not able to extactly understand whats going on, maybe you can help.
http://www.rotwang.co.uk/projects/bootloader.html
-
nice catch
yes it should work with some tests and code changes in mysensors lib to write into the i2c flash instead of spi flash...
-
@scalz , Thanks
I am not a Device Programmer, I am counting on some one to hack and make it possible. Hope somebody puts in efforts that will help everybody like me.
-
@Suresh-Mali : yep, I understand
unfortunately, I have no time, even if it is maybe not the biggest thing, tests take time..there are so much things I would like to do! Do you have a soldering iron??? What I have showed to you in previous post don't need big soldering skills...
-
@scalz Oh yes, soldering is not an issue. I have rework station and I am proficient with soldering Through Hole and SMD components.
Its about getting the EEPROMs.Buying from Ali would be my last resort. I am hunting for this chip in other parts of India. I should have an answer by day end, If I dont get them then I'll order from Ali. Waiting is the worst part of ordering from Ali. I had Order ATSHA204 on 12th Dec and still waiting.
-
Hey Guys, I managed to find a EEPROM chip from Bangalore, India. Its 25LC512, priced at Rs.65(~$1). I hope it works. I should get it next 2days. It has the same footprint as one used in SenseBenderMicro. Will try once I have them and update.
-
nice catch for the I2C bootloader.
Keep us updated please of your progress. I can't actually start this phase now as I am working on other things at the moment.
Good luck.
-
This post is deleted!
-
@Suresh-Mali please do not post the same question in more than one thread. Answer is available in http://forum.mysensors.org/topic/3060/home-made-ota-and-authentication-breakout-board/2
-
There is several alternatives in aliexpress to the memory @Anticimex posted above, SPI interface, same pin distribution, only change the memory capacity and voltage input. (and good prices)
