[Solved] Strange behavior of RFM69 OTA burning firmware after some research



  • Found strange behavior of RFM69 OTA burning firmware after some research
    OTA + RFM 69 (possible) glitches.
    Used MYScontroller – works super flawless
    Tried both MySensors Library 2.0 and MySensors Library v2.1.0
    Various boards made by Koresh 8 Mghz Atmega with RFM69 radios

    The issue: On larger sketches (like more than 20kbytes), OTA always gives CRC error. Smaller ones (4k or less) could be burned 1 out of 3 tries.

    In the picture below the blues screen is comparing two files (right column is HEX and two lines actually is single 16 bit OTA payload)
    Bin.txt file is parsed Arduino hex output to the way it is identical to JDEC flash content. TXT2 file is JDEC flash content.
    Log indicates NACK error for the block B=0065 and LEN !=29 (assume it is 0 length payload). After that, somehow instead of re-requesting that corrupted or 0 sized payload, it duplicates previously received (marked in yellow) into new address. This can happen sometimes 2-3 or 10 times during OTA process

    0_1483446249615_ota.jpg

    I spend many hours playing with MyOTAFirmwareUpdate , transport and RFM driver trying “fixing” logic there but failed. Solution found Koresh by increasing from 40ms to 100ms retryWaitTime in RFM69.h

    virtual bool sendWithRetry(uint8_t toAddress, const void* buffer, uint8_t bufferSize,  uint8_t retries=2, uint8_t retryWaitTime=100); //!< sendWithRetry (40ms roundtrip req for 61byte packets) 
    

    So may be something strange happens when rfm ack is received outside the circle in sendWithRetry function.

    My and Koresh concern it is not “real” solution. Since this kind of re-requesting of 0 sized payloads should be dealt very well in protocol layer. Otherwise if there is lots of traffic going on
    In the network, there will be lots of this kind of 0 payload situations.
    The fix actually gives good results (10 out of 10 without CRC errors). It works now but… I think would be great to look into deeply.

    @tekka


  • Admin

    @yury interesting finding, can you also post the MYSController log?

    Edit: Ok, this issue has a dual cause: RFM69 driver re-transmission timing and a missing payload check for incoming messages (needed as a consequence of the timing issue) - I will elaborate this today or later this week.



  • I will reproduce tonight. with MYSController log.


  • Admin

    @yury Please try with this, I've quickly added a check:

    https://github.com/tekka007/MySensors/tree/OTAUpdate
    PR: https://github.com/mysensors/MySensors/pull/718

    Edit: PR merged, try with the latest development: https://github.com/mysensors/MySensors/tree/development



  • @tekka

    with your new PR updated all works good now. I could not replicate @ 40 ms in retryWaitTime the mentioned above. so I put 10 ms like a stress test. It has survived! with lots of WRONG FWB but with CRC OK!

    the logs

    serial
    https://drive.google.com/open?id=0B11zpaGZaUo5c3NMT1VuSGoxU3c

    last OTA is correspoding to serial log
    https://drive.google.com/open?id=0B11zpaGZaUo5MVh2cUg2cWMxWER0aHEwWHhoY1VidEpuOURn


  • Contest Winner

    Just 2 cents... We couldn't replicate 40 ms in retryWaitTime the mentioned above in another location. And tonight Yury couldnt replicate this case at home too. But yesterday it was typical situation. So I can assume it was caused by high environment radio noise (city center with many potential RF sources). I think it is an ideal situation to test boards and improve library stability :wink:


  • Admin

    @yury @Koresh Thanks for your feedback and thorough analysis, that helped to identify the issue.


  • Hardware Contributor

    Hi,

    I might be a bit dumb , but I can't find RFM69 OTA bootloader ?

    Has anyone done it ?


  • Hardware Contributor

    @napo7
    this is sensebender micro bootloader, same as dualoptiboot from lowpowerlab
    https://www.mysensors.org/about/ota


  • Hardware Contributor

    Ah ah ! I was sure that I was dumb !!

    I'll have to give a try to port MYSBootLoader to rfm69 so ....


  • Admin

    @napo7 There is something cooking in the lab...will post more in a bit :)


  • Hardware Contributor

    Well, I'll wait a bit, so !

    BTW :how much is a bit for you ? :laughing:


  • Admin

    @napo7 Hmm...1/8th of a byte ;) I cannot give you an ETA right now, but feel free to write your own bootloader implementation of the RFM69 driver.


  • Hardware Contributor

    Well done ! I must admit I wasn't ready to read such a word play ;)
    Hum... In fact, I did a small try, cloned the RF24 bootloader, tried to search which pieces of the code I have to modify, but didn't go further....
    If you was about to release it in the future days (I'm willing to help/debug/test), I'd have waited, but my projects are going to be on a rush....

    I'll try to go a bit further in making my own version :)


  • Admin

    @napo7 perfect!



  • The bootloader and OTA update script + RFM69 work super nice with MYScontroller. I use it few times a week. Big Big Big ups to Tekka. Especially I like the sensebender micro bootloader boots instantly ))). Enjoy everybody!


Log in to reply
 

Looks like your connection to MySensors Forum was lost, please wait while we try to reconnect.