Navigation

    • Register
    • Login
    • OpenHardware.io
    • Categories
    • Recent
    • Tags
    • Popular
    1. Home
    2. MagKas
    • Profile
    • Following
    • Followers
    • Topics
    • Posts
    • Best
    • Groups

    MagKas

    @MagKas

    2
    Reputation
    13
    Posts
    975
    Profile views
    0
    Followers
    0
    Following
    Joined Last Online

    MagKas Follow

    Best posts made by MagKas

    • Problems with ENC28J60 losing connection/freezing (using UIPEthernet or etherShield)? READ THIS!

      Hi all!

      I just found a bug/flaw in the code for the ENC28J60 chip which is widely used in combination with Arduino cards.

      The problem I encountered was that the MySensors plugin in my Vera Lite controller stopped uppdating values from sensors in a unexplainable way. First I thought it was because of the used NRF2401L radios which can become unstable in some cases. A restart of the gateway solved the problem temporarily.

      But while debugging I noticed that the messages from the sensors were coming in in the gateway but not to the plugin. The gateway didn't hang but the ethernet part controlled by the ENC28J60 chip just stopped sending packets to the Vera Controller (MySensors plugin). Best way to test this is to ping the gateway from your PC. If you don't get a response you will probably have the same problem as I had.

      This 'hanging' would sometimes occur after some minutes and sometimes it could take hours between 2 hangings.

      After lots of hours of debugging and searching the internet I found that there are a lot of people experiencing the same problem and most of them simply changed to a W5100 based ethernetcard. Which also seem to have som problems in combination with the used NRF2401L radio, but that's a different story.

      After simplifying the sketch in the gateway so that it is not a real gateway anymore but just sends a 'temperature message' for Node 1 to the gateway every second. This way I was certain that the problem was only in the ENC28J60 chip or code.

      And now to the solution:
      Most of the developers which have made or adjusted libraries for the ENC28J60 chip are more or less aware of the problems which comes with this chip and which are written down in the following document from Microchip:
      http://ww1.microchip.com/downloads/en/DeviceDoc/80349c.pdf
      It is the ENC28J60 Silicon Errata and Data Sheet Clarification and the above problem is caused by point 12 (and maybe even point 14?).

      Looking at the code for the sendPacket() function (in Enc28J60Network.cpp) in the UIPEthernet library you will find some lines of code which ought to take care of this problem but this fix is implemented wrong:

      // TX start
      writeRegPair(ETXSTL, start);
      
      // Set the TXND pointer to correspond to the packet size given
      writeRegPair(ETXNDL, end);
      
      // send the contents of the transmit buffer onto the network
      writeOp(ENC28J60_BIT_FIELD_SET, ECON1, ECON1_TXRTS);
      
      // Reset the transmit logic problem. See Rev. B4 Silicon Errata point 12.
      if( (readReg(EIR) & EIR_TXERIF) )
        {
          writeOp(ENC28J60_BIT_FIELD_CLR, ECON1, ECON1_TXR**TS**);
        }
      

      In the errata document it says that you have to "reset the internal transmit logic" BEFORE setting the TXRTS flag. In the code above you will see that the reset code comes AFTER the code for setting this flag. The second problem is that it also is the wrong flag/bit that is used for this, namely the ECON1_TXRTS bit in stead of the supposed reset bit ECON1_TXRST. It's just a mix-up of the last two letters S and T but therefore it doesn't work at all.

      Because all of the libraries available for the ENC28J60 are based on eachother the faulty fix has been copied all the time. After looking for just this fix I found that there are a few different versions of the fix and the ones from tuxgraphics.org, EtherCard and NanodeUIP libraries are the same and the best ones.
      The only problem I encountered when implementing their fix is that a few times per day my whole Arduino hangs (deadlock). I also have the same problem with my Weather station based on the tuxgraphics board. There I 'solved' this by enabling the watchdog. I removed their while() loop and the problem of the hanging Arduino disappeared.

      So, after this VERY long explanation my solution to everyone experiencing problems with this ENC-chip is to change or add the following lines to the enc28j60.cpp or Enc28J60Network.cpp file for the function sendPacket(), like this:

      // Check no transmit in progress
      //  while (readOp(ENC28J60_READ_CTRL_REG, ECON1) & ECON1_TXRTS) // Might lead to deadlocks and not explicitly advised by Microchip Errata point 12 so commented out this, MagKas 2014-10-25
      //  {
      // Reset the transmit logic problem. See Rev. B4 Silicon Errata point 12.
               if( (readReg(EIR) & EIR_TXERIF) )
               {
                  writeOp(ENC28J60_BIT_FIELD_SET, ECON1, ECON1_TXRST);
                  writeOp(ENC28J60_BIT_FIELD_CLR, ECON1, ECON1_TXRST);
                  writeOp(ENC28J60_BIT_FIELD_CLR, EIR, EIR_TXERIF); // Might be overkill but advised by Microchip Errata point 12, //MagKas 2014-10-25
               }
      //   }
      

      This code has to come BEFORE setting the ECON1_TXRTS flag according to Microchip. And don't forget to remove all other existing code in sendPacket() function that tried to fix this problem.

      Please let me know if this solved your problem!

      I already contacted Guido Socher (Tuxgraphics.com), Jean-Claude Wippler (EtherCard), Norbert Truchsess (UIPEthernet), Pascal Stang (AVRLib), Stephen Early (NanodeUIP) and Jonathan Oxer (etherShield) about this and asked them to verify my findings and if necessary to update their libraries.

      Of the above libraries, UIPEthernet and etherShield are the ones that have the wrong implementation of this fix. The others have implemented correct but with the While() statement which accordning to me could lead to deadlocks.

      With best regards,
      Magnus Kasper, Sweden

      posted in Troubleshooting
      MagKas
      MagKas

    Latest posts made by MagKas

    • RE: MySensors plugin : Cannot send command - communications error

      @BulldogLowell
      Oh, I'm sorry! I missed the line where you wrote that you have to restart the gateway to get it to work again. In my case it was sufficient to reload Vera. So my solution probably won't help you...

      Very strange though that you are able to ping the gateway but that there is no communication with the plugin. What happens when you reload Vera? Do you get the error-message immediatly or after some longer time?

      The only place in the lua-code of the plugin where this error message is set is in the function sendCommandWithMessageType when luup.io.write(cmd) is failing.
      In theory this should be solved by reloading Vera, unless your communication is still failing.
      You might want to examine (read: Google) what could be the cause of luup.io.write() to fail.

      posted in Troubleshooting
      MagKas
      MagKas
    • RE: MySensors plugin : Cannot send command - communications error

      @BulldogLowell
      I experienced a similar problem a few months ago. The problem is that the MySensors-plugin sets the failure flag (luup.set_failure(true)) and the only way to reset this is to restart/reload Vera. It should be possible and rather simple to implement a reset button in the plugin to restart only the plugin.
      But I wanted the plugin to try to restart by itself and created a keep-alive function to test communication and if that fails even to try to reset the communication. This is done by trying to reopen the connection to the gateway, which only can be done by the plugin.

      I copied and pasted a little from the lua-file and found some other stuff about keep alive-functions/timers on the internet.

      I changed the L_Arduino.lua in the following way:

      Add these lines of code to the end of the startup function just before 'end':

      luup.log('start call_timer keepAlive')
      _G["keepAlive"] = keepAlive
      luup.call_timer("keepAlive", 1, "30m", "", "SomeStuff")
      

      This will start a timer which after 30 minutes ("30m") calls the function keepAlive.

      Now add the following new function just above the startup function:

      function keepAlive(stuff)
          luup.log('keepAlive!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!')
          sendCommandWithMessageType("0;0","INTERNAL",0,tonumber(tInternalTypes["VERSION"][1]),"Get Version")
        _G["keepAlive"] = keepAlive
          luup.call_timer("keepAlive", 1, "30m", "", "SomeStuff")
      end
      

      This will ask the gateway for the Lib version, only for the reason of testing communication. In the end you have to setup the call_timer again to keep it going.

      Now I changed the function sendCommandWithMessageType so that it looks like this:

      function sendCommandWithMessageType(altid, messageType, ack, variableId, value)
          local cmd = altid..";".. msgType[messageType] .. ";" .. ack .. ";" .. variableId .. ";" .. value
         log("Sending: " .. cmd)
      
          if (luup.io.write(cmd) == false)  then
          	-- Try to reopen the connection
          	luup.variable_set("urn:upnp-org:serviceId:VContainer1","Variable5","FEL",18)
      	    openConnectionToGateway()
      	    -- Try to resend commando
      	    if (luup.io.write(cmd) == false)  then
      		    task("Cannot send command - communications error", TASK_ERROR)
      		    luup.set_failure(true)
      		    return false
      	    end	
      	    luup.variable_set("urn:upnp-org:serviceId:VContainer1","Variable5","OK",18)
         end
         return true
      end
      

      This function now tries to reopen the connection to the gateway when the plugin can't send a command to the gateway. If even this doesn't work it sets the failure flag as before. NOTICE that I use "Variable5" in my VariableContainer with ID #18. Remove these lines or adjust them to your situation!!

      Last thing you have to do is to add the function openConnectionToGateway which you can put right before sendCommandWithMessageType:

      function openConnectionToGateway()
          log("openConnectionToGateway()")
      
          -- Set the last update in a human readable form for display on the console
          local timestamp = os.time()
          local variable = tVeraTypes["LAST_UPDATE"]
          local unit = luup.variable_get(ARDUINO_SID, "Unit", ARDUINO_DEVICE)
          local timeFormat = (unit == 'M' and '%H:%M' or '%I:%M %p')			
          luup.variable_set("urn:upnp-org:serviceId:VContainer1","Variable4",os.date(timeFormat, timestamp),18)
          -- End
      
         local ipa = luup.devices[ARDUINO_DEVICE].ip
      
          local ipAddress = string.match(ipa, '^(%d%d?%d?%.%d%d?%d?%.%d%d?%d?%.%d%d?%d?)')
          local ipPort    = string.match(ipa, ':(%d+)$')
      
          if (ipAddress ~= nil) then
             if (ipPort == nil) then ipPort = IP_PORT end
      
             log('Using network connection: IP address is '..ipAddress..':'..ipPort)
             luup.io.open(ARDUINO_DEVICE, ipAddress, ipPort)
          end
      end
      

      This function takes care of reopening the connection to the gateway. NOTICE that I use "Variable4" of my VariableContainer with ID #18 to show the moment this function was called. Remove this code or adjust it to your situation!!

      Don't know if this it what you want but it works perfectly for me! Never had to reload Vera because of the gateway loosing connection for a short moment of time!

      Good luck!

      Magnus

      posted in Troubleshooting
      MagKas
      MagKas
    • RE: Water Sensor Issue

      @Dan-S.
      Might be a long shot but can it be a message sent from another sensor you have where the radio-id 'changes' to the one of your water alarm sensor?

      I mean: when I was testing different values of capacitors on the radio modules and even played a little with the PA_LEVEL for the radios I noticed that sometimes the gateway received a radio message from my test sensor and sent it to Vera but with different values/radio-id. Not that the gateway changed the radio-id but the message was received with some bits changed. I hope you understand what I try to say here...
      And it couldn't have been another sensor because I only had made one at that time.

      Don't know if the radio message from sensor to gateway has a CRC check, but it definitely should have it.

      So, if you don't find another reason for this behaviour, you might check the Vera log for messages that seems messed upp a little. Or try to power down some other sensors that might generate same kind of messages as your water alarm sensor.

      Maybe you moved a sensor to a different place so that it's radio got a worse reception?

      posted in Troubleshooting
      MagKas
      MagKas
    • RE: Pressure sensor - arduino low memory (SOLVED)

      @hek: great, but not completely correct...

      It should be: if (minuteCount == 180)

      Could you also consider to update the sketch with my proposal in my other comment for the pressuresensor-sketch regarding changing

      float pressure = bmp.readPressure()/100;
      

      to

      float pressure = bmp.readSealevelPressure(205)/100; // 205 meters above sealevel
      

      I assume most people would like to have the same pressure values as the local wheather station or airport has.

      posted in Troubleshooting
      MagKas
      MagKas
    • RE: Problems with ENC28J60 losing connection/freezing (using UIPEthernet or etherShield)? READ THIS!

      @frol, @Thomas-Ihmann, @m26872

      My sensors are still working after I changed the code according to my own findings and solution from the first post. That means I don't have a while() loop which could hang the whole system. In my case I probably don't always get feedback if the packet transmitted OK or not and I probably will lose some packets too. But my gateway never hung itself either :).

      I might have some hints for debugging you guys may or may not already have checked.

      • Don't just reset the gateway, try other things first.
      • Like pinging the gateway
      • If you can ping the gateway, try Reload(ing) the Vera system which also will restart the MySensors-plugin and re-establish the connection. Check if this helps.
      • Make sure the sensor(s) are still working, start the serial monitor on the gateway and check for incoming messages (might add some debugging code for that)
      • DON'T activate the DEBUG-flag in MyConfig.h because this definitely will 'break' the gateway.
      • In case you are using a pressure-sensor BMP085/180 make sure NOT to use the sample() function because this will/might make your sensor hang after 180minutes.

      I didn't test Norberts fix myself, might do this when I feel for it... I have my system running perfectly now and are logging with DataMine so will rather not f... this up.

      posted in Troubleshooting
      MagKas
      MagKas
    • RE: Pressure sensor - arduino low memory (SOLVED)

      @niccodemi:

      Here I am again.

      The problem is in the sample() function. It saves the pressure every minute in the array pressureSamples[]. This one is declared for 180 values which means you can use pressureSamples[0] to pressureSamples[179]. Writing to pressureSamples[180] means you are writing outside the defined array and might be writing over some other stuff in your program.
      In my case it resulted in radio messages which got status 'fail' all the time. First I thought it was a problem with the radio but after debugging I found the problem to be the sample() function. It always happened exactly 180 minutes after I restarted the Arduino...

      You have to change the following lines of code:

      if (minuteCount > 180)
        minuteCount = 6;
      

      into:

      if (minuteCount == 180)
        minuteCount = 5;
      

      The problem with the original code was that it wrote a value to pressureSamples[180] before it restarted with 6 again (which should be 5 according to me).

      When looking at the code I also found that the wrong samples seem to be used for the first 5 minutes. If minuteCount has reached the value of 5 it means that samples 0 to 4 has been filled. The code is using 1 to 5 in stead.

      To fix this, change the code under if (minuteCount == 5) into:

      pressureAvg[0] = ((pressureSamples[0] + pressureSamples[1]
      		+ pressureSamples[2] + pressureSamples[3] + pressureSamples[4])
      		/ 5);
      

      To be clear: this bug is not something @hek introduced, he just used the code supplied by the link in the sample()-function.

      Edit: But maybe he can make sure the sketch is updated so no one else will get the same problems? 🙂

      posted in Troubleshooting
      MagKas
      MagKas
    • RE: Pressure sensor - arduino low memory (SOLVED)

      @niccodemi and @hek : unfortunately there is a bug in the sketch for the pressuresensor.
      I have to go now but I will look up how I fixed this and report here in the evening!

      posted in Troubleshooting
      MagKas
      MagKas
    • RE: Problems with ENC28J60 losing connection/freezing (using UIPEthernet or etherShield)? READ THIS!

      Thanks hek!

      Does anyone have a valid emailadress of Stephen Early? Tried the one I found on github but are getting a 'Failure Notice'.

      posted in Troubleshooting
      MagKas
      MagKas
    • Problems with ENC28J60 losing connection/freezing (using UIPEthernet or etherShield)? READ THIS!

      Hi all!

      I just found a bug/flaw in the code for the ENC28J60 chip which is widely used in combination with Arduino cards.

      The problem I encountered was that the MySensors plugin in my Vera Lite controller stopped uppdating values from sensors in a unexplainable way. First I thought it was because of the used NRF2401L radios which can become unstable in some cases. A restart of the gateway solved the problem temporarily.

      But while debugging I noticed that the messages from the sensors were coming in in the gateway but not to the plugin. The gateway didn't hang but the ethernet part controlled by the ENC28J60 chip just stopped sending packets to the Vera Controller (MySensors plugin). Best way to test this is to ping the gateway from your PC. If you don't get a response you will probably have the same problem as I had.

      This 'hanging' would sometimes occur after some minutes and sometimes it could take hours between 2 hangings.

      After lots of hours of debugging and searching the internet I found that there are a lot of people experiencing the same problem and most of them simply changed to a W5100 based ethernetcard. Which also seem to have som problems in combination with the used NRF2401L radio, but that's a different story.

      After simplifying the sketch in the gateway so that it is not a real gateway anymore but just sends a 'temperature message' for Node 1 to the gateway every second. This way I was certain that the problem was only in the ENC28J60 chip or code.

      And now to the solution:
      Most of the developers which have made or adjusted libraries for the ENC28J60 chip are more or less aware of the problems which comes with this chip and which are written down in the following document from Microchip:
      http://ww1.microchip.com/downloads/en/DeviceDoc/80349c.pdf
      It is the ENC28J60 Silicon Errata and Data Sheet Clarification and the above problem is caused by point 12 (and maybe even point 14?).

      Looking at the code for the sendPacket() function (in Enc28J60Network.cpp) in the UIPEthernet library you will find some lines of code which ought to take care of this problem but this fix is implemented wrong:

      // TX start
      writeRegPair(ETXSTL, start);
      
      // Set the TXND pointer to correspond to the packet size given
      writeRegPair(ETXNDL, end);
      
      // send the contents of the transmit buffer onto the network
      writeOp(ENC28J60_BIT_FIELD_SET, ECON1, ECON1_TXRTS);
      
      // Reset the transmit logic problem. See Rev. B4 Silicon Errata point 12.
      if( (readReg(EIR) & EIR_TXERIF) )
        {
          writeOp(ENC28J60_BIT_FIELD_CLR, ECON1, ECON1_TXR**TS**);
        }
      

      In the errata document it says that you have to "reset the internal transmit logic" BEFORE setting the TXRTS flag. In the code above you will see that the reset code comes AFTER the code for setting this flag. The second problem is that it also is the wrong flag/bit that is used for this, namely the ECON1_TXRTS bit in stead of the supposed reset bit ECON1_TXRST. It's just a mix-up of the last two letters S and T but therefore it doesn't work at all.

      Because all of the libraries available for the ENC28J60 are based on eachother the faulty fix has been copied all the time. After looking for just this fix I found that there are a few different versions of the fix and the ones from tuxgraphics.org, EtherCard and NanodeUIP libraries are the same and the best ones.
      The only problem I encountered when implementing their fix is that a few times per day my whole Arduino hangs (deadlock). I also have the same problem with my Weather station based on the tuxgraphics board. There I 'solved' this by enabling the watchdog. I removed their while() loop and the problem of the hanging Arduino disappeared.

      So, after this VERY long explanation my solution to everyone experiencing problems with this ENC-chip is to change or add the following lines to the enc28j60.cpp or Enc28J60Network.cpp file for the function sendPacket(), like this:

      // Check no transmit in progress
      //  while (readOp(ENC28J60_READ_CTRL_REG, ECON1) & ECON1_TXRTS) // Might lead to deadlocks and not explicitly advised by Microchip Errata point 12 so commented out this, MagKas 2014-10-25
      //  {
      // Reset the transmit logic problem. See Rev. B4 Silicon Errata point 12.
               if( (readReg(EIR) & EIR_TXERIF) )
               {
                  writeOp(ENC28J60_BIT_FIELD_SET, ECON1, ECON1_TXRST);
                  writeOp(ENC28J60_BIT_FIELD_CLR, ECON1, ECON1_TXRST);
                  writeOp(ENC28J60_BIT_FIELD_CLR, EIR, EIR_TXERIF); // Might be overkill but advised by Microchip Errata point 12, //MagKas 2014-10-25
               }
      //   }
      

      This code has to come BEFORE setting the ECON1_TXRTS flag according to Microchip. And don't forget to remove all other existing code in sendPacket() function that tried to fix this problem.

      Please let me know if this solved your problem!

      I already contacted Guido Socher (Tuxgraphics.com), Jean-Claude Wippler (EtherCard), Norbert Truchsess (UIPEthernet), Pascal Stang (AVRLib), Stephen Early (NanodeUIP) and Jonathan Oxer (etherShield) about this and asked them to verify my findings and if necessary to update their libraries.

      Of the above libraries, UIPEthernet and etherShield are the ones that have the wrong implementation of this fix. The others have implemented correct but with the While() statement which accordning to me could lead to deadlocks.

      With best regards,
      Magnus Kasper, Sweden

      posted in Troubleshooting
      MagKas
      MagKas
    • RE: Beware: FTDI driver update will brick *fake* FTDI USB to serial converters

      This happened to me for about a week ago... All 3 Nanos I bought on ebay stopped being recognized by my pc.

      After a lot of time on Google I updated the PID on the chip back to 6001 and installed the old drivers 2.10 and even 2.08. But this works only till Windows Update gets the updated drivers again!
      Solution for this is to completely disable getting updated drivers via Windows Update but I didn't want that.

      I now fixed this by leaving the PID at 0000 and 'fixed' the old drivers to work for this value. You then have to install the drivers manually (choose the option to install drivers for a unknown device by choosing type and make of it), first for the bus then for the serial port. You only have to do this for 1 ft232rl then the others will be recognized by your pc automatically.
      Because Windows doesn't recognize the device with PID 0000 it won't try to download new drivers for it either! !

      Beware that doing so is illegal according to a statement av FTDI in their drivers! Don't even know if I am allowed to tell you all of the above but I assume that it's ok because this information is widely spread on the internet by now.

      posted in Hardware
      MagKas
      MagKas