Watchdog on Ethernet Gateway

[BulldogLowell]

I'm still frustrated that my Ethernet Gateway will occaisionally lock up...

I can't predict it, I have been logging it and it seems to me that I cannot find the cause or culprit.

So, I was thinking about attaching an attiny45 to it, where it is constantly checking for a pulse... and can (hardware) reset the gateway in case of a lack of signal after a certain timeout. The less than $1.00 cost seems like reasonable insurance to keep things chugging along.

Has anyone already done something like this (@axillent , @hek calling on your experience)? I've searched here with no luck...

[axillent]

@BulldogLowell which version of EthernetGateway you are running?
does ping stops responding?

I'm running 1.4.1 without any issues

atmega contains a hardware watchdog without actual need of external one.
With mysensors 1.3 I made an AVR version with support of the atmel's watchdog
I think it can be a good idea to add watchdog support to the official arduino version of MySensors library.

[BulldogLowell]

@axillent said:

@BulldogLowell which version of EthernetGateway you are running?
does ping stops responding?

I'm running 1.4.1 without any issues

atmega contains a hardware watchdog without actual need of external one.
With mysensors 1.3 I made an AVR version with support of the atmel's watchdog
I think it can be a good idea to add watchdog support to the official arduino version of MySensors library.

Also 1.4.1 While it vastly improved my performance across the board, I still get the (every two to three weeks) occasional lock-up. I can ping the gateway, but radio communication is kaput.

I agree that we should include the watchdog functionality into both the Serial and Internet versions of gateway code. What can I do to help?

I'll try the:

wdt_enable(); 

and we shall see if it tis the arduino locking up or the communication somewhere...

[axillent]

@BulldogLowell wdt_enable alone cannot be sufficient

first of all you need a bootloader having watchdog support
official bootloader is not working with watchdog
optiboot is working
other alternative is to kill a bootloader but you will need ISP programmer to download sketches

second wdt_reset need to be embedded into many places across sketch and library sources

[BulldogLowell]

@axillent said:

second wdt_reset need to be embedded into many places across sketch and library sources

OK, so for now it's a hardware plan!

[axillent]

@BulldogLowell why not to play around with your own copy of the sources?

with success you can be a contributor of this nice feature

[tekka]

@axillent said:

second wdt_reset need to be embedded into many places across sketch and library sources

It's in general good practice to enable the watchdog to avoid deadlocks. What works best in my experience is to set the wdt to 8s and reset the wdt timer at the beginning of the main loop (as long as it takes max. 8s).

Important: Avoid blocking functions and delay() - substitute delay() for mysensors::wait(), where the wdt is reset.

The problems with the ethernet shield W5100 are discussed everywhere, resetting the arduino won't help in that case, but rather implementing an external watchdog that power-cycles the shield...

[BulldogLowell]

@axillent said:

you can be a contributor of this nice feature

OK, I'll give it a stab.

@tekka said:

What works best in my experience is to set the wdt to 8s and reset the wdt timer at the beginning of the main loop (as long as it takes max. 8s).

you mean generally, or you are using it already on this application?

[tekka]

@BulldogLowell both, i'm using the wdt with Optiboot and MYSBootloader. Reading MCUSR during bootloading will give a hint on the reset cause...

[tekka]

Here a simple sketch:

#include <avr\wdt.h>

void setup() {
  // set watchdog to 8s
  wdt_enable(WDTO_8S);
}

void loop() {
  // watchdog reset
  wdt_reset();
  
  // do some stuff that does not take longer than 8s
 // else watchdog will trigger
}

[BulldogLowell]

@tekka said:

Here a simple sketch:

yes, got that far ;) I thought you meant you had it working on a gateway.

Working with the (as @axillent said) the gateway, with all of its nuances and making it extensible for (the majority?) of the standard sensor sketches would be the challenge.

[axillent]

@BulldogLowell setting watchdog to timeout after 8 seconds can support you in most situations without a need to dig into library source

But you will be not guaranteed from all cases, it is possible that your arduino will be reset in normal situation where reset is not needed. But actually the same will happen if you will attach external watchdog as you state in you first post here

[BulldogLowell]

@axillent said:

But you will be not guaranteed from all cases, it is possible that your arduino will be reset in normal situation where reset is not needed. But actually the same will happen if you will attach external watchdog as you state in you first post here

I have a lot more flexibility with hardware, i believe.... there is an (8s) limit on the Atmel watchdog, no?

[axillent]

@BulldogLowell said:

I have a lot more flexibility with hardware, i believe.... there is an (8s) limit on the Atmel watchdog, no?

it is a limit of atmega328, other MCU can differ

but it is still possible to organize longer period by activating WDT ISR handler
this will be not the best practice but it will work
the best practice is to set very short timeout and to put wdt_reset() at all required points

[hek]

What happens with watchdog while sleeping?

[axillent]

@hek said:

What happens with watchdog while sleeping?

it depends on what do you want :)
if i'm not mistaken LowPower library you are using is using watchdog while sleeping
actually watchdog is only one timer running while POWER_DOWN
it is a common way to wake up because of watchdog event from the deepest sleep if you need to wake up by time, not by external event

[Gambituk]

Hi, i would be interested in any more updates along this line. In my case my gateway is mostly stable as far as i can tell. several weeks without issues. My problem is that most of my nodes seem to lockup, so i would be totally interested in making them more robust either by software or if necessary with external chip as overlord watchdog

[Dan S.]

@BulldogLowell Any progress on this? My Ethernet gateway is quite reliable after solving power supply issues and implementing soft spi. But about once a month it stops and rebooting it makes it operative again. The failure seems to be periodic--once a month. May be a clue there--something reaching a limit? At any rate, any failure rate where I can't start it up without physically rebooting is too much.

[maha]

millis() has a rollover after approximately 50 days. Could that be something?

[Dan S.]

@maha Don't know. But your question did trigger the thought of using milils() to periodically trigger a reboot. Rather than check for a lockup with a watchdog routine I may try just rebooting it at a shorter time interval, e.g., every 2 weeks, than I have experienced the lockups using millis to measure the interval.