ESP8266 Gateway (RFM69) Soft WDT reset every 5 - 10 min

[Stefan_NE]

Hello,

my Gateway connected to FHEM was runnig fine, but now it Soft WDT reset comes up ervery 5 to 10 minutes.

23:28:26.427 -> 47243 TSF:MSG:READ,59-59-0,s=97,c=1,t=47,pt=2,l=2,sg=0:-62
23:28:34.556 -> 55377 GWT:TSA:C=0,CONNECTED
23:28:34.605 -> 55427 GWT:RFC:C=0,MSG=0;0;3;0;2;
23:29:06.330 -> 87131 TSF:MSG:READ,56-56-0,s=99,c=1,t=2,pt=1,l=1,sg=0:1
23:29:06.377 -> 87215 TSF:MSG:READ,56-56-0,s=97,c=1,t=47,pt=1,l=1,sg=0:147
23:29:07.533 -> 88331 TSF:MSG:READ,56-56-0,s=98,c=1,t=47,pt=1,l=1,sg=0:185
23:29:13.186 -> 93993 GWT:RFC:C=0,MSG=53;1;1;1;2;1
23:29:15.261 ->
23:29:15.261 -> Soft WDT reset
23:29:15.261 ->
23:29:15.261 -> >>>stack>>>
23:29:15.307 ->
23:29:15.307 -> ctx: cont
23:29:15.307 -> sp: 3ffffc80 end: 3fffffc0 offset: 01a0

it looks like, if there are to much messages to repeat (Sensor Device powerd off e.g.) the gateway resets.
09:49:14.768 -> 313389 !TSF:MSG:SEND,0-0-53-53,s=1,c=1,t=2,pt=0,l=1,sg=0,ft=0,st=NACK:0
09:49:15.469 -> 314073 GWT:RFC:C=0,MSG=57;1;1;0;2;0
09:49:16.725 -> 315331 !TSF:MSG:SEND,0-0-57-57,s=1,c=1,t=2,pt=0,l=1,sg=0,ft=0,st=NACK:0
09:49:16.825 -> 315406 GWT:RFC:C=0,MSG=58;1;1;0;2;0
09:49:18.074 -> 316665 !TSF:MSG:SEND,0-0-58-58,s=1,c=1,t=2,pt=0,l=1,sg=0,ft=0,st=NACK:0
09:49:18.128 -> 316740 GWT:RFC:C=0,MSG=51;1;1;1;2;0
09:49:19.376 -> 318000 !TSF:MSG:SEND,0-0-51-51,s=1,c=1,t=2,pt=0,l=1,sg=0,ft=0,st=NACK:0
09:49:28.051 -> 326647 GWT:RFC:C=0,MSG=53;1;1;1;2;0
09:49:29.299 -> 327917 !TSF:MSG:SEND,0-0-53-53,s=1,c=1,t=2,pt=0,l=1,sg=0,ft=0,st=NACK:0
09:49:36.070 -> 334647 GWT:RFC:C=0,MSG=51;1;1;1;2;0
09:49:37.319 -> 335926 !TSF:MSG:SEND,0-0-51-51,s=1,c=1,t=2,pt=0,l=1,sg=0,ft=0,st=NACK:0
09:49:45.037 -> 343647 GWT:RFC:C=0,MSG=53;1;1;1;2;0
09:49:46.493 -> 345105 !TSF:MSG:SEND,0-0-53-53,s=1,c=1,t=2,pt=0,l=1,sg=0,ft=0,st=NACK:0

I have already checked with a second Gateway that there is no hardware defect. Both gateways same behavior.
Myssensors is 2.3.2 ESP Lib is the newest.

Does anybody know this problem ?

Have a nice day
Stefan

[refh href]

My loads up crash every once in a while and I don't have the foggiest idea why. After such accident the wd reset appears (boot mode:(1,6)), yet the board doesn't restart consequently. How might I make the sheets reset/restart naturally after a wdt reset?

[TRS-80]

Someone else was having some problems also with similar hardware (ESP8266 + RFM69) recently, I don't know yet whether those problems are related or not, just cross posting in case this turns into some sort of investigation.

I don't use this hardware, but I will try to assist nonetheless.

First things to check are the basics:

1. Sufficient / reliable power supply?
2. What changed in the meantime?

[Stefan_NE]

Hi,
the problem was instantaneous. Power supply, ESp hardware, antenna I have renewed. All nodes ( which were possible ) rebooted. What helped was to reduce the traffic to the gate. The reboots are less but not gone.
Meanwhile I tried the ESP32 instead of the ESP8266 ( big crap ), then a gateway with a Raspberry Pi ( I didn't think it through, only a new RFm 69 driver and the Radio ID I can't set )
I think the project is really great, I have almost 60 Mysensors devices in use. The sensors work fine in almost every vesrion.
A real problem is the stability of the ESP8266 Gateway. I started with NFR24, the range was too bad. Now RFM 69 range is great. I have two gateways in use. The problems with the gateway are real ... Bu..Sh.t.
My controller runs on a VM so only LAN gateways work. Arduino and RFM69 with ethernet does not work with Soft Spi.Raspberry only new RFM69 driver.
This leaves only ESP8266 and RFM69 ...
Lets see what is comming up next.

Have a nice day Stefan

[Oumuamua]

Hi Stefan,

I'm also facing a similar problem (here), but starting with the second sensor added (not 60 as you).

Do you know what is the reset cause? Mine is rst cause:4, boot mode:(3,7)

In my case I need to switch off all sensors and turn them on only after rebooting the gateway.

Will follow this post to check if there is any help for me.

Thanks,

[Stefan_NE]

Hi,
I don't have a solution yet. It's almost impossible to shut down all my sensors. What I can say for sure is that there is a lot of traffic on the gateway.
My solution is to redesign the entire environment. I have built a gateway with a Raspberry Pi 3B and RFM69 (new driver) and I am moving the most important devices first. I adapted the code (new rfm69 driver and less traffic).
During the last days I have done traffic test with very high load on the Pi, very stable.
The first 15 devices are running on the new gateway today. So far very good.
On the old gateway the error is still present without the 15 devices.
I am curious with which device is disconnected from the ESP gateway the error will go.

Have a nice Day Stefan

[Sasquatch]

Try adding 220-470uf capacitor across radio supply pins, had similar problem with my gateway(NodeMCU) and RFM69HW during node OTA firmware updates.

You know you can map serial port to VM, right?

[Oumuamua]

@Sasquatch

Hi Sasquatch,

I'm new to mysensors and arduino and not an electrical engineer. Could you the scheme on how to add the capacitor? I really appreciate any "for dummy" explanation.

Thanks!

[Sasquatch]

https://www.mysensors.org/uploads/57c1a2411730a6c715b6b1b0/image/RFM69.png
Solder it between GND and 3v3 pins.
sorry, can't find a picture on internet and can't take picture of mine atm.

[Oumuamua]

@Sasquatch

Thanks! It helped!!

[TRS-80]

@Oumuamua said in ESP8266 Gateway (RFM69) Soft WDT reset every 5 - 10 min:

Could you the scheme on how to add the capacitor?

We got into pretty good discussion of capacitors in my evidence based testing thread. I had read a lot that one larger + one smaller cap actually works the best. I am no EE either, so I tested several different combinations, the results you can see there along with a very big close up picture I took of how to solder the caps onto the radio.

[TRS-80]

@Stefan_NE said in ESP8266 Gateway (RFM69) Soft WDT reset every 5 - 10 min:

What I can say for sure is that there is a lot of traffic on the gateway...

It sounds to me like you are on the right track. Proceeding in an organized methodical fashion. I am sure you will get it sorted out this way. And thanks for sharing your progress in the meantime, it will likely help others later.

I am not even up to that many nodes myself yet either, so I'd be interested in anyone's feedback who is, because I do plan to have that many nodes (or more) "some day."

So I got wondering if there have already been any forum threads about large numbers of nodes, and I found this one where some "back of envelope" calculations stated that some thousands (actually 10s of thousands) of nodes were (at least theoretically) possible...

[Stefan_NE]

Hi,

I have installed 2 new Gateway with a Raspberry Pi and RFM69. During the last days I have moved one device after the other to the new gateways. Of course I have adapted all devices with a new firmware. The two old ESP8266 gateways were in monitoring,. What I noticed was that the less devices and less traffic, the less reboots could be detected, regardless of the Mysensor version started at 1.5 to 2.32. . The new RPi gateways are running very cool so far. The best thing is that you can monitor everything, traffic to the LAN and traffic to the RFM69.

Have a nice day
Stefan

[Sasquatch]

just a thought, since resets are happening under heavy trafffic:
What is your serial speed? 9600 could lead to buffer overflows under heavy traffic