@MasMat : I just forgot, that I reconfigured rsyslogd to log mysgw to a different file. syslog is correct.

I do not use mqtt for mysensors yet, but the syslog seems good?

@MasMat : what does your mysgw.log show?

Yesterday I cloned the latest version on a recent raspbian (9.3) and I applied the first patch of @marceloaqno .
We will see. So long there were no problems. I will send an update in a few days...
As a last help I could switch to mqtt and do a restart after a defined amount of time (if no events are received)....

Cheers,
Otto

@marceloaqno :
Hi, unfortunately the latest patch did not work for me.
Still the gw stops receiving, but without high cpu usage :-(
Fortunately I can restart the gw now using a button and this udp-scripts :-)

Of cource I could give dietpi a try, but I would think, that a general solution would be better (for other users as well)

cheers,
Otto

@marceloaqno :
THANKS! Just applied your patch.
I will report in a few days when I can see if the problem is solved...
Cheers,
Otto

@gohan :
dietpi looks interesting! did not know about this yet.
what hardware are you using? and what version of mysgw?

@marceloaqno :
Thank you!!! Should this patch be applied together with your first patch?
Interesting:
After some days without problems on the newest raspbian packages the gw hung again, but WITHOUT high cpu usage (and WITHOUT your first patch applied). There were just no more reads from the sensors in the log, only the re-connects of fhem. After restarting the service, everything is fine again.
Unfortunately I did see your post after that :-(
But I checked the log and could not find the words "too" or "Too" in the log (running service with -d).
Some minutes after restarting I checked how many open files there were with the command you supplied: 21

In the meantime I have created a small udp listening service which restarts the service if a certain string is received, so I can restart the service by double-clicking one of my switches without the need of login to the gw (also for the wife)....

@gieljnssns : I think you are experiencing exactly the same problem we are discussing :-)

Thanks again!
Cheers,
Otto

@marceloaqno : Thank you!
Just a notice: I do NOT disconnect, I just connect every 30 minutes. Has been working for me since more than 2 years (ancient ms-gw version on RPi).

Yesterday I did a dist-upgrade to stretch on the "problematic" gw (as mentioned above) - without your patch but with my monitoring-script running. It is a little bit to early I presume, but no high cpu problems so far. I will report back here in a few days. Maybe this problem is really related to old libraries or something?! If this is running stable for a few days I will apply your patch and see what happens.
Thanks again!

Cheers,
Pula

@marceloaqno : This is strange. I am using a RPi 1 on this gateway with raspbian Jessie. Beside the GW-service there is only a small python programm which listens on an UPD-Port and plays an mp3 if it receives suitable data (for doorbell). Since March 4, 20:15 there were several restarts due to high cpu usage (after dis-applying your patch):

2018-03-04 23:01:03 cpu (99.7) very high! --> restarting service
2018-03-05 07:31:03 cpu (99.7) very high! --> restarting service
2018-03-05 11:01:03 cpu (98.7) very high! --> restarting service
2018-03-05 13:31:04 cpu (94.7) very high! --> restarting service
2018-03-06 11:31:03 cpu (99.7) very high! --> restarting service
2018-03-06 12:31:03 cpu (98.7) very high! --> restarting service
2018-03-07 06:01:03 cpu (94.8) very high! --> restarting service
2018-03-07 22:31:03 cpu (99.8) very high! --> restarting service

With this "auto-restarts" the sensors are working fine, I did not notice any problems.

However, I just upgraded to the recent Stretch (what takes a LONG time on RPi 1 and made me struggle a little bit with the new network names). I also rebuilt the GW of course and will report here what will happen....

@Nick-Willis : did you apply the patch of marceloaqno or not? Did you enable debugging? (see here )

Cheers,
Otto

@marceloaqno :

At first, THANKS for your help. CPU usage does not rise anymore.
But unfortunately no more node READs after the same situation:

Mar  4 07:00:09 raspiez mysgw: TSF:MSG:READ,14-14-0,s=255,c=3,t=6,pt=1,l=1,sg=0:0
Mar  4 07:00:09 raspiez mysgw: TSF:MSG:READ,12-12-0,s=255,c=3,t=6,pt=1,l=1,sg=0:0
Mar  4 07:00:09 raspiez mysgw: GWT:RFC:C=1,MSG=14;255;3;0;6;M
Mar  4 07:00:09 raspiez mysgw: !TSF:MSG:SEND,0-0-14-14,s=255,c=3,t=6,pt=0,l=1,sg=0,ft=0,st=NACK:M
Mar  4 07:00:09 raspiez mysgw: TSF:MSG:READ,11-11-0,s=255,c=0,t=18,pt=0,l=3,sg=0:1.5
Mar  4 07:00:09 raspiez mysgw: GWT:RFC:C=1,MSG=12;255;3;0;6;M
Mar  4 07:00:09 raspiez mysgw: TSF:MSG:SEND,0-0-12-12,s=255,c=3,t=6,pt=0,l=1,sg=0,ft=0,st=OK:M
Mar  4 07:00:13 raspiez mysgw: TSM:READY:NWD REQ
Mar  4 07:00:13 raspiez mysgw: TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Mar  4 07:00:13 raspiez mysgw: TSF:SRT:OK
Mar  4 07:00:13 raspiez mysgw: TSF:SAN:OK
Mar  4 07:00:37 raspiez mysgw: Ethernet client disconnected.
Mar  4 07:00:37 raspiez mysgw: New connection from 192.168.1.11
Mar  4 07:00:37 raspiez mysgw: GWT:TSA:C=0,CONNECTED
Mar  4 07:00:37 raspiez mysgw: GWT:TSA:C=1,DISCONNECTED

After this point (again after a reconnect from the controller) no more messages from any node PLUS the cpu usage stayed normal (what makes it even worse at this moment, as I would have to create a script which monitors the logfile and restarts the mysgw service in this situation). As soon as the service is restarted, everything is working fine again.

What happens after this point in the log:

Mar  4 07:00:37 raspiez mysgw: GWT:RFC:C=0,MSG=0;0;3;0;2;
Mar  4 07:15:13 raspiez mysgw: TSF:SAN:OK
Mar  4 07:20:13 raspiez mysgw: TSM:READY:NWD REQ
Mar  4 07:20:13 raspiez mysgw: TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Mar  4 07:30:13 raspiez mysgw: TSF:SRT:OK
Mar  4 07:30:13 raspiez mysgw: TSF:SAN:OK
Mar  4 07:30:37 raspiez mysgw: GWT:TSA:C=0,DISCONNECTED
Mar  4 07:30:37 raspiez mysgw: Ethernet client disconnected.
Mar  4 07:30:37 raspiez mysgw: New connection from 192.168.1.11
Mar  4 07:30:37 raspiez mysgw: GWT:TSA:C=0,CONNECTED
Mar  4 07:30:37 raspiez mysgw: GWT:RFC:C=0,MSG=0;0;3;0;2;
Mar  4 07:40:13 raspiez mysgw: TSM:READY:NWD REQ
Mar  4 07:40:13 raspiez mysgw: TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Mar  4 07:45:13 raspiez mysgw: TSF:SAN:OK
Mar  4 08:00:13 raspiez mysgw: TSM:READY:NWD REQ
Mar  4 08:00:13 raspiez mysgw: TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Mar  4 08:00:13 raspiez mysgw: TSF:SRT:OK
Mar  4 08:00:13 raspiez mysgw: TSF:SAN:OK
Mar  4 08:00:37 raspiez mysgw: GWT:TSA:C=0,DISCONNECTED
Mar  4 08:00:37 raspiez mysgw: Ethernet client disconnected.
Mar  4 08:00:37 raspiez mysgw: New connection from 192.168.1.11
Mar  4 08:00:37 raspiez mysgw: GWT:TSA:C=0,CONNECTED

and so on....

It seems that the increasing cpu usage is only a symptom for the real problem.
As I am using this in production I will dis-apply the patch and use the daemon as before (beacuse using the python script for cpu-usage-monitoring makes the situation somehow controllable for me).
Maybe you could have an additional eye on this please? Could I help you with further information? Maybe I could add some additional logging somewhere in the code?

Thanks again and cheers,
Otto

@marceloaqno : Wow, you are fast! Thanks!
Just applied the patch and recompiled. I will report in a couple of days while my monitoring-script is still running (and logging)...

Cheers,
Otto

@marceloaqno : of course, thanks for the hint!

I had the following configure:

./configure --my-transport=nrf24 --my-rf24-irq-pin=15 --my-gateway=ethernet --my-port=5004

(The high CPU problem also occured before I connected PIN 15 to IRQ without the corresponding config option. I hoped to solve the Problem using IRQ, but this did not help. And I have a second GW with default port 5003 in my network. This is running an ancient version of Raspi GW Software and this is working fine -but slower). I am using current fhem as controller and I have it connecting every 30 minutes to the gateways due to stability issues in the past. There are only about 6 nodes (just motion, humidity & temp and lux) connected to this gateway.

In addition to that I changed the default channel in MyConfig.h (because I have two different MySensor networks):

#ifndef MY_RF24_CHANNEL
#define MY_RF24_CHANNEL (82)
#endif

I am pretty sure, that this is not important, but I also set up rsyslog to write gw logs to an own file instead of syslog....

Maybe this helps.

Cheers,
Otto

@rejoe2 :
Thanks a lot!
I am having some unusual sketches (washing mashine monitoring, entry-system with fingerprint) where I am using self-written sketches with mysensors). Maybe I should wait. It is just annoying that the radio-stuff is not always working as it should. Indeed, esp with mqtt could be a solution too. But I do not know yet about the stability of esp-stuff :-(

Cheers,
Otto

@Nick-Willis:
I do run the gateway in debug mode, but I could not see anything special.
Yesterday the service was auto-restarted by my script:

2018-02-27 13:31:03 high CPU-usage! --> restarting service```

On the same time in the "normal" mysgw.log (debugging enabled) (should be syslog normally, but I changed the logfile location for better investigation):

Feb 27 13:30:36 raspiez mysgw: TSF:MSG:READ,14-14-0,s=1,c=1,t=23,pt=7,l=5,sg=0:0.11
Feb 27 13:30:36 raspiez mysgw: TSF:MSG:READ,14-14-0,s=3,c=1,t=0,pt=7,l=5,sg=0:21.9
Feb 27 13:30:36 raspiez mysgw: TSF:MSG:READ,14-14-0,s=4,c=1,t=1,pt=7,l=5,sg=0:31.4
Feb 27 13:30:38 raspiez mysgw: New connection from 192.168.1.11 <-- fhem re-connects
Feb 27 13:30:38 raspiez mysgw: GWT:TSA:C=1,CONNECTED
Feb 27 13:31:04 raspiez mysgw: Received SIGTERM <-- AUTO-Restart

The only thing that is a little bit unusual is the re-connect of my controller (fhem) - but this happens every 30 minutes for stability reasons.
This should not happen, but I can not find out what the problem is.

Maybe you could also enable debugging by changing the following file (maybe we can find a pattern why this is happening):

/etc/systemd/system/mysgw.service

from

ExecStart=/usr/local/bin/mysgw

to

ExecStart=/usr/local/bin/mysgw -d

Please do not forget to issue a

sudo systemctl daemon-reload 

followed by

sudo systemctl restart mysgw.service

to enable the change.

My restarting-script might help until someone with better knowledge finds a solution...
I also will have an eye on the logs for findig out how often this is happening...

Cheers,
Otto

Hi,

I am running the current version (2.2.0) on a RPi 1.
I noticed that sometimes the cpu usage of the service is increasing dramatically (around 90% and more) - and then the service stops working.
As I am not a C-Pro, I wrote a little python-script, which is called every minute by cron and restarts the service, if it uses more than 50% cpu.
Maybe someone will find it useful:

# -*- coding: utf-8 -*-
import psutil, sys, os, datetime

PY3 = sys.version_info[0] == 2
LOOKFOR = u"mysgw"
for proc in psutil.process_iter():
    if proc.name() == "mysgw":
        dt = str(datetime.datetime.now()).split('.')[0]
        #print(proc)
        cpu = proc.cpu_percent(interval=1)
        #print(cpu)
        #print(proc.cpu_times())
        if cpu > 50:
                print(dt + " high CPU-usage! --> restarting service")
                os.system("/bin/systemctl restart mysgw.service")
        else:
                if datetime.datetime.now().strftime('%M') == "45":
                        print(dt + " everything OK, cpu: " + str(cpu))

and the crontab-entry (as root):

* * * * * /home/pi/pstest.sh >> /var/log/mysgw-logger.log

Cheers,
Pula

I am thinking about changing my Mysensor-Nodes from NRF24L01+ to RS485 because of stability. But while reading this, I am not sure anymore, if this is a good idea
Anyone has this up & running successfully already?

Hi again,

and thanks again. signing seems to work now, unfortunately the sketch is too big for an UNO and I am right now struggling with the pinout and code adaptions to get it working with my W5100 shield and a mega...
/edit: got it working.

MyConfig.h:

#ifdef SOFTSPI
    // Define the soft SPI pins used for NRF radio
    //MEGA
    const uint8_t SOFT_SPI_MISO_PIN = 15;
    const uint8_t SOFT_SPI_MOSI_PIN = 14;
    const uint8_t SOFT_SPI_SCK_PIN = 16;
#endif

EthernetGateway.ino:

#define RADIO_CE_PIN        35 // 5  // radio chip enable
#define RADIO_SPI_SS_PIN    34 // 6  // radio SPI serial select

And exactly those pins I connected from radio to the mega... (just in case that anyone else is in need of this)

Might be a little OT, but maybe someone here knows a solution for a strange problem:
Whenever I am using an UNO or MEGA with my W5100 shield, I have to reset after every power-on to make the sketch run. Does anyone know this? Is there a solution? I am trying to adapt optiboot to wait additional 500ms when powering on, but no luck so far...

Cheers,

Otto

Did someone get the GW with mega and W5100 working????

@tomkxy Thanks, that helped! I seemed to have two problems:

1. I did not know where to place the necessary includes! Thanks for the examples! Helped me a lot

2)Regarding my problems from above (error: 'DEFAULT_CE_PIN' was not declared in this scope): I removed the whole libraries/MySensors dir and copied it again from the recent devel branch - this helped, error is gone and everything is compiling fine now :-)

Thanks again and cheers,

Otto

Hi,

this might be a little OT, cause it is not directly related to mysensors, so sorry.
But maybe someone knows this problem or even a solution.

I got some UNO R3 boards and W5100 ethernet shields on ali.
Uploaded some sketch and everything worked fine as long as I was using the USB cable connected to my pc.
But then I wanted to go in production and used different power sources:
A 9V adaptor connected to the power plug, even with a DC-DC module with voltage from 5V up to 9V, 5V directly to the USB port of the arduino, but nothing worked.
What the arduino does when I turn power on or connect it to the already plugged in power source: The power led is on and the TX LED is blinking several times. Until I press the reset button on the ethernet shield - then the arduino is booting the sketch and everything is fine.
Does anyone know this problem? Some hints would be really fine!

Cheers,

Pula