💬 Building a Raspberry Pi Gateway

zboblamont

@pepson Let's try the baby steps I referred to earlier.

The current stable version of MySensors is 2.1. Those using 2.2 tend to be experienced people versed in programming and experimenting and problem solving, hence it is not an official release.... yet you are using it and asking for solutions... Hello ?

I am and have been for the last few months using a Pro-Mini with RFM69 as nodes and gateway using 2.1 attached to a Pi3 running Domoticz.... No problems, but slowly coming to understand most of what I am doing...

I am still on baby steps, I don't mind at all, it works.....

pepson

Ok but i dont know how build Gateway on version 2.1.1 because when i do step by step as show on mysensors.org it default build me version gateway 2.2.0-rc2

zboblamont

@pepson As in this page? 2.1.1 - Latest Release ?
https://www.mysensors.org/download

pepson

I found bug in Domoticz BETA 8796. No reconized version gateway in HARDWARE. On stable 8153 works with no problem and correct reconized version gateway 2.2.0-rc2...
And again problem with this:
CODE: SELECT ALL

2017-12-28 20:01:32.587 MySensors: retrying in 30 seconds...
2017-12-28 20:02:08.752 MySensors: Using serial port: /dev/serial0
2017-12-28 20:02:08.752 MySensors: Gateway Version: 2.2.0-rc.2
2017-12-28 20:02:08.752 Error: Serial Port closed!... Error: End of file
2017-12-28 20:02:09.752 MySensors: retrying in 30 seconds...
2017-12-28 20:02:38.755 MySensors: Using serial port: /dev/serial0
2017-12-28 20:02:38.755 Error: Serial Port closed!... Error: End of file
2017-12-28 20:02:39.755 MySensors: retrying in 30 seconds...
2017-12-28 20:03:08.757 MySensors: Using serial port: /dev/serial0
2017-12-28 20:03:08.761 Error: Serial Port closed!... Error: End of file
2017-12-28 20:03:09.758 MySensors: retrying in 30 seconds...
On beta is problem.
On stable works correct.

pepson

This post is deleted!

pepson

@macvictor said in Building a Raspberry Pi Gateway:

@marceloaqno Thank's for you help. I have other question.. general I try configure my gateway to work with security & signing, but they doesn't work. Where is problem? On the bottom i send point what it do it:

Gateway:

./configure --my-gateway=mqtt --my-controller-ip-address=127.0.0.1 --my-mqtt-publish-topic-prefix=mysensors-out --my-mqtt-subscribe-topic-prefix=mysensors-in --my-mqtt-client-id=mygateway1 --my-mqtt-password=PASS --my-mqtt-user=USER --my-rf24-encryption-enabled --my-signing=software --my-signing-request-signatures --my-signing-request-gw-signatures-from-all
make
sudo make install
sudo systemctl enable mysgw.service
sudo systemctl start mysgw.service
sudo mysgw --gen-soft-hmac-key
sudo mysgw --set-soft-hmac-key=02417628BC1573CC380...
sudo mysgw --gen-soft-serial-key
sudo mysgw --set-soft-serial-key=00762CD152B868F7EB
sudo mysgw --gen-aes-key
sudo mysgw --set-aes-key= A790AD78B2D8B4DBBB5B2C4B9DD6DAF3

Node:

Load script SecurityPersonalizer.ino
Uncomment and send sketch to arduino

#define USE_SOFT_SIGNING
#define STORE_SOFT_KEY
#define USER_SOFT_KEY
#define STORE_SOFT_SERIAL
#define STORE_AES_KEY
#define USER_AES_KEY

#define MY_SOFT_HMAC_KEY 0X2,0X41,0X76,0X28,0XBC,0X15,0X73,0XCC,0X38...
#define MY_AES_KEY 0XA7,0X90,0XAD,0X78,0XB2,0XD8...

Read all value from EPROM (test save):

Personalization sketch for MySensors usage.
-------------------------------------------
EEPROM configuration:
SOFT_HMAC_KEY | 02417628BC1573CC380DE0FADF7C87D7D3C77272A783E6E438786A07C3A8C6CF
SOFT_SERIAL   | 39BB1370C289643251
AES_KEY       | A790AD78B2D8B4DBBB5B2C4B9DD6DAF3
--------------------------------
Personalization is now complete.

Send sample sketch to arduino:

//!< Enable debug prints to serial monitor
#define MY_DEBUG
//!< Enable signing related debug prints to serial monitor
#define MY_DEBUG_VERBOSE_SIGNING
// Enable NRF24L01 radio type
#define MY_RADIO_NRF24

// Select soft signing
#define MY_SIGNING_SOFT

// Enable lockdown of node if suspicious activity is detected
//-->> #define MY_NODE_LOCK_FEATURE

// Enable node whitelisting
//#define MY_SIGNING_NODE_WHITELISTING {{.nodeId = GATEWAY_ADDRESS,.serial = {0x09,0x08,0x07,0x06,0x05,0x04,0x03,0x02,0x01}}}

// Enable this if you want destination node to sign all messages sent to this node.
#define MY_SIGNING_REQUEST_SIGNATURES

// SETTINGS FOR MY_SIGNING_SOFT
#define MY_SIGNING_SOFT_RANDOMSEED_PIN 7 //!< Unconnected analog pin for random seed

// Add repeater function to this sensor
#define MY_REPEATER_FEATURE


#include <SPI.h>
#include <MySensors.h>
#include <Bounce2.h>

#define RELAY_PIN  4    // Arduino Digital I/O pin number for relay 
#define BUTTON_PIN  3   // Arduino Digital I/O pin number for button 
#define CHILD_ID 1      // Id of the sensor child
#define RELAY_ON 1
#define RELAY_OFF 0

Bounce debouncer = Bounce();
int oldValue = 0;
bool state;

MyMessage msg(CHILD_ID, V_LIGHT);

void setup()
{
  // Setup the button
  pinMode(BUTTON_PIN, INPUT);
  // Activate internal pull-up
  digitalWrite(BUTTON_PIN, HIGH);

  // After setting up the button, setup debouncer
  debouncer.attach(BUTTON_PIN);
  debouncer.interval(5);

  // Make sure relays are off when starting up
  digitalWrite(RELAY_PIN, RELAY_OFF);
  // Then set relay pins in output mode
  pinMode(RELAY_PIN, OUTPUT);

  // Set relay to last known state (using eeprom storage)
  state = loadState(CHILD_ID);
  digitalWrite(RELAY_PIN, state ? RELAY_ON : RELAY_OFF);
}

void presentation()  {
  // Send the sketch version information to the gateway and Controller
  sendSketchInfo("Relay & Button", "1.0");
  
  // Register all sensors to gw (they will be created as child devices)
  present(CHILD_ID, S_LIGHT, "Switch Secure Sensor", false);
}

/*
   Example on how to asynchronously check for new messages from gw
*/
void loop()
{
  debouncer.update();
  // Get the update value
  int value = debouncer.read();
//  if (value != oldValue && value == 0) {
    send(msg.set(state ? false : true), true); // Send new state and request ack back
//  }
//  oldValue = value;
  delay(5000);
}

void receive(const MyMessage &message) {
  // We only expect one type of message from controller. But we better check anyway.
  if (message.isAck()) {
    Serial.println("This is an ack from gateway");
  }

  if (message.type == V_LIGHT) {
    // Change relay state
    state = message.getBool();
    digitalWrite(RELAY_PIN, state ? RELAY_ON : RELAY_OFF);
    // Store state in eeprom
    // saveState(CHILD_ID, state);

    // Write some debug info
    Serial.print("Incoming change for sensor:");
    Serial.print(message.sensor);
    Serial.print(", New status: ");
    Serial.println(message.getBool());
  }
}

And finally in Arduino Serial Monitor I see:

0 MCO:BGN:INIT REPEATER,CP=RNNRAS-,VER=2.1.1
4 TSM:INIT
5 TSF:WUR:MS=0
11 TSM:INIT:TSP OK
13 TSF:SID:OK,ID=105
15 TSM:FPAR
16 Will not sign message for destination 255 as it does not require it
58 TSF:MSG:SEND,105-105-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
2065 !TSM:FPAR:NO REPLY
2067 TSM:FPAR
2068 Will not sign message for destination 255 as it does not require it
2110 TSF:MSG:SEND,105-105-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
4117 !TSM:FPAR:NO REPLY
4119 TSM:FPAR
4120 Will not sign message for destination 255 as it does not require it
4162 TSF:MSG:SEND,105-105-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
6169 !TSM:FPAR:NO REPLY
6171 TSM:FPAR
6172 Will not sign message for destination 255 as it does not require it
6214 TSF:MSG:SEND,105-105-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
8221 !TSM:FPAR:FAIL
8222 TSM:FAIL:CNT=1
8224 TSM:FAIL:PDT

and in gateway i see:

sudo mysgw -d
mysgw: Starting gateway...
mysgw: Protocol version - 2.1.1
mysgw: MCO:BGN:INIT GW,CP=RNNG-S-,VER=2.1.1
mysgw: TSF:LRT:OK
mysgw: TSM:INIT
mysgw: TSF:WUR:MS=0
mysgw: TSM:INIT:TSP OK
mysgw: TSM:INIT:GW MODE
mysgw: TSM:READY:ID=0,PAR=0,DIS=0
mysgw: MCO:REG:NOT NEEDED
mysgw: MCO:BGN:STP
mysgw: MCO:BGN:INIT OK,TSP=1
mysgw: Attempting MQTT connection...
mysgw: connected to 127.0.0.1
mysgw: MQTT connected
mysgw: Sending message on topic: mysensors-out/0/255/0/0/18
mysgw: TSF:MSG:READ,171-107-54,s=221,c=2,t=131,pt=4,l=25,sg=0:-1418183441
mysgw: !TSF:MSG:PVER,3=2
mysgw: TSF:MSG:READ,135-23-172,s=20,c=3,t=200,pt=7,l=25,sg=0:-0.00000178
mysgw: !TSF:MSG:PVER,1=2
mysgw: TSF:MSG:READ,34-66-89,s=168,c=1,t=9,pt=1,l=25,sg=0:56
mysgw: !TSF:MSG:PVER,1=2
mysgw: TSF:MSG:READ,223-46-240,s=112,c=5,t=192,pt=5,l=11,sg=1:1407647670
mysgw: TSF:MSG:REL MSG
mysgw: !TSF:RTE:240 UNKNOWN
mysgw: !TSF:MSG:SEND,223-0-240-240,s=112,c=5,t=192,pt=5,l=11,sg=1,ft=0,st=NACK:1407647670
mysgw: TSF:MSG:READ,183-139-129,s=37,c=6,t=155,pt=1,l=21,sg=0:48
mysgw: !TSF:MSG:PVER,0=2
mysgw: TSF:MSG:READ,139-214-233,s=61,c=3,t=193,pt=7,l=0,sg=1:0.00000000
mysgw: TSF:MSG:REL MSG
mysgw: !TSF:RTE:233 UNKNOWN
mysgw: !TSF:MSG:SEND,139-0-233-233,s=61,c=3,t=193,pt=7,l=0,sg=1,ft=0,st=NACK:0.00000000
mysgw: TSF:MSG:READ,83-33-7,s=179,c=2,t=27,pt=4,l=25,sg=1:-2057270722
mysgw: !TSF:MSG:PVER,3=2
mysgw: TSF:MSG:READ,110-148-67,s=113,c=0,t=158,pt=1,l=9,sg=1:251
mysgw: !TSF:MSG:PVER,1=2
mysgw: TSF:MSG:READ,75-209-102,s=204,c=7,t=25,pt=1,l=11,sg=1:52
mysgw: !TSF:MSG:PVER,0=2
mysgw: TSF:MSG:READ,184-209-235,s=58,c=1,t=140,pt=6,l=23,sg=1:1DC404A5401119FACF0000000000000000000000000000
mysgw: !TSF:MSG:PVER,3=2
mysgw: TSF:MSG:READ,172-203-51,s=140,c=2,t=173,pt=0,l=1,sg=1:?
mysgw: TSF:MSG:REL MSG
mysgw: !TSF:RTE:51 UNKNOWN
mysgw: !TSF:MSG:SEND,172-0-51-51,s=140,c=2,t=173,pt=0,l=1,sg=1,ft=0,st=NACK:?

Where is problem?

@macvictor
But how use itn on RFM69HW ? I dont see to be available #define MY_RFM69_ENABLE_ENCRYPTION... Only for #define MY_RF24_ENABLE_ENCRYPTION...

mfalkvidd

@pepson are you asking how to enable encryption on rfm69? (it is a bit unclear what you mean by "itn")

In that case, see https://www.mysensors.org/apidocs-beta/group__RFM69SettingGrpPub.html#ga17a7a919994e569f078e3787c2c4ecd2

pepson

@mfalkvidd

Yes Thanks

BTW
MySensors gateway as you see is problem with latest beta domoticz.

mfalkvidd

@pepson yes I saw that. Thanks for checking. Have you reported the problem to the Domoticz team?

pepson

@mfalkvidd
Yes write on forum Domoticz in section BUG

mfalkvidd

@pepson great!

The report is at https://www.domoticz.com/forum/viewtopic.php?f=6&t=21125&sid=e414507bb1ca4596e93cf885047275a9 in case someone needs to find it in the future.

ghiglie

Hello!
I'm trying to "make" on Alpine Linux in a Docker (ok, maybe it's useless, but way interesting!).

With a Debian image it's ok, but I'd like to use Alpine.

I get this error:

g++ -MT build/drivers/Linux/SPIDEV.o -MMD -MP -march=armv7-a -mtune=cortex-a7 -mfpu=neon-vfpv4 -mfloat-abi=hard -DMY_GATEWAY_LINUX -DMY_GATEWAY_MQTT_CLIENT -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DSPI_SPIDEV_DEVICE=\"/dev/spidev0.0\"  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/SPIDEV.cpp -o build/drivers/Linux/SPIDEV.o
In file included from /usr/include/sys/ioctl.h:7:0,
                 from drivers/Linux/SPIDEV.cpp:27:
drivers/Linux/SPIDEV.cpp: In static member function 'static uint8_t SPIDEVClass::transfer(uint8_t)':
drivers/Linux/SPIDEV.cpp:153:18: error: '_IOC_SIZEBITS' was not declared in this scope
  ret = ioctl(fd, SPI_IOC_MESSAGE(1), &tr);
                  ^
drivers/Linux/SPIDEV.cpp: In static member function 'static void SPIDEVClass::transfernb(char*, char*, uint32_t)':
drivers/Linux/SPIDEV.cpp:175:18: error: '_IOC_SIZEBITS' was not declared in this scope
  ret = ioctl(fd, SPI_IOC_MESSAGE(1), &tr);
                  ^
make: *** [Makefile:99: build/drivers/Linux/SPIDEV.o] Error 1
root@local-mysgw-alpine:/opt/MySensors3$ make
g++ -MT build/drivers/Linux/SPIDEV.o -MMD -MP -march=armv7-a -mtune=cortex-a7 -mfpu=neon-vfpv4 -mfloat-abi=hard -DMY_GATEWAY_LINUX -DMY_GATEWAY_MQTT_CLIENT -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DSPI_SPIDEV_DEVICE=\"/dev/spidev0.0\"  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/SPIDEV.cpp -o build/drivers/Linux/SPIDEV.o
eIn file included from /usr/include/sys/ioctl.h:7:0,
                 from drivers/Linux/SPIDEV.cpp:27:
drivers/Linux/SPIDEV.cpp: In static member function 'static uint8_t SPIDEVClass::transfer(uint8_t)':
drivers/Linux/SPIDEV.cpp:153:18: error: '_IOC_SIZEBITS' was not declared in this scope
  ret = ioctl(fd, SPI_IOC_MESSAGE(1), &tr);
                  ^
drivers/Linux/SPIDEV.cpp: In static member function 'static void SPIDEVClass::transfernb(char*, char*, uint32_t)':
drivers/Linux/SPIDEV.cpp:175:18: error: '_IOC_SIZEBITS' was not declared in this scope
  ret = ioctl(fd, SPI_IOC_MESSAGE(1), &tr);
                  ^
make: *** [Makefile:99: build/drivers/Linux/SPIDEV.o] Error 1

I'm surely missing a package... but can't state one. Any help?

@macvictor thanks for your post, it will help me a lot, I'm trying the same way.

mfalkvidd

@ghiglie https://lists.yoctoproject.org/pipermail/yocto/2016-January/028233.html seems to suggest that adding

#include <asm/ioctl.h>

would solve the problem. Could you try that?

sineverba

Was my fault!
**When copy and past command from word, for example, pay attention that software doesn't substitute the single "-" with a "-" longer.

I have a word with all commands, to replicate installation very fast. Was my fault (errr... Word fault.... but you understand). Have a nice day!

~~Hi,~~
~~I cannot go over the initialization of command.~~

RPI3 + Amplified antenna by GertSander (https://www.openhardware.io/view/17/Raspberry-Pi2-GPIO-interface-for-NRF24L01). Power from 5v/3A and power for antenna taken from both 5V via the 3.3v stepper down.

~~RPI3 is new, has about 12h (got yesterday).~~

Yesterday it did work at first attempt. Today I would replicate so I did delete the uSD and restart over. No way to get the gateway on work.

~~I did try 3 amplified antenna + 2 normal. No way.~~

~~I did enable from raspi-config the SPI (and only it).~~

~~It hangs on second line of output.~~

~~These are the logs:~~

sineverba@raspberrypi:~/MySensors $ ./configure --my-transport=nrf24 --my-gateway=serial --my-serial-is-pty --my-serial-pty=/dev/ttyUSB020
[SECTION] Detecting target machine.
  [OK] machine detected: SoC=BCM2837, Type=rpi3, CPU=armv7l.
[SECTION] Detecting SPI driver.
  [OK] SPI driver detected:BCM.
[SECTION] Detecting init system.
  [OK] init system detected: systemd.
[SECTION] Saving configuration.
[SECTION] Cleaning previous builds.
[OK] Finished.
sineverba@raspberrypi:~/MySensors $ make
gcc -MT build/drivers/Linux/log.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/log.c -o build/drivers/Linux/log.o
g++ -MT build/drivers/Linux/IPAddress.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/IPAddress.cpp -o build/drivers/Linux/IPAddress.o
g++ -MT build/drivers/Linux/noniso.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/noniso.cpp -o build/drivers/Linux/noniso.o
g++ -MT build/drivers/Linux/GPIO.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/GPIO.cpp -o build/drivers/Linux/GPIO.o
g++ -MT build/drivers/Linux/SPIDEV.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/SPIDEV.cpp -o build/drivers/Linux/SPIDEV.o
g++ -MT build/drivers/Linux/Print.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/Print.cpp -o build/drivers/Linux/Print.o
g++ -MT build/drivers/Linux/EthernetClient.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/EthernetClient.cpp -o build/drivers/Linux/EthernetClient.o
g++ -MT build/drivers/Linux/compatibility.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/compatibility.cpp -o build/drivers/Linux/compatibility.o
g++ -MT build/drivers/Linux/SerialPort.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/SerialPort.cpp -o build/drivers/Linux/SerialPort.o
g++ -MT build/drivers/Linux/Stream.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/Stream.cpp -o build/drivers/Linux/Stream.o
g++ -MT build/drivers/Linux/interrupt.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/interrupt.cpp -o build/drivers/Linux/interrupt.o
g++ -MT build/drivers/Linux/SerialSimulator.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/SerialSimulator.cpp -o build/drivers/Linux/SerialSimulator.o
g++ -MT build/drivers/Linux/SoftEeprom.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/SoftEeprom.cpp -o build/drivers/Linux/SoftEeprom.o
g++ -MT build/drivers/Linux/EthernetServer.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/Linux/EthernetServer.cpp -o build/drivers/Linux/EthernetServer.o
g++ -MT build/examples_linux/mysgw.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c examples_linux/mysgw.cpp -o build/examples_linux/mysgw.o
gcc -MT build/drivers/BCM/bcm2835.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/BCM/bcm2835.c -o build/drivers/BCM/bcm2835.o
g++ -MT build/drivers/BCM/BCM.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/BCM/BCM.cpp -o build/drivers/BCM/BCM.o
g++ -MT build/drivers/BCM/SPIBCM.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/BCM/SPIBCM.cpp -o build/drivers/BCM/SPIBCM.o
g++ -MT build/drivers/BCM/Wire.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/BCM/Wire.cpp -o build/drivers/BCM/Wire.o
g++ -MT build/drivers/BCM/RPi.o -MMD -MP -march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_SERIAL -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_LINUX_SERIAL_PTY=\"/dev/ttyUSB020\" -DMY_LINUX_IS_SERIAL_PTY  -Ofast -g -Wall -Wextra  -I. -I./core -I./drivers/Linux -I./drivers/BCM -c drivers/BCM/RPi.cpp -o build/drivers/BCM/RPi.o
g++ -pthread  -o bin/mysgw build/drivers/Linux/log.o build/drivers/Linux/IPAddress.o build/drivers/Linux/noniso.o build/drivers/Linux/GPIO.o build/drivers/Linux/SPIDEV.o build/drivers/Linux/Print.o build/drivers/Linux/EthernetClient.o build/drivers/Linux/compatibility.o build/drivers/Linux/SerialPort.o build/drivers/Linux/Stream.o build/drivers/Linux/interrupt.o build/drivers/Linux/SerialSimulator.o build/drivers/Linux/SoftEeprom.o build/drivers/Linux/EthernetServer.o build/examples_linux/mysgw.o build/drivers/BCM/bcm2835.o build/drivers/BCM/BCM.o build/drivers/BCM/SPIBCM.o build/drivers/BCM/Wire.o build/drivers/BCM/RPi.o
sineverba@raspberrypi:~/MySensors $ sudo ./bin/mysgw –d
[sudo] password for sineverba:
mysgw: Starting gateway...
mysgw: Protocol version - 2.2.0-rc.2

sineverba

Hi to all,
I'm testing an ethernet gateway on RPI3, with nrf24 + 1 node with a relay.

I want introduce security and signing. On Arduino gateway / node no problem at all, but, how to do on the RPI3 as gateway?

I would get / create / set a software serial on PI itself and whitelisting it on the node, after I will introduce the AES Key and signing.

Cannot find in documentation how do it on PI3. Thank you very much!

ghiglie

@sineverba said in Building a Raspberry Pi Gateway:

Hi to all,
I'm testing an ethernet gateway on RPI3, with nrf24 + 1 node with a relay.

I want introduce security and signing. On Arduino gateway / node no problem at all, but, how to do on the RPI3 as gateway?

I would get / create / set a software serial on PI itself and whitelisting it on the node, after I will introduce the AES Key and signing.

Cannot find in documentation how do it on PI3. Thank you very much!

Earlier in this forum... https://forum.mysensors.org/topic/4803/building-a-raspberry-pi-gateway/350

ghiglie

@mfalkvidd I'm trying today, but to compile correctly the Docker image I'll have to clone to my github account or at least locally. My Dockerfile for the "Alpine edition" for Hass.io is simply:

ARG BUILD_FROM
ARG BUILD_ARCH
ARG BUILD_DATE
ARG BUILD_REF
ARG BUILD_VERSION

FROM ${BUILD_FROM}

ENV CONFIG_PATH=/data/options.json
ENV APPDIR=/opt/MySensors

RUN apt-get update && apt-get install -y --force-yes \
    make \
	 g++ \
	 git \
	 && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* ~/.cache \
	 && git clone https://github.com/mysensors/MySensors.git --branch development $APPDIR


# Copy root filesystem
COPY rootfs /

#CMD [ "/run.sh" ]

I deleted the LABELS 'cause... the data is still too rawly "buried" from others.

The Configure and Make is in run.sh.

BTW, I still need to figure out how to start mysgw as service... Mainlyt I'd like to redirect the log: how can I do this safely?

And... can I use the rx/tx/err led feature on Raspi too (aside being in a Docker) ?

EDIT: the code add works!
Another question: I'm not getting far from this... what is it trying to do?

: Starting gateway...
: Protocol version - 2.2.0-rc.2
: MCO:BGN:INIT GW,CP=RNNGL---,VER=2.2.0-rc.2
: TSF:LRT:OK
: TSM:INIT
: TSF:WUR:MS=0

sineverba

@ghiglie Thank you, I did see that was for 2.1.1. Just playing with them!

alowhum

Does the Pi Gateway support the MY_SIGNING_SIMPLE_PASSWD option? I couldn't find anything about that. I've been gearing up to use that feature in my network, but I've also been playing with moving my gateway onto my Pi Zero W.

gohan

@alowhum If you want to just encrypt data, just add the encryption key in the myconfig.h file and compile it

MasMat

@gohan N00b question, but which steps from the step-by-step need to be redone to "compile it" after altering myconfig.h? Or can you just post the command lines?

gohan

As soon as you do the changes in myconfig.h before the MAKE, you are fine

sentur

Anyone tried using the new Pi Zero WH as a Raspberry Pi Gateway. Presumably by MQTT and using NRF24 radio?

@alowhum Have you been able to setup a gateway with your Raspberry Pi Zero?

MasMat

@sentur That's what I'm doing now. Except mine is Zero W. Soldered the pins on myself.

marceloaqno

@alowhum yes, to use MY_SIGNING_SIMPLE_PASSWD feature, add the following to your list of configure options:

... --my-signing=password --my-signing-password=<PASSWORD>

Richard van der Plas

Is there anyway where i can see if i have IRQ configured ?
ive build it a long time ago, but now upgraded to a nrf24 PA/LNA and have more an more sensors.

gohan

look inside makefile.inc in mysensors folder, that should contain the parameters you set in ./configure

Richard van der Plas

@gohanm, as it seems not ?, can i rebuild it easily? ```

SOC=BCM2837
CPPFLAGS=-march=armv8-a+crc -mtune=cortex-a53 -mfpu=neon-fp-armv8 -mfloat-abi=hard -DMY_RADIO_NRF24 -DMY_GATEWAY_LINUX -DMY_DEBUG -DLINUX_SPI_BCM -DLINUX_ARCH_RASPBERRYPI -DMY_PORT=5003
LDFLAGS=-pthread
PREFIX=/usr/local
CC=gcc
CXX=g++
BUILDDIR=build
BINDIR=bin
GATEWAY_DIR=/usr/local/bin
INIT_SYSTEM=systemd
SPI_DRIVER=BCM```

gohan

make a copy of it and make a ./configure with the IRQ enabled and see what you get

Richard van der Plas

@gohan make a copy of the makefile.inc and restart the ./configure command again you mean ?
ill try it tomorrow, too afraid to break things late in the evening

gohan

as far as you don't execute the MAKE command you are ok. Also make a copy of the mysgw executable just as backup if you never did it

Richard van der Plas

but ill have to do it sometime , just ./configure with all the options, build , test and install ?

gohan

if you only want to check if irq is enabled, otherwise you could just build it fresh with irq enabled and you can go back by replacing the mysgw executable

Richard van der Plas

@gohan ty

Richard van der Plas

This post is deleted!

Richard van der Plas

Moved to normal troubleshooting forum

Richard van der Plas

This post is deleted!

alowhum

@MasMat Yes I was able to build it on a Pi Zero. I did a fresh install on Dietpi. It worked fine when I didn't give any configuration options.

Now, however, I am still trying to run something on the PGIO pins in paralel to MySensors again, and this is not working. As I speak the Pi reboots every 5 minutes.. Not sure what to blame yet. Domoticz, PiGPIO, or MySensors.

Latest DietPi Stretch, latest Domoticz beta, latest MySensors beta.

gohan

@alowhum said in Building a Raspberry Pi Gateway:

PiGPIO

what are you using that for?

MasMat

@alowhum I'm not using the GPIO for anything else.
I had everything working (mysgw-MQTT, Mosquitto, Node-Red and OH2) for almost 6h. Then I moved the set-up and everything went south: cant get mosquitto to start, cant get mysgw to connect to that or another Mosquitto broker, OH2 seems to ignore everything...
Spent the last 24h trying to resurrect Mosquitto but it's beyond me what's wrong.

I remember reading that mysgw renders rest of the gpio unusable. Maybe that's giving you problems.

gohan

I am running 2 gateway instances, one nrf24 and one rfm69 so I guess you can run other things

alowhum

@gohan: but you had to do a lot of black magic to get there right?

gohan

not at all, I just followed the guide in the forum

alowhum

@masmat I got the Pi Zero W working fine with other things using the GPIO.

A fresh install helped
It could be that I now use the PiGPIO library for the project.
It could be the switch to another powersupply. I was using the official pi powersupply, but I suspect something was wrong with it. A cheap 3A chinese adapter is working fine.

Either way: it all works now.

I have a few n00bish questions about enabling encryption (without signing, so save space and ram):

Is putting an AES key in MyConfig.h really all I have to do? Don't I also have to put that in all the sensor nodes? And is that proces complicated, using a signing sketch and whatnot?
Can I have an 'opportunistic' network, where I have some encrypted nodes, and some unencrypted ones? If I could mix them this would make switching to encryption less stressful. I can't easily change two of my nodes.
If I change the network, I'd like to move the NRF to channel 100 (juist outside of the normal WiFi frequencies). Do I also have to set that in all nodes, or is setting it in the gateway enough?

Anticimex

@alowhum encryption is all or nothing. You can't leave one node without encryption if your gateway encrypts the messages it send.
And you will need to personalize your nodes to get the AES key into them. The documentation on security describes that process.

gohan

@alowhum if you put it in myconfig.h also on the arduino library you can just recompile the nodes and you are good.

Anticimex

@gohan not anymore. Only for pre 2.2.0 releases.

gohan

It is still working on mine

Anticimex

@gohan in other words, you use an old release

gohan

I can assure you I am using 2.2 from development as I am also running new driver. So unless you changed something in the last 3 weeks on the development branch it still works, unless the nodes are keeping the key saved.

Anticimex

@gohan have you verified your communications are actually encrypted?

Anticimex

@gohan If you see here, you will find that unless you use the simple password option, AES key is fetched from EEPROM: https://github.com/mysensors/MySensors/blob/28c4f3f19e7026f56fd9bbc61d566f770fd7a9d2/hal/transport/MyTransportRF24.cpp#L68
So either you are using the simple flag, nothing at all, or not using the key you think you use.

gohan

I ll have to check, but I put the key in the gateway myconfig.h

Anticimex

@gohan Perhaps on a rpi GW where there is no EEPROM, but I am talking about the nodes.

gohan

Well, if I have communication working it means the node should be encrypting data. I'll have to try with clear eeprom and try flashing sketch again

Anticimex

@gohan not if there is no encryption at all. You could try to use simple password option on a node with various passwords and see if it is detected by your gw at all.

gohan

@anticimex I guess you were right... the gw and node aren't using the AES key. So where am I supposed to add it in the gateway?

Anticimex

@gohan you set that with the configure arguments I suspect, but I can't see it in the documentation. @marceloaqno perhaps knows how the rPi port handles this. I'm on a phone so I can't check the code easily. But you should be able to find it in the transport driver for rPi. On the rPi it can probably be defined in the code as it does not have a personalizer.

gohan

@anticimex It doesn't read it from the mysonfig.h for sure as it is where I always put it

Anticimex

@gohan MyConfig.h does to my knowledge not have a define value for that. You cannot add new definitions to it and expect them to have an effect. If there is one already defined in there, as a comment or not, and it does not have an effect, please report a bug on it.

rajeev2301

@macvictor
I followed all the step that you mentioned in your post to establish the serial gateway. However I am still not able to receive the message from arduino. Could you please what to do?

import RPi.GPIO as GPIO
from lib_nrf24 import NRF24
import time
import spidev

GPIO.setmode(GPIO.BCM)

pipes = [[0xE8, 0xE8, 0xF0, 0xF0, 0xE1], [0xF0, 0xF0, 0xF0, 0xF0, 0xE1]]
radio = NRF24(GPIO, spidev.SpiDev())

radio.begin(0,8)

radio.setPayloadSize(32)
radio.setChannel(0x76)
radio.setDataRate(NRF24.BR_1MBPS)
radio.setPALevel (NRF24.PA_MAX)
radio.setAutoAck(True)
radio.enableDynamicPayloads()
radio.enableAckPayload()
radio.openReadingPipe(1 ,pipes[1])
radio.printDetails()
radio.startListening()
radio.GPIO.cleanup()

Please find below my raspberry code..can you please suggest , if there is any issue with code or am I missing anything ?

mfalkvidd

@rajeev2301 please don't split our effort to help you into multiple threads. It wastes people's time when some information is in one thread and some in another thread.

wes

To upgrade to the latest release, can I use git pull, or is it better to delete the directory and start again?

Anticimex

@wes you can pull. Just as long as you verify that your working directory is clean or only contains changes you intend to have. And you have the appropriate branch or tag checked out.

pepson

@macvictor
Hi
Is any chance to add security & signing for radio RFM69HW ? Please help me how ?

Anticimex

@pepson signing is radio agnostic. It has nothing to do with the radio.

sineverba

@anticimex @wes Can I please ask how to update? I mean, the right command to type.
I have the folder "MySensors" on documents of my user in Pi:

/home/sineverba/MySensors

Of course I have all installed and perfectly working now (I'm on the 2.2.0 rc2).

I need also to relaunch the ./config command?

Thank you to both and all!

Anticimex

@sineverba https://www.mysensors.org/download

Syna

Hello, I configure the raspberry gateway to use MQTT. I set mosquitto as the broker. The problem that I encounter is that when I subscribe to test if the gateway publish on the broker, I have empty message. So my first guess is that there is an issue somewhere which cause that only empty message are sent.

Here is my arduino code : https://pastebin.com/YKi4EAad
The RasperryPI3 config & logs : https://pastebin.com/Kjn8CC8A
The MQTT Client logs : https://pastebin.com/ZP8GrkTj

Sergio Rius

Regarding the communication issues described in the instructions page with the 2.2 stable version, is that still valid and still recommended to use the development branch?

gohan

Go for stable now, I need to remove that line from the page

Sergio Rius

@gohan Thank you verys muchos.

marceloaqno

@syna It is empty because it's a message of type I_ID_REQUEST that your node is sending. You need to set up a controller to distribute IDs or you can set it manually by adding this to your node's sketch:

#define MY_NODE_ID X

where X is a unique number from 1 to 254.

sineverba

Hi to all,
tonight I made update from 2.2.0 RC2 to 2.2.0 (stable) on my Raspberry PI3.
It is a live system (my heater control).

By the way the update is very simple.

First of all, backup or better CLONE your PI.

1 - Disable the daemon with

sudo systemctl stop mysgw.service && sudo systemctl disable mysgw.service

2 - Make a copy of the MySensors folder ( I made a copy with the name of version coming, so...)

cp -r MySensors MySensorsX.X.X(rc2)/

(the slash at the end means "It's a folder!"

3 - Remove the MySensors folder

sudo rm -r MySensors

4 - Git the "new" MySensors folder && configure it as usual. Remember to activate the daemon and see on Domoticz (for example) the gateway with new version

otto001

Hi,

I am running the current version (2.2.0) on a RPi 1.
I noticed that sometimes the cpu usage of the service is increasing dramatically (around 90% and more) - and then the service stops working.
As I am not a C-Pro, I wrote a little python-script, which is called every minute by cron and restarts the service, if it uses more than 50% cpu.
Maybe someone will find it useful:

# -*- coding: utf-8 -*-
import psutil, sys, os, datetime

PY3 = sys.version_info[0] == 2
LOOKFOR = u"mysgw"
for proc in psutil.process_iter():
    if proc.name() == "mysgw":
        dt = str(datetime.datetime.now()).split('.')[0]
        #print(proc)
        cpu = proc.cpu_percent(interval=1)
        #print(cpu)
        #print(proc.cpu_times())
        if cpu > 50:
                print(dt + " high CPU-usage! --> restarting service")
                os.system("/bin/systemctl restart mysgw.service")
        else:
                if datetime.datetime.now().strftime('%M') == "45":
                        print(dt + " everything OK, cpu: " + str(cpu))

and the crontab-entry (as root):

* * * * * /home/pi/pstest.sh >> /var/log/mysgw-logger.log

Cheers,
Pula

Nick Willis

@otto001 Thanks for that! I'm also getting this annoying service failure on 2.2 on a RPi 2. It seems to happen at random at night when nothing much is going on in the house. It probably happens once every 1 to 2 weeks.
CPU use is almost 100% and the service stops working.
Are there any logs I can check? I'd prefer to know what's wrong although I'll go with the auto restart on high cpu use to prevent the house from ceasing to function at random!

alowhum

I once noticed this too, on a pi zero system with domoticz beta.

otto001

@Nick-Willis:
I do run the gateway in debug mode, but I could not see anything special.
Yesterday the service was auto-restarted by my script:

2018-02-27 13:31:03 high CPU-usage! --> restarting service```

On the same time in the "normal" mysgw.log (debugging enabled) (should be syslog normally, but I changed the logfile location for better investigation):

Feb 27 13:30:36 raspiez mysgw: TSF:MSG:READ,14-14-0,s=1,c=1,t=23,pt=7,l=5,sg=0:0.11
Feb 27 13:30:36 raspiez mysgw: TSF:MSG:READ,14-14-0,s=3,c=1,t=0,pt=7,l=5,sg=0:21.9
Feb 27 13:30:36 raspiez mysgw: TSF:MSG:READ,14-14-0,s=4,c=1,t=1,pt=7,l=5,sg=0:31.4
Feb 27 13:30:38 raspiez mysgw: New connection from 192.168.1.11 <-- fhem re-connects
Feb 27 13:30:38 raspiez mysgw: GWT:TSA:C=1,CONNECTED
Feb 27 13:31:04 raspiez mysgw: Received SIGTERM <-- AUTO-Restart

The only thing that is a little bit unusual is the re-connect of my controller (fhem) - but this happens every 30 minutes for stability reasons.
This should not happen, but I can not find out what the problem is.

Maybe you could also enable debugging by changing the following file (maybe we can find a pattern why this is happening):

/etc/systemd/system/mysgw.service

from

ExecStart=/usr/local/bin/mysgw

to

ExecStart=/usr/local/bin/mysgw -d

Please do not forget to issue a

sudo systemctl daemon-reload

followed by

sudo systemctl restart mysgw.service

to enable the change.

My restarting-script might help until someone with better knowledge finds a solution...
I also will have an eye on the logs for findig out how often this is happening...

Cheers,
Otto

marceloaqno

Please, could all of you who experienced this issue post the configuration options you used to build the gateway?
This would help to investigate the problem.

otto001

@marceloaqno : of course, thanks for the hint!

I had the following configure:

./configure --my-transport=nrf24 --my-rf24-irq-pin=15 --my-gateway=ethernet --my-port=5004

(The high CPU problem also occured before I connected PIN 15 to IRQ without the corresponding config option. I hoped to solve the Problem using IRQ, but this did not help. And I have a second GW with default port 5003 in my network. This is running an ancient version of Raspi GW Software and this is working fine -but slower). I am using current fhem as controller and I have it connecting every 30 minutes to the gateways due to stability issues in the past. There are only about 6 nodes (just motion, humidity & temp and lux) connected to this gateway.

In addition to that I changed the default channel in MyConfig.h (because I have two different MySensor networks):

#ifndef MY_RF24_CHANNEL
#define MY_RF24_CHANNEL (82)
#endif

I am pretty sure, that this is not important, but I also set up rsyslog to write gw logs to an own file instead of syslog....

Maybe this helps.

Cheers,
Otto

marceloaqno

@otto001 I think I found the problem. I wrote a fix for it:
https://github.com/marceloaqno/MySensors/commit/ca8865d8e1fe1b08195943fae1640d5fddb81924

otto001

@marceloaqno : Wow, you are fast! Thanks!
Just applied the patch and recompiled. I will report in a couple of days while my monitoring-script is still running (and logging)...

Cheers,
Otto

mvader

@marceloaqno I am also recompiling and testing.
thanks for the (hopeful) fix
I will report back with my status in a few days

otto001

@marceloaqno :

At first, THANKS for your help. CPU usage does not rise anymore.
But unfortunately no more node READs after the same situation:

Mar  4 07:00:09 raspiez mysgw: TSF:MSG:READ,14-14-0,s=255,c=3,t=6,pt=1,l=1,sg=0:0
Mar  4 07:00:09 raspiez mysgw: TSF:MSG:READ,12-12-0,s=255,c=3,t=6,pt=1,l=1,sg=0:0
Mar  4 07:00:09 raspiez mysgw: GWT:RFC:C=1,MSG=14;255;3;0;6;M
Mar  4 07:00:09 raspiez mysgw: !TSF:MSG:SEND,0-0-14-14,s=255,c=3,t=6,pt=0,l=1,sg=0,ft=0,st=NACK:M
Mar  4 07:00:09 raspiez mysgw: TSF:MSG:READ,11-11-0,s=255,c=0,t=18,pt=0,l=3,sg=0:1.5
Mar  4 07:00:09 raspiez mysgw: GWT:RFC:C=1,MSG=12;255;3;0;6;M
Mar  4 07:00:09 raspiez mysgw: TSF:MSG:SEND,0-0-12-12,s=255,c=3,t=6,pt=0,l=1,sg=0,ft=0,st=OK:M
Mar  4 07:00:13 raspiez mysgw: TSM:READY:NWD REQ
Mar  4 07:00:13 raspiez mysgw: TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Mar  4 07:00:13 raspiez mysgw: TSF:SRT:OK
Mar  4 07:00:13 raspiez mysgw: TSF:SAN:OK
Mar  4 07:00:37 raspiez mysgw: Ethernet client disconnected.
Mar  4 07:00:37 raspiez mysgw: New connection from 192.168.1.11
Mar  4 07:00:37 raspiez mysgw: GWT:TSA:C=0,CONNECTED
Mar  4 07:00:37 raspiez mysgw: GWT:TSA:C=1,DISCONNECTED

After this point (again after a reconnect from the controller) no more messages from any node PLUS the cpu usage stayed normal (what makes it even worse at this moment, as I would have to create a script which monitors the logfile and restarts the mysgw service in this situation). As soon as the service is restarted, everything is working fine again.

What happens after this point in the log:

Mar  4 07:00:37 raspiez mysgw: GWT:RFC:C=0,MSG=0;0;3;0;2;
Mar  4 07:15:13 raspiez mysgw: TSF:SAN:OK
Mar  4 07:20:13 raspiez mysgw: TSM:READY:NWD REQ
Mar  4 07:20:13 raspiez mysgw: TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Mar  4 07:30:13 raspiez mysgw: TSF:SRT:OK
Mar  4 07:30:13 raspiez mysgw: TSF:SAN:OK
Mar  4 07:30:37 raspiez mysgw: GWT:TSA:C=0,DISCONNECTED
Mar  4 07:30:37 raspiez mysgw: Ethernet client disconnected.
Mar  4 07:30:37 raspiez mysgw: New connection from 192.168.1.11
Mar  4 07:30:37 raspiez mysgw: GWT:TSA:C=0,CONNECTED
Mar  4 07:30:37 raspiez mysgw: GWT:RFC:C=0,MSG=0;0;3;0;2;
Mar  4 07:40:13 raspiez mysgw: TSM:READY:NWD REQ
Mar  4 07:40:13 raspiez mysgw: TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Mar  4 07:45:13 raspiez mysgw: TSF:SAN:OK
Mar  4 08:00:13 raspiez mysgw: TSM:READY:NWD REQ
Mar  4 08:00:13 raspiez mysgw: TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Mar  4 08:00:13 raspiez mysgw: TSF:SRT:OK
Mar  4 08:00:13 raspiez mysgw: TSF:SAN:OK
Mar  4 08:00:37 raspiez mysgw: GWT:TSA:C=0,DISCONNECTED
Mar  4 08:00:37 raspiez mysgw: Ethernet client disconnected.
Mar  4 08:00:37 raspiez mysgw: New connection from 192.168.1.11
Mar  4 08:00:37 raspiez mysgw: GWT:TSA:C=0,CONNECTED

and so on....

It seems that the increasing cpu usage is only a symptom for the real problem.
As I am using this in production I will dis-apply the patch and use the daemon as before (beacuse using the python script for cpu-usage-monitoring makes the situation somehow controllable for me).
Maybe you could have an additional eye on this please? Could I help you with further information? Maybe I could add some additional logging somewhere in the code?

Thanks again and cheers,
Otto

marceloaqno

@otto001 I'm trying to reproduce this problem, but so far I've got nothing beyond the normal behavior of mysgw.
I tried with a RPi1 and RPi3 using the fhem controller with the latest raspbian.

mvader

@marceloaqno said in Building a Raspberry Pi Gateway:

@otto001 I'm trying to reproduce this problem, but so far I've got nothing beyond the normal behavior of mysgw.
I tried with a RPi1 and RPi3 using the fhem controller with the latest raspbian.

My uptime is at 4 days now. so far still working with your patch. CPU is at 2% and sensor network still working correctly. will report if that changes.

Nick Willis

I've been using your restarting method and its been working ok and has done one restart in 5 days.
However I now see something else where altho cpu use is fine, none of my nodes can communicate to the gateway. My controller can connect to the gateway no problem but there are no messages received. Restarting nodes does not help. Only a manual restart of the gateway gets things going again - at least for a while anyway.
Where is the log output from the gateway to be found? I know it can be run from the bash prompt to write the output to screen but how about logging the installed service output?

mfalkvidd

@nick-willis the log is sent to syslog by default. See https://www.mysensors.org/build/raspberry#troubleshooting for details.

otto001

@marceloaqno : This is strange. I am using a RPi 1 on this gateway with raspbian Jessie. Beside the GW-service there is only a small python programm which listens on an UPD-Port and plays an mp3 if it receives suitable data (for doorbell). Since March 4, 20:15 there were several restarts due to high cpu usage (after dis-applying your patch):

2018-03-04 23:01:03 cpu (99.7) very high! --> restarting service
2018-03-05 07:31:03 cpu (99.7) very high! --> restarting service
2018-03-05 11:01:03 cpu (98.7) very high! --> restarting service
2018-03-05 13:31:04 cpu (94.7) very high! --> restarting service
2018-03-06 11:31:03 cpu (99.7) very high! --> restarting service
2018-03-06 12:31:03 cpu (98.7) very high! --> restarting service
2018-03-07 06:01:03 cpu (94.8) very high! --> restarting service
2018-03-07 22:31:03 cpu (99.8) very high! --> restarting service

With this "auto-restarts" the sensors are working fine, I did not notice any problems.

However, I just upgraded to the recent Stretch (what takes a LONG time on RPi 1 and made me struggle a little bit with the new network names). I also rebuilt the GW of course and will report here what will happen....

@Nick-Willis : did you apply the patch of marceloaqno or not? Did you enable debugging? (see here )

Cheers,
Otto

marceloaqno

@otto001 In a new attempt to trigger this problem, I am using a script that will force the fhem controller to reconnect every 30s to the gateway (built with mysensors master branch, unpatched), and sends me a note in case of high CPU usage:

I'll leave it running for a few days to see what happens.

Here is the script, if anyone is interested:

#!/bin/sh

while true; do
  (echo "set mysgw disconnect"; echo "quit") | telnet localhost 7072 > /dev/null 2>&1
  sleep 1
  (echo "set mysgw connect"; echo "quit") | telnet localhost 7072 > /dev/null 2>&1
  sleep 1

  cpu_percent=$(ps -C "mysgw" -o %cpu=)
  cpu_percent=${cpu_percent%%.*}
  if [ "$cpu_percent" -ge 90 ]
  then
    echo "ALERT: HIGH CPU USAGE"
    # pushbullet script from https://gist.github.com/outadoc/189bd3ccbf5d6e0f39e4
    /home/pi/pushbullet.sh "RPi1: HIGH CPU USAGE"
    exit
  fi

  sleep 30
done

otto001

@marceloaqno : Thank you!
Just a notice: I do NOT disconnect, I just connect every 30 minutes. Has been working for me since more than 2 years (ancient ms-gw version on RPi).

Yesterday I did a dist-upgrade to stretch on the "problematic" gw (as mentioned above) - without your patch but with my monitoring-script running. It is a little bit to early I presume, but no high cpu problems so far. I will report back here in a few days. Maybe this problem is really related to old libraries or something?! If this is running stable for a few days I will apply your patch and see what happens.
Thanks again!

Cheers,
Pula

marceloaqno

@otto001 After a few hours running the reconnect script, I noticed several error messages in my gateway log:

accept(): Too many open files

The problem was that the gateway wasn't releasing the socket descriptor after each disconnection of the controller. After reaching the limit (mine was 1036) the controller can't connect.

I'm not entirely sure that this problem is related to what you're having. You can check how many file descriptors currently opened by the gateway with the command below

sudo lsof -u root |grep mysgw |wc -l

The fix:
https://github.com/marceloaqno/MySensors/commit/a40d4441b7460225100398ff6f2581c2b0df36ea

How do I update from master branch to development branch?

gohan

Why do you want to do that? Development version is still work in progress

I'm using a raspberry pi gateway in home-assistant.
Sometimes my gateway doesn't work anymore, and when I restart the service, it is running good
I used ./configure --my-transport=nrf24 --my-rf24-irq-pin=15 --my-gateway=ethernet --my-port=5003

gohan

Read the article again, there is the instruction on how to download the development version, but before you have to uninstall the mysgw service and rename the current mysensors folder to keep as backup.

marceloaqno

@gieljnssns The fixes I mentioned above haven't yet been applied to the development branch. Eventually they will be, but I'm still doing some testing.

otto001

@marceloaqno :
Thank you!!! Should this patch be applied together with your first patch?
Interesting:
After some days without problems on the newest raspbian packages the gw hung again, but WITHOUT high cpu usage (and WITHOUT your first patch applied). There were just no more reads from the sensors in the log, only the re-connects of fhem. After restarting the service, everything is fine again.
Unfortunately I did see your post after that
But I checked the log and could not find the words "too" or "Too" in the log (running service with -d).
Some minutes after restarting I checked how many open files there were with the command you supplied: 21

In the meantime I have created a small udp listening service which restarts the service if a certain string is received, so I can restart the service by double-clicking one of my switches without the need of login to the gw (also for the wife)....

@gieljnssns : I think you are experiencing exactly the same problem we are discussing

Thanks again!
Cheers,
Otto

gohan

I'm using dietpi and I am not having big issues even running 2 gateway instances with 2 different radios

marceloaqno

@otto001 The second patch can be applied without the first one.

💬 Building a Raspberry Pi Gateway

Suggested Topics

Missing Notification Settings
Bug Reports • • edoardoo

💬 Building a Orange Pi Gateway
Announcements • • hek

💬 Building a WiFi Gateway using ESP8266
Announcements • • hek

💬 Battery Powered Sensors
Announcements • battery • • hek

💬 Building a wired RS485 sensor network
Announcements • • hek

💬 Ago Control
Announcements • • hek

0
Online

11.4k
Users

11.1k
Topics

112.7k
Posts

💬 Building a Raspberry Pi Gateway

Suggested Topics

Missing Notification Settings Bug Reports • • edoardoo

💬 Building a Orange Pi Gateway Announcements • • hek

💬 Building a WiFi Gateway using ESP8266 Announcements • • hek

💬 Battery Powered Sensors Announcements • battery • • hek

💬 Building a wired RS485 sensor network Announcements • • hek

💬 Ago Control Announcements • • hek

0Online

11.4kUsers

11.1kTopics

112.7kPosts

Missing Notification Settings
Bug Reports • • edoardoo

💬 Building a Orange Pi Gateway
Announcements • • hek

💬 Building a WiFi Gateway using ESP8266
Announcements • • hek

💬 Battery Powered Sensors
Announcements • battery • • hek

💬 Building a wired RS485 sensor network
Announcements • • hek

💬 Ago Control
Announcements • • hek

0
Online

11.4k
Users

11.1k
Topics

112.7k
Posts