Code for beta-testing?

jka

@andriej
Hi, I had problem using the main branch mysensors/Raspberry, that because of the bit-field struct (header_s) in Sensor.h They will get packed in different order on the arduino compared to the RPi (at least with my compiler, gcc). In order to fix this I created the 1.4dev branch. It uses full bytes instead of the bit-fields, but you need (of cause) to compile both the sensors (arduino) and the gateway (RPi) using the 1.4dev branch.

We need to fix this in the main branch later on (and use the same files as in mysensors/Arduino/libraries/MySensors)

andriej

@jka I can't (or I don't know how) to compile the 1.4-dev code. I've tried everything I guess.

$ make RadioGateway
make -C librf24
make[1]: Wejście do katalogu `/home/pi/mysensors/Raspberry-1.4dev/librf24'
g++ -shared -Wl,-soname,librf24.so.1 -Ofast -mfpu=vfp -mfloat-abi=hard -march=armv6zk -mtune=arm1176jzf-s  -o librf24.so.1 compatibility.o gpio.o spi.o RF24.o
make[1]: Opuszczenie katalogu `/home/pi/mysensors/Raspberry-1.4dev/librf24'
make -C /home/pi/mysensors/libraries/MySensors
make[1]: Wejście do katalogu `/home/pi/mysensors/libraries/MySensors'
make[1]: Nie ma nic do zrobienia w `all'.
make[1]: Opuszczenie katalogu `/home/pi/mysensors/libraries/MySensors'
g++ -o RadioGateway RadioGateway.o -Wall -Ofast -mfpu=vfp -DDEBUG -DRPI -mfloat-abi=hard -march=armv6zk -mtune=arm1176jzf-s -I./librf24 -I/home/pi/mysensors/libraries/MySensors /home/pi/mysensors/libraries/MySensors/Gateway.o 			/home/pi/mysensors/libraries/MySensors/Relay.o /home/pi/mysensors/libraries/MySensors/Sensor.o ./librf24/librf24.so.1
RadioGateway.o: In function `main':
/home/pi/mysensors/Raspberry-1.4dev/RadioGateway.cpp:103: undefined reference to `Gateway::begin(unsigned char)'
collect2: error: ld returned 1 exit status
make: *** [RadioGateway] Błąd 1
pi@domoticzpi ~/mysensors/Raspberry-1.4dev $

jka

@andriej
OK, there where a few glitches in the Makefile, you can update it now.

andriej

@jka
I seem to have new problem then, guess it's with g++?

pi@domoticzpi ~/mysensors/Raspberry $ make
make -C librf24
make[1]: Wejście do katalogu `/home/pi/mysensors/Raspberry/librf24'
g++ -Wall -fPIC -Ofast -mfpu=vfp -mfloat-abi=hard -march=armv6zk -mtune=arm1176jzf-s -c RF24.cpp
In file included from /usr/include/unistd.h:203:0,
             from spi.h:13,
             from RF24_config.h:23,
             from RF24.cpp:10:
/usr/include/arm-linux-gnueabihf/bits/posix_opt.h:131:3: error: #error "Never use <bits/syscall.h> directly; include <sys/syscall.h> instead."
/usr/include/arm-linux-gnueabihf/bits/posix_opt.h:20:0: error: unterminated #ifndef
In file included from RF24.cpp:11:0:
RF24.h: In constructor ‘RF24::RF24(std::string, uint32_t, uint8_t)’:
RF24.h:52:12: warning: ‘RF24::spispeed’ will be initialized after [-Wreorder]
RF24.h:50:11: warning:   ‘uint8_t RF24::ce_pin’ [-Wreorder]
RF24.cpp:241:1: warning:   when initialized here [-Wreorder]
make[1]: *** [RF24.o] Błąd 1
make[1]: Opuszczenie katalogu `/home/pi/mysensors/Raspberry/librf24'
make: *** [RF24] Błąd 2
pi@domoticzpi ~/mysensors/Raspberry $

Bmlsx

dont know if this should be on new subject but spare me this one

i recompiled both arduino and rpi part with 1.4 branch

but i get on the RPi side CRC errors

Starting Gateway...
Sensor-1Gateway created...
SPI device = /dev/spidev0.0
SPI speed = 8000000
CE GPIO = 25
STATUS = 0x0e RX_DR=0 TX_DS=0 MAX_RT=0 RX_P_NO=7 TX_FULL=0
RX_ADDR_P0-1 \0 = 0xabcdabc000 0xabcdabc000
RX_ADDR_P2-5 \0 = 0xff 0xc4 0xc5 0xc6
TX_ADDR = 0xabcdabc000
RX_PW_P0-6 \0 = 0x20 0x20 0x20 0x00 0x00 0x00
EN_AA = 0x3f
EN_RXADDR \0 = 0x07
RF_CH = 0x4c
RF_SETUP \0 = 0x07
CONFIG = 0x0f
DYNPD/FEATURE \0 = 0x3f 0x04
Data Rate = 1MBPS
Model = nRF24L01+
CRC Length = 16 bits
PA Power = PA_MAX
Radio setup complete-0;0;4;11;Arduino startup complete.
Begin called
Dynamic payload size=12
Received: from=9, to=0, childId=255, mtype=0, type=17, crc=47, '1.4'
Message crc error.
header.type=17, header.to=0, radioId=0
Dynamic payload size=10
Received: from=9, to=0, childId=255, mtype=4, type=7, crc=115, '0'
Message crc error.
header.type=7, header.to=0, radioId=0
Dynamic payload size=12
Received: from=9, to=0, childId=0, mtype=0, type=7, crc=15, '1.4'
Message crc error.
header.type=7, header.to=0, radioId=0
Dynamic payload size=12
Received: from=9, to=0, childId=1, mtype=0, type=6, crc=187, '1.4'
Message crc error.
header.type=6, header.to=0, radioId=0
Dynamic payload size=13
Received: from=9, to=0, childId=1, mtype=1, type=0, crc=228, '20.5'
Message crc error.
header.type=0, header.to=0, radioId=0
Dynamic payload size=13
Received: from=9, to=0, childId=0, mtype=1, type=1, crc=118, '50.3'
Message crc error.

while on the arduino just the failed send

Started sensor.
Relay=0, distance=1
Relaying message back to gateway.
Tx: fr=9,to=0,la=9,ne=0,ci=255,mt=0,ty=17,cr=47: 1.4
Send failed.
Relaying message back to gateway.
Tx: fr=9,to=0,la=9,ne=0,ci=255,mt=4,ty=7,cr=115: 0
Send failed.
Relaying message back to gateway.
Tx: fr=9,to=0,la=9,ne=0,ci=0,mt=0,ty=7,cr=15: 1.4
Send failed.
Relaying message back to gateway.
Tx: fr=9,to=0,la=9,ne=0,ci=1,mt=0,ty=6,cr=187: 1.4
Send failed.
Relaying message back to gateway.
Tx: fr=9,to=0,la=9,ne=0,ci=1,mt=1,ty=0,cr=228: 20.5
Send failed.
T: 20.50
Relaying message back to gateway.
Tx: fr=9,to=0,la=9,ne=0,ci=0,mt=1,ty=1,cr=118: 50.3
Send failed.
H: 50.30
Relaying message back to gateway.
Tx: fr=9,to=0,la=9,ne=0,ci=1,mt=1,ty=0,cr=195: 20.6
Send failed.
T: 20.60
Relaying message back to gateway.
Tx: fr=9,to=0,la=9,ne=0,ci=0,mt=1,ty=1,cr=81: 50.0
Send failed.
H: 50.00

also changed the Data Rate = 1MBPS on arduino to match the RPI preet

any ideas?

thanx a mil

hek

You are experiencing the problem @jka describes above. When the message header bitfield is packed (by the compiler) if differs on the Arduino and RPI which result in garbled header and CRC-check-errors.

We have to introduce our own bit manupilation when adding/extracting stuff from the (bit-field)-header to overcome this cross platform problems.

Bmlsx

@hek ooooooh, I thought it was already done in the 1.4 branch... Silly me.

Thanx

hek

You could do like @jka did and modify header to only use full bytes for all fields temporarily (on the sensor side as well). This way you will be able to communicate until we fix the problem properly.

Bmlsx

please excuse the frivolity

u mean like this both on arduino and Pi?

// The message structure
typedef struct {
unsigned char crc; // uint8_t crc; // 8 bits crc
unsigned char version; // (3 bits) protocol version
unsigned char binary; // (1 bit). Data is binary and should be encoded when sent to sensor net gateway
unsigned char from; // 8 bits. RadioId of sender node
unsigned char to; // 8 bits. RadioId of destination node
unsigned char last; // 8 bits. RadioId of last node this message passed
unsigned char childId; // 8 bits. Up to MAX_CHILD_DEVICES child sensors per radioId
unsigned char messageType; // (4 bits). Type of message. See messageType
unsigned char type; // 8 bits. variableType or deviceType depending on messageType
} header_s;

hek

@Bmlsx
Yes. :)

wannabee

@andriej and @jendrush, I commited some code to my repository for the python gateway with half (oneway) Domoticz integration. It also support Openhab, The Openhab integarion is what I think I will integarate for my self and continue to develop. Domoticz needs provide a proper API that I can use without spending a lot of time trying to figure out how it works. My code is at https://github.com/wbcode/ham. You can try it out if you want. The installation part and configuration is missing a lot of instructions.

Will create my own thread for my gateway code when I have a name for it and I have spent the time to do a proper update to the wiki and better a presenation on what you cand do.

Please note that iit's not a full replacment for a Vera and it's my first time codeing python...

Bmlsx

@hek

hi there,

nagging time ...did all that (unsigned char on both ends arduino and rpi) but all i get in my rpi is below

any ideas?!!

thanx
G

Starting Gateway...
Sensor-1Gateway created...
SPI device = /dev/spidev0.0
SPI speed = 8000000
CE GPIO = 25
STATUS = 0x0e RX_DR=0 TX_DS=0 MAX_RT=0 RX_P_NO=7 TX_FULL=0
RX_ADDR_P0-1 \0 = 0xe7e7e7e7e7 0xc2c2c2c2c2
RX_ADDR_P2-5 \0 = 0xff 0xc4 0xc5 0xc6
TX_ADDR = 0xe7e7e7e7e7
RX_PW_P0-6 \0 = 0x00 0x00 0x20 0x00 0x00 0x00
EN_AA = 0x3f
EN_RXADDR \0 = 0x07
RF_CH = 0x4c
RF_SETUP \0 = 0x07
CONFIG = 0x0c
DYNPD/FEATURE \0 = 0x3f 0x04
Data Rate = 1MBPS
Model = nRF24L01+
CRC Length = 16 bits
PA Power = PA_MAX
Radio setup complete-0;0;4;11;Arduino startup complete.
Begin called
Dynamic payload size=11
Received: from=16, to=18, childId=0, mtype=50, type=49, crc=120, '.7'
Message crc error.
header.type=49, header.to=18, radioId=0
Dynamic payload size=11
Received: from=16, to=2, childId=1, mtype=54, type=53, crc=231, '.2'
Message crc error.
header.type=53, header.to=2, radioId=0

Bmlsx

Hi there again

A quick question @jka I used the 1.4 branch on both arduino and rpi and all I get o rpi are messed up messages with crc errors, can u point me a direction on what I do wrong?

Thanx
G

Bmlsx

Can anyone pls help on this ...I'm about to eat the raspberry

hek

Sorry, I don't know why you are experiencing problem. We will soon have a look at the RPI stuff again and make a usable version.
The last week all our mysensors-time has been spend on server infrastructure (prod/stage environment etc) to prepare for the new services.
Getting this right took more time that we initially anticipated.

In the meantime. Anyone having a good low footprint node-database-candidate for storing sensor data in the RPI?
The purpose is having a local backup of all historic data and to keep a record of which data has been sent to cloud-service.
There won't be much local querying, mostly storing.

Right now I'm considering:
NeDB/NEDBLogger (https://github.com/louischatriot/nedb)
Levelup/Leveldb (https://github.com/rvagg/node-levelup)

ToSa

The "Message crc error." is a bit misleading. The code in the validate() routine actually checks the crc as well as the protocol version. Looking at the Arduino and the RPi 1.4 branch the versions don't match. I've adjusted the library and protocol version in the Sensor.h file on the Rpi to:

LIBRARY_VERSION "1.4"
PROTOCOL_VERSION 2

and the crc error disappeared.

While changing the data rate from 2M to 1M in line 30 of the Arduino Sensor.cpp, I had to fix a syntax error to recompile successfully in line 361: a missing "}" to close the do/while. The full line should be "} while (--retry);"

With these two fixed the RPi and the Arduino now communicate - but I get a lot of "Send failed" messages on the Arduino and a lot of "Error looking up radioIds" on the RPi now:

RPi output:
Received: from=255, to=0, childId=255, mtype=4, type=5, crc=139, ''
139, 2, 0, 255, 0, 255, 255, 4, 5
Message crc ok.
header.type=5, header.to=0, radioId=0
Got message
Message type: 4
msgTypeId=4 int=4
Error looking up radioIds
...and so on...

Android in DEBUG mode:
Started sensor.
Relay=0, distance=1
No radio id found in EEPROM fetching one from sensor net gateway
Relaying message back to gateway.
Tx: fr=255,to=0,la=255,ne=0,ci=255,mt=4,ty=5,cr=139:
Send failed.
Relaying message back to gateway.
Tx: fr=255,to=0,la=255,ne=0,ci=255,mt=4,ty=5,cr=139:
Send failed.
...and so on...

ToSa

Changing the mongo database URL in the dbDetails,js file
from:
mongodb://mysensors:27017/pi
to:
mongodb://localhost:27017/mysensors

fixed the "Error looking up radioIdo" issue on the RPi. Now the Rpi generates a new radioId but the attempt to send this to the Arduino fails:

RPi output:
Received: from=255, to=0, childId=255, mtype=4, type=5, crc=139, ''
139, 2, 0, 255, 0, 255, 255, 4, 5
Message crc ok.
header.type=5, header.to=0, radioId=0
Got message
Message type: 4
msgTypeId=4 int=4
Sending data 255;255;4;5;1
Writing back messageWe have input
Nread=14
INP:255;255;4;5;1
.
INGOT radioId=255, childId=255, messageType=4, type=5, value=1
Sending data to GATEWAYRelaying message back to gateway.
Sending: from=0, to=255, childId=255, mtype=4, type=5, crc=193, '1
', sent via 0
193, 2, 0, 0, 255, 0, 255, 4, 5
radioId: Inserted id=1
Send failed. No ack received.
Sent to JS
Dynamic payload size=9

Bmlsx

@ToSa thank you ...u saved my rpi;)

ToSa

@BMLSX - probably rather saved your stomach - eating the raspberry sounds noxious :)

I got one step further (maybe) - the Arduino now receives the radioId... looking at the log from the RPi above I stumbled over the RPi sending the data to the gateway ??? the RPi is the gateway !!! Comparing the code to the SerialGateway which seems to work for a lot of folks I found that the sendData() function is missing in the Relay.cpp code used for the RPi. Once that procedure is in place the RPi actually submits the new radioId to the sensor:

RPi : now "sent via 255" instead of "sent via 0"
Dynamic payload size=9
Received: from=255, to=0, childId=255, mtype=4, type=5, crc=65, ''
Message crc ok.
header.type=5, header.to=0, radioId=0
Got message
Message type: 4
Writing back message
msgTypeId=4 int=4
Sending data 255;255;4;5;1
We have input
Nread=14
INP:255;255;4;5;1
.
INGOT radioId=255, childId=255, messageType=4, type=5, value=1
Sending data to GATEWAY
No route... try sending direct.
Sending: from=0, to=255, childId=255, mtype=4, type=5, crc=11, '1
', sent via 255
radioId: Inserted id=1
Send failed. No ack received.
Sent to JS

Arduino
Started sensor.
Open ping reading pipe: 255
Tx: fr=255,to=255,la=255,ne=255,ci=255,mt=4,ty=9,cr=235:
Message available on pipe 1
Rx: fr=0,to=255,la=0,ci=255,mt=4,t=10,cr=106(ok): 0
Message addressed for this node.
Using relay 0. Distance is 0
Relay=0, distance=1
No radio id found in EEPROM fetching one from sensor net gateway
TH: get: n=0,c=255,st=4,rt=4,vt=5
Relaying message back to gateway.
Tx: fr=255,to=0,la=255,ne=0,ci=255,mt=4,ty=5,cr=65:
Ack: receive timeout
Message available on pipe 1
Sent ack msg to 0
Rx: fr=0,to=255,la=0,ci=255,mt=4,t=5,cr=11(ok): 1

Message addressed for this node.
Radio id received: 1
Radio id stored in EEPROM was: 1
Relaying message back to gateway.
Tx: fr=1,to=0,la=1,ne=0,ci=255,mt=0,ty=17,cr=17: 1.3b3 (7afb55c)
Ack: receive timeout
Relaying message back to gateway.
Tx: fr=1,to=0,la=1,ne=0,ci=255,mt=4,ty=7,cr=176: 0
Ack: receive timeout
Relaying message back to gateway.
Tx: fr=1,to=0,la=1,ne=0,ci=255,mt=4,ty=14,cr=21: Temperature Sensor
Ack: receive timeout
Relaying message back to gateway.
Tx: fr=1,to=0,la=1,ne=0,ci=255,mt=4,ty=15,cr=13: 1.0
Ack: receive timeout
Relaying message back to gateway.
Tx: fr=1,to=0,la=1,ne=0,ci=0,mt=0,ty=6,cr=99: 1.3b3 (7afb55c)
Ack: receive timeout
TH: get: n=0,c=255,st=4,rt=4,vt=13
Relaying message back to gateway.
Tx: fr=1,to=0,la=1,ne=0,ci=255,mt=4,ty=13,cr=200:
Ack: receive timeout

I'm still not sure if the missing ACK for the majority of the messages is by purpose or by accident... would be relatively easy to fix by either not waiting for an ACK for all messages or by always sending an ACK - but I don't want to mess with this code without knowing why it was implemented this way.

ToSa

RPi and Arduino now talk to each other, the Arduino receives the radioId from the Rpi and stores it in the EEPROM and the Arduino presents itself to the network. I still see some timing issues (ACKs too late / replies too early while the other end is still waiting for an ACK / ...) but in general it works.
To get there I forked both the Arduino and the RPi project and tried to get to the latest and greatest code by merging some of the branches (mainly master and 1.4dev)... When I ran into additional issues due to the RPi code not getting the latest updates and bug fixes available in the Arduino project, I decided to merge across the two and ended up with a common set of RF files: a common codebase for RF24 as well as Sensor/Relay/Gateway. If any of this is of interest for themain development then please let me know and I'll create a pull request.

What I did in detail:

Arduino:

started new branch tosa-dev based on latest master branch
adjusted the message header to use full bytes as mentioned earlier in this thread
no need to merge the 1.4dev branch as the message header was essentially the only remaining relevant change in there
no need to merge other branches as the majority of these were already merged back into master

Raspberry:

started new branch tosa-dev based on latest master branch
merged the 1.4dev branch into tosa-dev
a couple of minor tweaks (mongoDb URL / start and stop scripts / data rate / ... majority listed earlier in this thread)

When I found the missing Relay::sendData() routine I decided to stop tweaking bit by bit and instead looked at what it would take to use the more advanced Arduino code for RF24 and Sensor/Relay/Gateway for the RPi as well. Based on comments in the forum this seems to work for several people using a SerialGateway or EthernetGateway so my expectation was to find well maintained code there avoiding some of the low level communication issues I ran into so far with the RPi code. This code is available in a separate branch "tosa-dev-commonrf" for both Arduino and Raspberry.

The code is not perfect yet but it works. I'm not a GIT expert but as far as I can tell the fork is public and everybody should be able to see it:
https://github.com/ToSa27/Raspberry/tree/tosa-dev-commonrf
https://github.com/ToSa27/Arduino/tree/tosa-dev-commonrf

Again: happy to create pull requests if you want to use this setup as part of the master branch at some point. The main reason for me to do this is to get a solid foundation to start working on the OTA update code...

Code for beta-testing?

17

11.7k

11.2k

113.1k