[SOLVED] Weird serial GW problem

[kisse66]

Been looking into this a few days now, but maybe it's worth asking if someone else has seen similar:

Have RFM69HW based serial GW on 8MHz pro mini. This is then hooked up to RPi2 via direct serial (RPi runs a nodejs based simple controller). I'm using latest (few days ago) dev branch from git. Radio does have separate regulator and proper bypass caps. Serial GW sketch is empty, i.e. no extra functionality. About default settings, new RFM driver, no signing, no encryption.

Basically it works, but there are two issues. Main one is the controller over the serial line (added a debug print of raw data in serial event handler) sometimes gets concatenated messages, i.e. the separating newline is randomly missing. This makes the message parser fail with incorrect payload for instance. This is very random and seems it gets more likely as I increase the rate of messaging (attempting OTA for instance). Originally I used the default 38600 bps, but tried to increase it to 57600. No real change and the message rate I see should not exceed the serial speed (a few msg per second). Not only connected to the message length as I have seen it also with short messages (2 byte payload), but mostly happens when attempting OTA.
Slow sensors seen to work fine, but OTA never goes through. When testing there's only GW and test node powered.

Another issue is lost messages towards the GW. Haven't looked deeper yet, but I do get radio acks every time, but sometimes one message just never appears out of the gw (nodes are a few meters apart with good RSSI). I mean nothing comes out.

I'll keep looking and simplifying the setup, but if someone has seen something like this I'd appreciate any info.

[kisse66]

ok/FYI,
seems there is nothing wrong with GW HW nor arduino code - I removed my GW module from RPi and used a USB serial to attach it to a windows PC running MYSController in GW mode instead. My nodejs controller connects to it rather than the local serial port and suddenly everything works, no more missing newlines. I even feed power only through FTDI cable and it still feels stable, even OTA completed fine.

Before this I did try increasing Arduino serial TX_BUFFER to 128 bytes, but it had no visible effect.

So it must be something in RPi serial port or nodejs serial interface. At this point I don't think its in my controller code (loosely based on the MySensorsSampleController), since I print the raw data from serial port first at the event handler and that already shows the missing data:

}).on('data', function(rd) {
                                logger.debug('gw' + i + " data: appending '" + rd + "'");
                                appendRfData(rd.toString(), db, i);

e.g. I saw

[2017-06-14 12:23:06.975] [DEBUG] [default] - gw 2: appending '243;255;4;0;2;03'
[2017-06-14 12:23:07.980] [DEBUG] [default] - gw 2: appending '243;255;'
[2017-06-14 12:23:07.982] [DEBUG] [default] - gw 2: appending '4;0;2;03'
[2017-06-14 12:23:07.984] [DEBUG] [default] - gw 2: appending '000200E7'
[2017-06-14 12:23:07.986] [DEBUG] [default] - gw 2: appending '05
'
[2017-06-14 12:23:07.987] [DEBUG] [default] - rfm69 RX: 243;255;4;0;2;03243;255;4;0;2;03000200E705

so one newline missing assembling two messages together.

So problem kind of solved, at least there is no issue in MySensors side. It's been running now for 30 minutes, 3 ping-pong test msg per second and none lost.

[kisse66]

One more update:

have not fully verified, but it seems like this was RF interference issue (maybe a transmitted ACK??).

I added a 100n ceramic cap to power in line (at the GPIO connector) and 10nF caps from UART TX and RX to ground (small SMD caps at the connector). I also moved the antenna a bit higher by soldering coaxial cable to RFM and then peeling off the outer shield of the coax for the ~80mm. Problem gone!

I'll add a ground plane to the antenna by putting aluminium tape to the top cover (missing here) and connecting the coax shield to it. Makes a better antenna and better shielding towards RPi. Have not re-checked range yet, but checked the UART signals with scope and the 10nF looks good as noise filter.