[solved] RFM69 based nodes unable to report Lib Version

jpaulin

I get exactly the same bug. I found a way to solve it, but not sure if it's the root cause.
I use RFM69W and the latest release from the Development Branch.

From your message dump:

TSF:MSG:SEND,101-101-0-0,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
!TSF:MSG:SEND,101-101-0-0,s=255,c=0,t=17,pt=0,l=10,sg=0,ft=0,st=NACK:2.0.1-beta
TSF:MSG:SEND,101-101-0-0,s=255,c=3,t=6,pt=1,l=1,sg=0,ft=1,st=OK:0

The Internal Presentation Message with Lib Version as payload always fails. (!TSF: c=0, t=17, st=NACK:2.0.1-beta)

The message is sent immediately after the previously sent internal message (TSF: c=3, t=15, st=OK:0100) (t=15 => I_REQUEST_SIGNING).
The Gateway responds the Signing Preference Message to the node exactly at the same time the node tries to send the Lib Version Presentation Message to the Gateway. Seems that won't work. There's no buffering?
I added a 1s delay for test purpose in MySensorsCore.cpp to give time to finish the response from the gateway before sending the Lib Version Presentation Message.

New message dump from my test-node after changes:

TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
TSF:MSG:READ,0-0-3,s=255,c=3,t=15,pt=6,l=2,sg=0:0100
TSF:MSG:SEND,3-3-0-0,s=255,c=0,t=17,pt=0,l=10,sg=0,ft=0,st=OK:2.0.1-beta
TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=6,pt=1,l=1,sg=0,ft=0,st=OK:0

Now a new message appears in the node, the response message from the Gateway to the Signing Preference Message, and the presentation of the Lib Version works as expected.

TSF:MSG:READ,0-0-3,s=255,c=3,t=15,pt=6,l=2,sg=0:0100 
TSF:MSG:SEND,3-3-0-0,s=255,c=0,t=17,pt=0,l=10,sg=0,ft=0,st=OK:2.0.1-beta

In MySensorsCore.cpp at line 216 I added a wait(1000); to mitigate the issue.

	// Send signing preferences for this node to the GW
	signerPresentation(_msgTmp, GATEWAY_ADDRESS);

wait(1000);

		// Send presentation for this radio node
	#if defined(MY_REPEATER_FEATURE)
		(void)present(NODE_SENSOR_ID, S_ARDUINO_REPEATER_NODE);
	#else
		(void)present(NODE_SENSOR_ID, S_ARDUINO_NODE);
	#endif

I guess the final patch would look different and would need to be looked into by @Anticimex or @hek. :smiley:

Anticimex

@jpaulin the signing backend is already waiting for the GW to send a message. So if it is not waiting long enough I believe the existing delay should be increased instead: https://github.com/mysensors/MySensors/blob/development/core/MySigning.cpp#L158

tekka

@Anticimex This is a conditional wait(), i.e. only when signing is enabled: https://github.com/mysensors/MySensors/blob/development/core/MySigning.cpp#L154-L160

Since the GW always replies to signing preferences, but the node only waits if signing is enabled - this message will eventually collide with the following lib version message, as seen above.

I suggest removing the surrounding #ifdef.

Anticimex

@tekka true. Moving the existing delay outside the preprocessor condition should help.

Anticimex

@jpaulin could you please file a pull request with the delay moved outside the preprocessor condition (as you have the rig to verify the change works)?

jpaulin

@Anticimex
I don't know how to file a pull request, so I put the test results here.

Modified as follows to remove the preprocessor condition at: https://github.com/mysensors/MySensors/blob/development/core/MySigning.cpp#L154-L160

// #if defined(MY_SIGNING_FEATURE)
    // If we do support signing, wait for the gateway to tell us how it prefer us to transmit our messages
    if (destination == GATEWAY_ADDRESS) {
	    SIGN_DEBUG(PSTR("Waiting for GW to send signing preferences...\n"));
	    wait(2000, C_INTERNAL, I_SIGNING_PRESENTATION);
}
// #endif

solves the issue.

At the same time the internal message received from the gateway seems to be erroneously transferred to the receive() function in a sketch. Adding to the sketch

void receive(const MyMessage &message) {
    Serial.println("something came in");
}

gets the message dump:

2310 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
2332 TSF:MSG:READ,0-0-3,s=255,c=3,t=15,pt=6,l=2,sg=0:0100
something came in
2409 TSF:MSG:SEND,3-3-0-0,s=255,c=0,t=17,pt=0,l=10,sg=0,ft=0,st=OK:2.0.1-beta
2496 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=6,pt=1,l=1,sg=0,ft=0,st=OK:0

To solve this I made the following change to _processInternalMessages(void) in MySensorsCore.cpp.
Line https://github.com/mysensors/MySensors/blob/development/core/MySensorsCore.cpp#L407 is replaced with:

	else if (type == I_SIGNING_PRESENTATION) {
	}
	else return false;

The message dump now looks like this:

2250 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
2269 TSF:MSG:READ,0-0-3,s=255,c=3,t=15,pt=6,l=2,sg=0:0100
2331 TSF:MSG:SEND,3-3-0-0,s=255,c=0,t=17,pt=0,l=10,sg=0,ft=0,st=OK:2.0.1-beta
2441 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=6,pt=1,l=1,sg=0,ft=0,st=OK:0

At the API description page https://www.mysensors.org/download/serial_api_20 seems to be another error.
For the internal message t=15 the name description

I_REQUEST_SIGNING 	15 	Used between sensors when initialting signing.

should be changed to

I_SIGNING_PRESENTATION 	15 	Provides signing related preferences.

This I think is relevant both for the master and the development branch.

Anticimex

@jpaulin thanks for the updates. I will have a look at making a pr when I get opportunity. @hek does the documentation issue sound familiar? ;)

Anticimex

@jpaulin I have made a pull request. My solution differs slightly from your as the I_SIGNING_PRESENTATION should never reach the _processInternalMessages function. I do however not have the ability to test so I would appreciate if you could test the PR for me?
Thanks for finding and pointing out the flaws! :D

jpaulin

@Anticimex
I updated my node and GW with your pull request and made some basic tests and it seems to work ok with my sketch. I added MY_DEBUG_VERBOSE_SIGNING and got the following messages.

From the node:

2205 TSM:READY
2220 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
2226 Waiting for GW to send signing preferences...
2280 TSF:MSG:READ,0-0-3,s=255,c=3,t=15,pt=6,l=2,sg=0:0100
2285 Received signing presentation, but signing is not supported (message ignored)
2349 TSF:MSG:SEND,3-3-0-0,s=255,c=0,t=17,pt=0,l=10,sg=0,ft=0,st=OK:2.0.1-beta
2450 TSF:MSG:SEND,3-3-0-0,s=255,c=3,t=6,pt=1,l=1,sg=0,ft=0,st=OK:0

From the GW:

0;255;3;0;9;TSF:MSG:PINGED,ID=3,HP=1
0;255;3;0;9;TSF:MSG:SEND,0-0-3-3,s=255,c=3,t=25,pt=1,l=1,sg=0,ft=0,st=OK:1
0;255;3;0;9;TSF:MSG:READ,3-3-0,s=255,c=3,t=15,pt=6,l=2,sg=0:0100
0;255;3;0;9;Informing node 3 that we do not require signatures because we do not support it
0;255;3;0;9;TSF:MSG:SEND,0-0-3-3,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
0;255;3;0;9;TSF:MSG:READ,3-3-0,s=255,c=0,t=17,pt=0,l=10,sg=0:2.0.1-beta
0;255;3;0;9;Sending message on topic: my_RFM69_gw1-out/3/255/0/0/17
0;255;3;0;9;TSF:MSG:READ,3-3-0,s=255,c=3,t=6,pt=1,l=1,sg=0:0

Do you need some more testing?

Anticimex

@jpaulin thanks for testing. The change should have little effect for people using signing. The issue is for people who does not use it as the node did not wait for a gw response in that case.

korttoma

tried the latest development branch on my test system and the results are good so far. Now I just need to update my "real" system also. Thanks to everyone that participated in solving this!!

korttoma

Updated now also 2 nodes in my "real" system and both now successfully reported Lib version. I did not update the GW. Maybe it is safe to say that the Lib reporting problem is solved now.

Now I will try to look in to the sofSerial RFM69 solution that @scalz is working on, do you have any eta on when you will try to include our solution in the official MySensors development branch?
Any recommendations for the wiring of an W5100/RFM69 Gateway?

scalz

i have removed my stuff for the moment..
for integration in mysensors, it's coreteam work in progress, thx to @Tekka ;)
Once, someone smart told me that 1+1=3, quite true :) and with Tekka i think we can say = 4 ahah
So for an eta.., i prefer not saying anything lol, just i'm back on this for the week, i was playing with other stuff..

jpaulin

@korttoma I have an up and running RFM69 GW with W5100 (actually a W5500, but it's the same). I'm not using softSerial in my setup. With this it is possible to integrate with eg. Moteino or Anarduino, which comes with the radio chip preinstalled on the Arduino compatible boards. But a standard arduino with a separate chip would work as well (lower price). The GW-RFM69 has been stable and flawless, as far as I can see, for quite some time. The SPI for the RFM69 and W5500 uses the following pins in my setup.

RFM69W:

SCK = 13
MISO = 12
MOSI = 11
CS = 10
IRQ = 2

W5500:

SCK = 13
MISO = 12
MOSI = 11
CS = 7

I'll try to share wire diagrams and software setup later. I had to do some tweakings in the ethernet.h file to make it work.

korttoma

@jpaulin I am very interested in your solution. Could it work with a normal Arduino Pro mini 3,3V version? Just so that I know if I need to order some parts or can work with what I have.

jpaulin

@korttoma
Yes, it should work with an Arduino Pro Mini 3.3V. Personally I'm using for the moment an Anarduino. But it's almost the same.

jpaulin

@korttoma
My RFM69 - Ethernet - GW has the following setup.

On the RFM69 radio I'm using D10 as CS pin (Chip Select). This is because in my case the RFM69 radio is hard-wired to pin D10, as with eg. a Moteino or Anarduino. But this is not mandatory. If you use an Arduino Pro Mini you could select another pin for the RFM69 CS-pin on the Arduino, if you want.

The Ethernet W5100 uses the Arduino IDE pre-installed library Ethernet.h. In the library the CS-pin is as well fixed to D10. CS of course has to be unique for each SPI slave device. I changed directly in the Ethenet.h library the CS pin to be D7. But D9, D8 or some other pin is ok to use as well.

Take care, if none of your SPI Slave devices (radio and W5100) uses D10 as CS-pin. In this case make sure to set the D10 pin to OUTPUT in your sketch to keep the Arduino as SPI Master. On the ATmega328 the D10 pin is hard-wired to be the SS pin (Slave Select). The Arduino has to always be the SPI Master and the devices SPI Slaves on the SPI bus to get everything to work.

Another consideration is:
The RFM69 radio uses the SPI functions during the interrupt sequence (Pin D2). The Ethernet.h library won't work correctly if another device uses SPI signalling during an interrupt sequence. The fix I'm using is to block interrupts from the radio-unit during SPI usage of the W5100 unit. I haven't seen any flaws with this fix so far. I've found other people on the internet using the same fix too without issues.

To use RFM69 CS=D10 pin and W5100 CS=D9 pin do the following change in the W5100.h that is part of the Ethernet library.
If you use a Windows computer you'll find the file here:

C:\Program Files (x86)\Arduino\libraries\Ethernet\src\utility\w5100.h

Look for the following lines at the end of the file.

#else
  inline static void initSS()    { DDRB  |=  _BV(2); };
  inline static void setSS()     { PORTB &= ~_BV(2); };
  inline static void resetSS()   { PORTB |=  _BV(2); };
#endif

If you use D9 as CS pin for the W5100 module change as follows:

#else
  inline static void initSS()    { DDRB  |=  _BV(1); };
  inline static void setSS()     { cli(); PORTB &= ~_BV(1); };
  inline static void resetSS()   { PORTB |=  _BV(1); sei(); };
#endif

If you uses, as I do, D7 as CS pin then change as follows:

#else
  inline static void initSS()    { DDRD  |=  _BV(7); };
  inline static void setSS()     { cli(); PORTD &= ~_BV(7); };
  inline static void resetSS()   { PORTD |=  _BV(7); sei(); };
#endif

If you want to keep D10 for the W5100 just add the cli(); and sei(); macros. (cli = clear interrupt. sei = set interrupt).

If you want to use another SPI CS pin than D10 on the RFM69 radio (eg. D9) it should be safe to modify it directly in the sketch with the preprocessor constant:

#define MY_RF69_SPI_CS 9

For the MOSI, MISO and SCK pins use as mentioned above, they are as well hard-wired in the ATmega328 chip to D11,D12 and D13.

If you have any doubts please let me know.

korttoma

Hi @jpaulin thanks for this, I will test it immediately.

Would you mind sharing your complete sketch? The MySensors W5100 GW example has all this Soft SPI stuff, I might make a mistake clearing out all the unnecessary parts.

Btw, do you see any problem with me adding some local "sensors" to this type of GW? I would like to add 4 relays and 4 buttons to it also.

korttoma

It is alive :D

I managed to get a W5100 RFM69 Gateway working.
I wanted minimal change to the w5100.h so I just added the cli(); and sei(); macros.
(I would still like to see your sketch @jpaulin to compare)

There are some random failures in the RF communication but these might be because of the breadboard setup. I just have one temp sensor sending in some value now to test with.

jpaulin

@korttoma
Here's my sketch. It's an MQTT GW with LEDs.

/**
 * The MySensors Arduino library handles the wireless radio link and protocol
 * between your home built sensors/actuators and HA controller of choice.
 * The sensors forms a self healing radio network with optional repeaters. Each
 * repeater and gateway builds a routing tables in EEPROM which keeps track of the
 * network topology allowing messages to be routed to nodes.
 *
 * Created by Henrik Ekblad <henrik.ekblad@mysensors.org>
 * Copyright (C) 2013-2015 Sensnology AB
 * Full contributor list: https://github.com/mysensors/Arduino/graphs/contributors
 *
 * Documentation: http://www.mysensors.org
 * Support Forum: http://forum.mysensors.org
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * version 2 as published by the Free Software Foundation.
 *
 *******************************
 *
 * REVISION HISTORY
 * Version 1.0 - Henrik Ekblad
 *
 * DESCRIPTION
 * The W5100 MQTT gateway sends radio network (or locally attached sensors) data to your MQTT broker.
 * The node also listens to MY_MQTT_TOPIC_PREFIX and sends out those messages to the radio network
 *
 * LED purposes:
 * - RX (yellow) - blink fast on radio message recieved. In inclusion mode will blink fast only on presentation received
 * - TX (green) - blink fast on radio message transmitted. In inclusion mode will blink slowly
 * - ERR (red) - fast blink on error during transmission error or receive crc error
 *
 */


// Enable debug prints to serial monitor
#define MY_DEBUG

// Enables and select radio type (if attached)
//#define MY_RADIO_NRF24
#define MY_RADIO_RFM69

#define MY_RFM69_FREQUENCY   RF69_433MHZ


#define MY_GATEWAY_MQTT_CLIENT

// Set this node's subscribe and publish topic prefix
#define MY_MQTT_PUBLISH_TOPIC_PREFIX "my_RFM69_gw1-out"
#define MY_MQTT_SUBSCRIBE_TOPIC_PREFIX "my_RFM69_gw1-in"

// Set MQTT client id
#define MY_MQTT_CLIENT_ID "mysensors_RFM69-1"

// W5100 Ethernet module SPI enable (optional if using a shield/module that manages SPI_EN signal)
//#define MY_W5100_SPI_EN 4

// Enable these if your MQTT broker requires username/password
//#define MY_MQTT_USER "username"
//#define MY_MQTT_PASSWORD "password"

// Enable MY_IP_ADDRESS here if you want a static ip address (no DHCP)
#define MY_IP_ADDRESS 192,168,1,93

// If using static ip you need to define Gateway and Subnet address as well
#define MY_IP_GATEWAY_ADDRESS 192,168,1,1
#define MY_IP_SUBNET_ADDRESS 255,255,255,0

// MQTT broker ip address or url. Define one or the other.
//#define MY_CONTROLLER_URL_ADDRESS "m20.cloudmqtt.com"
#define MY_CONTROLLER_IP_ADDRESS 192, 168, 1, 90

// The MQTT broker port to to open
#define MY_PORT 1883


// Set blinking period
#define MY_DEFAULT_LED_BLINK_PERIOD 50

// Flash leds on rx/tx/err
// Uncomment to override default HW configurations
#define MY_DEFAULT_ERR_LED_PIN A3  // Error led pin
#define MY_DEFAULT_RX_LED_PIN  A2  // Receive led pin
#define MY_DEFAULT_TX_LED_PIN  A0  // Transmit led pin


#include <Ethernet.h>    // modified for W5500 module and with CS pin = D7 (Default CS pin D10 is already used by RFM69W module)
#include <MySensors.h>

void setup() {
}

void presentation() {
  // Present locally attached sensors here
}

void loop() {
  // Send locally attached sensors data here
}

[solved] RFM69 based nodes unable to report Lib Version

7

11.7k

11.2k

113.0k