Node doesn't receive data from RPI MQTT Gateway

[cvdenzen]

Is the mysgw output looking good? The sent message doesn”t include the message type (3). Has anybody tested this with a topic that contains a slash?

[evb]

@cvdenzen I don't have experience with a Raspberry Pi as MQTT gateway, but your last remark about the slash is maybe a path to debug...

From the docs for a Arduino MQTT gateway:
The topic is build like this: MY_MQTT_PUBLISH_TOPIC_PREFIX/FROM-NODE-ID/SENSOR-ID/CMD-TYPE/ACK-FLAG/SUB-TYPE
Read more about the serial protocol here. An example topic for data received from your gateway could look like this:

mygateway1-out/2/1/1/0/49

If you want to send data to your sensors use MY_MQTT_SUBSCRIBE_TOPIC_PREFIX defined in your sketch. Should be looking like this (using default sketch):

mygateway1-in/2/1/1/0/49

Can you try to change your mqtt topic build?

[cvdenzen]

@evb It took some time (have also another home renovation project), but now I have set up a test gateway.
The mqtt topic with a slash in its name seems to be the problem. I changed the mqtt topic to mysensorsa and now the sketch receives the message. The mqtt gateway outputs:

Aug 15 22:50:33 DEBUG GWT:IMQ:TOPIC=mysensorsa/157/0/1/0/3, MSG RECEIVED
Aug 15 22:50:33 DEBUG TSF:MSG:SEND,0-0-157-157,s=0,c=1,t=3,pt=0,l=2,sg=0,ft=0,st=OK:35
Aug 15 22:50:34 DEBUG TSF:MSG:READ,157-157-0,s=0,c=1,t=2,pt=1,l=1,sg=0:1
Aug 15 22:50:34 DEBUG GWT:TPS:TOPIC=mysensors/all/157/0/1/0/2,MSG SENT
Aug 15 22:50:34 DEBUG TSF:MSG:READ,157-157-0,s=0,c=1,t=3,pt=2,l=2,sg=0:35
Aug 15 22:50:34 DEBUG GWT:TPS:TOPIC=mysensors/all/157/0/1/0/3,MSG SENT

I will have a look at the MySensors code and if I cannot find the problem, maybe I will change my mqtt subscription topic.

[cvdenzen]

@cvdenzen
I have added a GATEWAY_DEBUG line in core/MyProtocol.cpp at line 122:

	for (str = strtok_r(topic + strlen(MY_MQTT_SUBSCRIBE_TOPIC_PREFIX) + 1, "/", &p);
	        str && index < 5;
	        str = strtok_r(NULL, "/", &p), index++
	    ) {
	    GATEWAY_DEBUG(PSTR("GWT:TPS:Index=%d, mqttstr=%s\n"), str);

My conclusion is that the length of the mqtt subscription topic is the problem. The first call shows that it thinks "rs" is part of the topic beyond the PREFIX. And "rs" gets translated to "0" in the outgoing mqtt message, shifting the real items one position further and dropping the last item.
Later on I will have a look at the cause of this limitation (or maybe someone else is quicker than I am).
The output of mysgw, first try is with topic carl/mysensors, second try (that succeeds) is with mqtt topic c/mysensors:

Aug 15 23:53:19 DEBUG GWT:TPS:TOPIC=mysensors/all/157/255/3/0/21,MSG SENT
Aug 15 23:53:24 DEBUG GWT:IMQ:TOPIC=carl/mysensors/157/0/1/0/3, MSG RECEIVED
Aug 15 23:53:24 DEBUG GWT:TPS:Index=0, mqttstr=rs
Aug 15 23:53:24 DEBUG GWT:TPS:Index=1, mqttstr=157
Aug 15 23:53:24 DEBUG GWT:TPS:Index=2, mqttstr=0
Aug 15 23:53:24 DEBUG GWT:TPS:Index=3, mqttstr=1
Aug 15 23:53:24 DEBUG GWT:TPS:Index=4, mqttstr=0
Aug 15 23:53:24 DEBUG GWT:TPS:TOPIC=mysensors/all/0/157/0/1/0,MSG SENT
Aug 15 23:54:42 DEBUG GWT:IMQ:TOPIC=c/mysensors/157/0/1/0/3, MSG RECEIVED
Aug 15 23:54:42 DEBUG GWT:TPS:Index=0, mqttstr=157
Aug 15 23:54:42 DEBUG GWT:TPS:Index=1, mqttstr=0
Aug 15 23:54:42 DEBUG GWT:TPS:Index=2, mqttstr=1
Aug 15 23:54:42 DEBUG GWT:TPS:Index=3, mqttstr=0
Aug 15 23:54:42 DEBUG GWT:TPS:Index=4, mqttstr=3
Aug 15 23:54:42 DEBUG TSF:MSG:SEND,0-0-157-157,s=0,c=1,t=3,pt=0,l=2,sg=0,ft=0,st=OK:35
Aug 15 23:54:42 DEBUG TSF:MSG:READ,157-157-0,s=0,c=1,t=2,pt=1,l=1,sg=0:1
Aug 15 23:54:42 DEBUG GWT:TPS:TOPIC=mysensors/all/157/0/1/0/2,MSG SENT

[cvdenzen]

The problem is clear: the subscription topic that I use contains a wildcard: “+/mysensors”.
I send my message with prefix “carl/mysensors”.
The MySensors code assumes the length of the prefix I send is the same as the length of the subscription prefix. But it isn’t because of the wildcard character.
Is there anybody who likes to change the code to make it work? Or should I try it myself?
I see two possible solutions:

1. Parse the message from end to begin and not from begin to end.
2. Try to match the wildcard character(s) with the sent string.

[cvdenzen]

I have made some modifications to the code, tested it and it seems to work.
There is a pull request https://github.com/mysensors/MySensors/pull/1497 but I don't know how this will be handled by the MySensors developers.

Can anybody give me a link or some information about the development process? A review of my code will be necessary, I am not a C programmer.

[mfalkvidd]

Contribution guidelines are available at https://www.mysensors.org/download/contributing

[cvdenzen]

@mfalkvidd Thank you for the link. I followed the instructions, had to download/compile some packages.
It will take some time for me, I have to solve this error:

 git commit -m "Accept wildcards in mqtt subscription prefix"
cppcheck: Failed to load library configuration file 'std.cfg'. File not found
cppcheck: Failed to load library configuration file 'avr'. File not found
nofile:0:0: information: Failed to load the library avr [failedToLoadCfg]

Few minutes later, resolved, had to add CFGDIR to make cppcheck:

sudo make install FILESDIR=/usr/share/cppcheck CFGDIR=/usr/share/cppcheck

[cvdenzen]

It seems like building MySensors with my edit failed but I don’t understand the error messages.
The first error: https://ci.mysensors.org/job/MySensors/job/MySensors/job/PR-1497/2/warnings2Result/new/file.702256833/source.9006459693321295180/#1

[mfalkvidd]

@cvdenzen I don't understand the build system, but I have permission to restart builds. Sometimes that helps. I've restarted the build for your PR. Hopefully we'll get a better result this time.

[mfalkvidd]

https://ci.mysensors.org/job/MySensors/job/MySensors/job/PR-1497/3/warnings2Result/new/#1 looks better now

[cvdenzen]

@mfalkvidd Thank you!