Guide: Setting up and testing MQTT Client Gateway

[hek]

(this post currently applies to the development branch)

You can run the MQTT gateway on an Arduino+W5100 ethernet module or the ESP8266. Connect radio and ethernet module exactly like for the normal gateway.

Download and install the MySensors development branch

The use one of the following sketches depending on the hardware you use:
https://github.com/mysensors/Arduino/blob/development/libraries/MySensors/examples/GatewayW5100MQTTClient/GatewayW5100MQTTClient.ino
or
https://github.com/mysensors/Arduino/blob/development/libraries/MySensors/examples/GatewayESP8266MQTTClient/GatewayESP8266MQTTClient.ino

The topic resembles the serial protocol. You must define you preferred subscribe and publish prefix in the sketch. The topic is build like this:
MY_MQTT_PUBLISH_TOPIC_PREFIX/FROM-NODE-ID/SENSOR-ID/CMD-TYPE/ACK-FLAG/SUB-TYPE

Read more about the serial protocol here. An example topic for data received from your gateway could look like this:

mygateway1-out/2/1/1/0/49

If you want to send data to your sensors use MY_MQTT_SUBSCRIBE_TOPIC_PREFIX defined in your sketch. Should be looking like this (using default sketch):

mygateway1-in/2/1/1/0/49

You can test your gateway using mosquitto as broker.

Make sure to set MY_IP_GATEWAY_ADDRESS in gateway pointing to you computers ip-number (on the same lan).

Start the broker (leave this running):
> mosquitto

Now start your gateway and you should see something like this in the gateway log. Note that I have a GPS sensor running in my radio network which send data to the gateway all the time:

0;0;3;0;9;gateway started, id=0, parent=0, distance=0
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;MQTT connected
0;0;3;0;9;read: 2-2-0 s=1,c=1,t=49,pt=0,l=22,sg=0:55.722519;13.018120
0;0;3;0;9;Sending message on topic: mygateway1-out/2/1/1/0/49
0;0;3;0;9;read: 2-2-0 s=1,c=1,t=49,pt=0,l=22,sg=0:55.722519;13.018121
0;0;3;0;9;Sending message on topic: mygateway1-out/2/1/1/0/49

Subscribe to messages (in another shell on your computer)
> mosquitto_sub -v -t 'mygateway1-out/#'

You should see the messages like you saw in the gateway-log showing up like this:

mygateway1-out/2/1/1/0/49 55.722519;13.018121;13
mygateway1-out/2/1/1/0/49 55.722519;13.018114;12
mygateway1-out/2/1/1/0/49 55.722527;13.018120;11
mygateway1-out/2/1/1/0/49 55.722534;13.018122;10

Sending (publishing) messages to your sensor network:
> mosquitto_pub -t 'mygateway1-in/2/1/1/0/49' -m '0,29'

[Mike Cayouette]

@hek said:

MY_MQTT_SUBSCRIBE_TOPIC_PREFIX

You must have read my mind, I was coming here to ask how do you subscribe? LOL

Excellent job as usual.

Mike

[Mike Cayouette]

Not sure I'm doing this properly. I started with only this

#define MY_MQTT_TOPIC_PREFIX "mygateway"

My sketch was able to subscribe and publish to the mygateway topic.

I then went to this:

#define MY_MQTT_TOPIC_PREFIX "mygateway"
#define MY_MQTT_SUBSCRIBE_TOPIC_PREFIX "mygateway1-in"

but my gateway was not able to receive anything from "mygateway1-in" but it was still receiving from "mygateway".

I then went to this:

#define MY_MQTT_TOPIC_PREFIX "mygateway"
#define MY_MQTT_SUBSCRIBE_TOPIC_PREFIX "mygateway1-in"
#define MY_MQTT_PUBLISH_TOPIC_PREFIX "mygateway1-out"

but still not able to send or get anything from "mygateway1-in" or "mygateway1-out"

What am I doing wrong?

Also, when I send the following from mosquitto:
mygateway/106/3/1/0/2

I get a st=fail error.

"0;0;3;0;9;Message arrived on topic: mygateway/106/3/1/0/2
0;0;3;0;9;send: 0-0-0-106 s=3,c=1,t=2,pt=0,l=3,sg=0,st=fail:100"

Any help would would be appreciated.

Thank you,

Mike

[Mike Cayouette]

@hek said:
MY_MQTT_SUBSCRIBE_TOPIC_PREFIX

I am unable to find this anywhere in the mysensors library files. I find MY_MQTT_TOPIC_PREFIX in MyGatewayTransportMQTTClient.cpp, but that is it.

Should MY_MQTT_SUBSCRIBE_TOPIC_PREFIX not be there also?

Mike

[hek]

@Mike-Cayouette
Had to let the pull request containing MY_MQTT_SUBSCRIBE_TOPIC_PREFIX /MY_MQTT_PUBLISH_TOPIC_PREFIX build before merging it. Now it's in place.

"st=fail" means that your gw doesn't receive any ack on the sent radio message.

[Mike Cayouette]

@hek said: "st=fail" means that your gw doesn't receive any ack on the sent radio message

Thank you for the quick response. Do I have to recompile all my sketches using the dev branch? Right now all my sensor sketches are compiled with version 1.5.

Everything works fine with my serial gateway. When I do my testing with the ESP MQTT gateway I turn off my serial gateway to avoid conflicts and when I restart one of my sensors it sends the sketch version and it presents its self and I see this in the gateways serial interface and I also see it in mosquitto. My sketches seem to have no problem sending to the gateway, but the gateway does not appear to be able to send anything to the sketch.

MIke

[hek]

@Mike-Cayouette said:

Do I have to recompile all my sketches using the dev branch? Right now all my sensor sketches are compiled with version 1.5.

No, shouldn't be needed.

[Mike Cayouette]

@hek

I remembered that ack is disabled on my sketches, as I did not see a point in using it with the serial gateway connected to node-red. I will recompile my sketch with ack enabled.

Thank you for your help.

Mike

[hek]

Not the same type of ack. This is the one sent automatically by the NRF radio. You can't disable it in sketch.

[Mike Cayouette]

I have tried a few things but without success, I keep getting the following output on the gateway when I try to send a message to a sensor

0;0;3;0;9;send: 0-0-0-106 s=3,c=1,t=2,pt=0,l=1,sg=0,st=fail:0

Oddly the sensors have no problem sending a message to the gateway. For example when I reset my dimmer the serial output on the gateway is the following

;0;3;0;9;read: 106-106-0 s=255,c=0,t=17,pt=0,l=3,sg=0:1.5
0;0;3;0;9;Sending message on topic: mygateway1-out/106/255/0/0/17
0;0;3;0;9;read: 106-106-0 s=255,c=3,t=6,pt=1,l=1,sg=0:0
0;0;3;0;9;Sending message on topic: mygateway1-out/106/255/3/0/6

I have used an external power supply rather than connect the radio to the 3.3v pin on the nodemcu breakout board but that did not change anything, I also added the following but without luck

#define MY_RF24_PA_LEVEL RF24_PA_LOW

The only thing I have not done was compile the sensor with the library in the dev branch, but I was under the impression I did not need to.

Any help would be appreciated.

Thank you,

Mike

[ahmedadelhosni]

@Mike-Cayouette Can you check what I have found and tell me whether it temporary solved your problem or not ? http://forum.mysensors.org/topic/2193/gatewayesp8266mqttclient-in-development-branch/11

[Mike Cayouette]

@ahmedadelhosni That worked, I can now send messages from my gateway.

Thank you for your help

Mike

[ahmedadelhosni]

@Mike-Cayouette Glad that it worked. Please note that these would be temporary workarounds for MQTT support only. I guess that won't work if the sensor node is far away from the gateway and needs a repeater in the middle, because in that workaround you don't check your look up table.

[Mike Cayouette]

@ahmedadelhosni I have not had to add any repeaters to my network yet so I should be good until a more permanent solution is worked into the library. Thank you again for you solution.

@hek Do you expect a more permanent fix for this solution to be pushed to the dev branch?
http://forum.mysensors.org/topic/2193/gatewayesp8266mqttclient-in-development-branch/11

Thank you,

Mike

[hek]

Ok, verified sending a message to a node here now when repeater mode is enabled. Works fine and message is received.

Here is the change I made. Soon merged into development.
https://github.com/mysensors/Arduino/pull/251

[Lars65]

Great work. I use an ESP8266 12E, and it works just fine.

[John]

All though this topic is related to the development branch, has the topic structure been fixed (except for prefix)?

[ahmedadelhosni]

@hek What about the first issue which I reported about ? Still returns TRUE and returns.

if (strcmp_P(str, MY_MQTT_SUBSCRIBE_TOPIC_PREFIX) != 0) {
	// Message not for us or malformed!
       	return;
}

Is it reproducible at your side ?

[hek]

@ahmedadelhosni

No, I couldn't reproduce any problems with topic naming.

I use the default subscribe/publish prefix in the sketch and can send messages without any problem.

[Samuel235]

@hek said:

You can run the MQTT gateway on an Arduino+W5100 ethernet module or the ESP8266. Connect radio and ethernet module exactly like for the normal gateway.

Have we managed to get a MQTT Gateway working on a Arduino with an ENC28J60 module rather than a w5100?

[hek]

@samuel235

I didn't bother trying. The ENC-library is gigantic and the MQTT implementation used does not support the ENC-module.

[ahmedadelhosni]

@samuel235 I don't really encourage you to use ENC. I spent months trying to make it stable and I failed. There are several manufactures and as far as I know that there are several software workarounds to fix problem in the silicon chips. I turned to use W5100 and it is much better till now.
N.B.: Some people managed to run ENC with good stability but as I have said, there may be different manufacturers so I got confused and tried the W5100.

[Samuel235]

@ahmedadelhosni I have now turned to the W5100 shield myself. Works a charm, super smooth!

[vickey]

I tried to make MQTT client gateway but it is giving following error while compiling

C:\Users\Ahmed\AppData\Local\Temp\build8359488613260197686.tmp/arduino.ar(core_esp8266_postmortem.c.o): In function `abort':
C:\Users\Ahmed\AppData\Roaming\Arduino15\packages\esp8266\hardware\esp8266\2.0.0-rc2\cores\esp8266/core_esp8266_postmortem.c:177: multiple definition of `abort'
GatewayESP8266MQTTClient.cpp.o:C:\Users\Ahmed\Documents\Arduino\libraries\MySensors/core/MyMainESP8266.cpp:46: first defined here
collect2.exe: error: ld returned 1 exit status
Error compiling.

Any suggestions @hek ?

[hek]

@vickey
Sorry haven't seen this error. Which ESP board version do you have installed in the IDE? (See Board manager).

Please do no double post on the forum! Only creates annoyance.

[vickey]

@hek I am using Stable ESP8266 version 2.0.0 from stable release

[ciprian]

If it helps:
in ESP8266 ver. 2.0.0 was added abort in
here
cores/esp8266/core_esp8266_postmortem.c:

void abort() __attribute__((noreturn));

void abort(){
    // cause exception
    s_abort_called = true;
    do {
        *((int*)0) = 0;
    } while(true);
} 

i comment the lines in MySensors/core/MyMainESP8266.cpp and it compiled ok.

[hek]

@ciprian
This has been fixed since a while back.

[dakky]

Hello.

I tried the development branch because I prefer having my gateway not as message broker.
I had some problems resulting in conflicting entries for the gateway in MyConfig.h and the sketch. MyConfig.h always wins for the release it might be more transparent for the user if there is only one location, where the gateway settings are defined: example or MyConfig.h

So much for the feedback, now to the problems:
I use the w5100 version of the sketch with a static IP.

In the serial monitor i see:

0;0;3;0;9;Starting gateway (RNNGA-, 1.6.0-beta)
0;0;3;0;9;Radio init successful.
IP: 0.40.68.10
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Init complete, id=0, parent=0, distance=0
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...

in the log of the moquitto server i see:

1450112394: New connection from 192.168.100.10 on port 1883.
1450112394: New connection from 192.168.100.10 on port 1883.
1450112394: Socket error on client <unknown>, disconnecting.
1450112409: New connection from 192.168.100.10 on port 1883.
1450112409: New connection from 192.168.100.10 on port 1883.

The gateway tries to open a lot of connections and all of them seem to die after a short while
Any idea?

[Mike Cayouette]

@dakky The gateway tries to open a lot of connections and all of them seem to die after a short while

I had a similar problem a while back, not with the mysensors dev branch, but while using the subpub MQTT library on a ESP8266. After upgrading my MQTT (mosquitto) server to version 1.3.5 it corrected to the problem. You may want to take a look at the version you are running.

Mike

[dakky]

Thx 4 replying

mosquitto version 1.4.5 (build date Sun, 08 Nov 2015 21:43:12 +0000)

unfortunetly this seems not to be the problem

EDIT: i updated the PubSubClient lib to version 2.4 instead of the delivered 1.9. Same behavior
Another EDIT: mosquitto itself seems to work fine: OpenHab successfully connects to the broker and delivers its messages

[drock1985]

Hi,

I'm trying to set up the MQTT gateway to work with Home-Assistant; since the serial gateway plugin lacks in binary switches (ex relays, and other things).

I have the MQTT sketch installed on a Nano with W5100 module, and serial output from the Arduino is good. But when I add a sensor to the Gateway, there is no communication.

Here is my output from Arduino, where would I go next?

0;0;3;0;9;Starting gateway (RNNGA-, 1.6.0-beta)
0;0;3;0;9;Radio init successful.
IP: 192.168.86.112
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;MQTT connected
0;0;3;0;9;Init complete, id=0, parent=0, distance=0
ad: 0-0-0 s=0,c=0,t=0,pt=0,l=0,sg=0:
0;0;3;0;9;ver mismatch
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;MQTT connected```

[dakky]

@dakky said:

The gateway tries to open a lot of connections and all of them seem to die after a short while
Any idea?

Narghs!

Seems to be a hardware problem. I changed the w5100 module with an w5100 arduino shield and now it works fine:

0;0;3;0;9;Starting gateway (RNNGA-, 1.6.0-beta)
0;0;3;0;9;Radio init successful.
IP: 192.168.100.10
0;0;3;0;9;Attempting MQTT connection...
0;0;3;0;9;MQTT connected
0;0;3;0;9;Init complete, id=0, parent=0, distance=0

I'm still interested in the reason. Hardware broken or wiring fail?

• pinging the arduino worked
• i can see the connections on the broker

=> basic networking seems to work, but "delivering content" failed. Any guesses?

Strange fact: with the "broken" hardware:

0;0;3;0;9;Radio init successful.
IP: 0.40.68.10

with the shield:

0;0;3;0;9;Radio init successful.
IP: 192.168.100.10

Same sketch, 192.168.100.10 is the correct Ip (obviously)

Greeting
Dakky

[ahmedadelhosni]

@dakky I guess I had that problem once or twice before but I don't pay attention to it as there was other bugs which were more important.

I remember that I just reset the nano and it worked at once. Glad that it worked for you but as you have said, knowing the root cause it important.

Will try to reproduce it again when I have time.

[dakky]

Another question:

I don't have any experience with the classic setup (broker on gateway). But does

The topic resembles the serial protocol. You must define you preferred subscribe and publish prefix in the sketch. The topic is build like this:
MY_MQTT_PUBLISH_TOPIC_PREFIX/FROM-NODE-ID/SENSOR-ID/CMD-TYPE/ACK-FLAG/SUB-TYPE

mean, that the consuming controllers configuration (openhab in my case) has to be changed?
From

sensor-gw1-out/99/3/V_TRIPPED

To

sensor-gw1-out/99/3/1/0/16

in order to represent what is really pushed to the mqtt broker? Because thats what is posted to the broker?

[moskovskiy82]

A little bit cofused. Can this sketch be uploaded to ESP12 set as in the "ESP8266 WiFi Gateway"? Without the arduino?
After uploading it to esp all i get

load 0x4010f000, len 1264, room 16 
tail 0
chksum 0x42
csum 0x42
~ld

[moskovskiy82]

Can more detailed instructions be given on the development branch? As i understand this is not a simple copy the code from here https://github.com/mysensors/Arduino/blob/development/libraries/MySensors/examples/GatewayESP8266MQTTClient/GatewayESP8266MQTTClient.ino paste into arduino IDE compile and upload process?

[hek]

@moskovskiy82

Yes, after the MySensors library has been installed you should be able to compile/upload the sketch and run it.

[moskovskiy82]

Well i had to move all the libs from production branch and extract clean only the development branch in arduino IDE to get it to work.
Afterwards had to do the same thing back to production to compile the humidity sensor as it gave me the following error during compilation

Arduino: 1.6.5 (Windows 7), Board: "Arduino Pro or Pro Mini, ATmega328 (5V, 16 MHz)"

In file included from Mysensors_DHT_Wireless.ino:31:0:
\My Documents\Arduino\libraries\MySensors/MySensor.h:201:3: error: #error No forward link or gateway feature activated. This means nowhere to send messages! Pretty pointless.
  #error No forward link or gateway feature activated. This means nowhere to send messages! Pretty pointless.
   ^
Mysensors_DHT_Wireless:39: error: 'MySensor' does not name a type
Mysensors_DHT_Wireless.ino: In function 'void setup()':
Mysensors_DHT_Wireless:50: error: 'gw' was not declared in this scope
Mysensors_DHT_Wireless.ino: In function 'void loop()':
Mysensors_DHT_Wireless:75: error: 'gw' was not declared in this scope
Mysensors_DHT_Wireless:85: error: 'gw' was not declared in this scope
Mysensors_DHT_Wireless:90: error: 'gw' was not declared in this scope
'MySensor' does not name a type

[moskovskiy82]

Is there some miscompatability betweed development/production branches?

My humidity sensor reports in the serial console just fine.

req id
send: 255-255-0-0 s=255,c=3,t=3,pt=0,l=0,sg=0,st=ok:
req id
send: 255-255-0-0 s=255,c=3,t=3,pt=0,l=0,sg=0,st=ok:
T: 24.60
req id
send: 255-255-0-0 s=255,c=3,t=3,pt=0,l=0,sg=0,st=ok:
H: 87.00

On the gateway i see the following

0;255;3;0;9;read: 255-255-0 s=255,c=3,t=3,pt=0,l=0,sg=0:
0;255;3;0;9;Sending message on topic: mygateway1-out/255/255/3/0/3
0;255;3;0;9;read: 255-255-0 s=255,c=3,t=3,pt=0,l=0,sg=0:
0;255;3;0;9;Sending message on topic: mygateway1-out/255/255/3/0/3
0;255;3;0;9;read: 255-255-0 s=255,c=3,t=3,pt=0,l=0,sg=0:
0;255;3;0;9;Sending message on topic: mygateway1-out/255/255/3/0/3

But in mosquitto...

mygateway1-out/255/255/3/0/3 (null)
mygateway1-out/255/255/3/0/3 (null)
mygateway1-out/255/255/3/0/3 (null)

[moskovskiy82]

Please ignore the last post. Solved by finding out that Openhab doesn't handle Node ID assigning.

[tante ju]

Hi,

Just tried to set up an ESP-12E as MQTT gateway and noticed one bug in the actual development branch:
As soon as one of the three LED PIN assignments is not defined (commented out), the Example sketch will lock itself after initializing NRF radio, and running DHCP. So, no further NRF communication nor signup at the broker. WDT steps in then.

Just defining all three LEDs, without any other parameter or use the LED blinking feature, and everything is fine.

[hek]

@tante-ju

http://forum.mysensors.org/topic/2749/1-6-gateway-possible-error/8

[gregl]

Im lost...

Ive just setup a new MQTT client gateway ( from link at the top of this post..) and have a few nodes successfully sending data to a mosquito server.

What im trying to now do is, using the RelayActuator sketch, is control the relay via a MQTT message...but im confused as to what i need to send it!

Here is the serial output from the RelayActuator sketch:

Starting repeater (RNNRA-, 2.0.0-beta)
Radio init successful.
send: 9-9-0-0 s=255,c=3,t=15,pt=1,l=1,sg=0,st=ok:0
send: 9-9-0-0 s=255,c=0,t=18,pt=0,l=10,sg=0,st=ok:2.0.0-beta
send: 9-9-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=ok:0
send: 9-9-0-0 s=255,c=3,t=11,pt=0,l=5,sg=0,st=ok:Relay
send: 9-9-0-0 s=255,c=3,t=12,pt=0,l=3,sg=0,st=ok:1.0
send: 9-9-0-0 s=1,c=0,t=3,pt=0,l=0,sg=0,st=ok:
Init complete, id=9, parent=0, distance=1

So what message do i send it from another MQTT client ?
ive been trying things such as mygateway1-in/9/0/1/1 with message of 0 or 1

Pls help !

[hek]

What happens if you send to child sensor 1 (ack disabled)

mygateway1-in/9/1/0/1

[gregl]

Hi @hek
Nothing...
there is no message received from the node. ( based off the serial output)

How can i serial.print out all the messages it receives?

[gregl]

@hek

mmm..perhaps there is a problem between mosquito and the mqttclient gateway. As ive now attached a serial monitor to the mqttclient gateway and sending the message doesnt appear in the console. I assume it should...

Opening port
Port open
0;255;3;0;9;read: 9-9-0 s=255,c=3,t=15,pt=1,l=1,sg=0:0
0;255;3;0;9;Sending message on topic: mygateway1-out/9/255/3/0/15
0;255;3;0;9;read: 9-9-0 s=255,c=0,t=18,pt=0,l=10,sg=0:2.0.0-beta
0;255;3;0;9;Sending message on topic: mygateway1-out/9/255/0/0/18
0;255;3;0;9;read: 9-9-0 s=255,c=3,t=6,pt=1,l=1,sg=0:0
0;255;3;0;9;Sending message on topic: mygateway1-out/9/255/3/0/6
0;255;3;0;9;read: 9-9-0 s=255,c=3,t=11,pt=0,l=5,sg=0:Relay
0;255;3;0;9;Sending message on topic: mygateway1-out/9/255/3/0/11
0;255;3;0;9;read: 9-9-0 s=255,c=3,t=12,pt=0,l=3,sg=0:1.0
0;255;3;0;9;Sending message on topic: mygateway1-out/9/255/3/0/12
0;255;3;0;9;read: 9-9-0 s=1,c=0,t=3,pt=0,l=0,sg=0:
0;255;3;0;9;Sending message on topic: mygateway1-out/9/1/0/0/3
0;255;3;0;9;read: 1-1-0 s=0,c=1,t=0,pt=7,l=5,sg=0:26.8
0;255;3;0;9;Sending message on topic: mygateway1-out/1/0/1/0/0
0;255;3;0;9;read: 1-1-0 s=1,c=1,t=0,pt=7,l=5,sg=0:26.7
0;255;3;0;9;Sending message on topic: mygateway1-out/1/1/1/0/0

as you can see no incoming messages from mosqutto.

[gregl]

I can see in mosquitto logs a subscription from the gateway

1452645694: mosquitto version 1.3.4 terminating
1452645694: Sending CONNACK to mysensors-1 (0)
1452645694: Received SUBSCRIBE from mysensors-1
1452645694: 	mygateway1-in/+/+/+/+/+ (QoS 0)
1452645694: Sending SUBACK to mysensors-1

So looks like i need to add an extra level to the message i send!
So i just tried sending a message to topic:

mygateway1-in/9/1/0/1/1

and now i can see in my serial outputs of both the mqttclient gateway and the relaysketch node the message received!
( but its not swithing my relay yet! )

@hek is this the correct MQTT formatting:
MY_MQTT_PUBLISH_TOPIC_PREFIX/FROM-NODE-ID/SENSOR-ID/CMD-TYPE/ACK-FLAG/SUB-TYPE

[gregl]

Ok got it... ( sorry for making mess of this discussion!)
If i publish to
mygateway1-in/9/1/1/0/2 and send message values of 0 or 1 it works....

Hope this helps someone else!

Greg

[tante ju]

@hek
That thread does not directly touch on the issue I noticed, although that might be connected.

Some other questions:
Why is the topic structure as it is? I like my MQTT topics in hierarchical order. So, having different prefixes for send and receive (in and out) is not nice, but ok. BUt why is the ACK flag before the data type, which is usually linked to a value or variable in the sensor? I would consider ACK to be a sub-value of that one.

And is there a way to have more speaky names instead of numbers? So, the source does not have that option and as that gateway is now included in the library I only see the possibility of changing the source in the library, which will disconnect from updates. Not nice. What about an option of excluding gatewayTransportSend and incomingMQTT via define from the library and define then in the own sketch?

[moskovskiy82]

What is the correct way to run DHCP? Tried commenting out
#define MY_IP_ADDRESS
but that didn't help.

Also is it possible to use fqdn for the mqtt broker instead of the ip?

[gregl]

@moskovskiy82

I also tried to use DHCP and fqdn for mosqitto, but no luck....so I cant help but I too would like to know if its possible.

[hek]

@gregl

A PR was merged 5 days ago where you can specify
MY_CONTROLLER_URL_ADDRESS

https://github.com/mysensors/Arduino/pull/375

I haven't verified it myself yet though.

[mbj]

Finally I have the ESP8266 gateway running with Mosquitto and Openhab. Because the ESP8266 Gateway installation was painless and Mosquitto and Openhab the major problems I have posted some general info at this link in case anyone is interested, see http://forum.mysensors.org/topic/3005/switching-from-v1-5-mqtt-gateway-to-esp8266-mqtt-client-gateway.
(Sorry if this is the wrong procedure but found no way to add other tags to a message in this thread).

[gregl]

@hek
Thanks Hek - will give it a try soon!

[moskovskiy82]

I have been successfully running the MQTT client gateway receiving the data from sensors. Now build the first mysensors relay and would like to know if the MQTT client gateway can act as a transmitter?

I have assigned the nod id of 55 to the relay sensor and trying to send
mosquitto_pub -t mygateway1-in/55/1/0/0/3 -m "1"

But doesn't seem to help

[hek]

Yes, it should work as a transmitter as well. What does the log say on gateway and node?

[moskovskiy82]

Resolved. Power supply was the problem

[mbj]

@mbj said:

Finally I have the ESP8266 gateway running with Mosquitto and Openhab. Because the ESP8266 Gateway installation was painless and Mosquitto and Openhab the major problems I have posted some general info at this link in case anyone is interested, see http://forum.mysensors.org/topic/3005/switching-from-v1-5-mqtt-gateway-to-esp8266-mqtt-client-gateway.
(Sorry if this is the wrong procedure but found no way to add other tags to a message in this thread).

Do not know if this info is of any interest but my ESP8266 MQTT gateway (based on a NodeMCU board) has been running without hiccups for almost a month now. So it has been very stable. I have also made a very short test using a Wemos D1 Mini for the gateway and this one seemed to be working as expected as well (not very surprised but you never know until it has been tested :-)).

[DirkB19]

Hi,

I've uploaded the GatewayW5100MQTTClient.ino to a UNO with W5100.

I have changed IP adresses to my Mosquitto broker (on Raspberry Pi) as instructed.

However,
I don't see the gateway connecting to the Mosquitto (in the /var/log/mosquitto/mosquitto.log on RPI) and also I don't see anything in the serial monitor. I've got 10+ sensornodes that work well when using another gateway (to OpenHab).
What am I overlooking here ?

The same hardware UNO+W5100 worked before as MQTTgateway(+broker), so I'm guessing it's not the hardware?

Any help is appreciated.
Thanks,
DirkB

[gregl]

@DirkB19

Hi Dirk - did you change the MAC address ( these need to be unique too and it could have the same one "DEADBEEFFEED" as your existing GW )

[mbj]

@DirkB19 If you do not see anything in the serial monitor for the gateway then the problem must be with the gateway itself. Have you observed that there is a special #define MY_W5100_SPI_EN to uncomment when compiling for an Uno with the shield installed ? If not, you can try that.

[DirkB19]

@mbj said:

MY_W5100_SPI_EN

Sorry for this NOOB question, but where is this to be uncommented.
I don't see it in the gateway sketch ?
I use this sketch GatewayW5100MQTTClient.ino

[mbj]

@DirkB19

It is looking like this, just uncomment the #define - line:

// W5100 Ethernet module SPI enable (optional if using a shield/module that manages SPI_EN signal)
//#define MY_W5100_SPI_EN 4

[DirkB19]

@mbj

Thanks for the help, but I'm still very confused because OK, I found the line to uncomment.
I uncommented, but still nothing on serial monitor.
So then I see lines below this in the sketch and I realize that my radio is wired as follows ;
SCK -- Pin 13
MISO -- Pin 12
MOSI -- Pin 11
CS -- Pin 6
CE -- Pin 5
?
But there's no Pins 14,15,16 to be found on my W5100/UNO ? So I'm very confused ...

#if !defined(MY_W5100_SPI_EN) && !defined(ARDUINO_ARCH_SAMD)
#define MY_SOFTSPI
#define MY_SOFT_SPI_SCK_PIN 14
#define MY_SOFT_SPI_MISO_PIN 16
#define MY_SOFT_SPI_MOSI_PIN 15
#endif

// When W5100 is connected we have to move CE/CSN pins for NRF radio
#define MY_RF24_CE_PIN 5
#define MY_RF24_CS_PIN 6

[martinhjelmare]

@DirkB19

In this case I think pin 14-16 are meant to be aliases for analog pins A0-A2.

Read the build page for the Ethernet gateway. You have to wire the radio in a non standard way in combination with the 5100.

But it's optional when using the shield. You could enable the SPI for that instead, like you were talking about a few posts up.

[mbj]

@DirkB19 The Arduino pinouts are, to say the least, confusing and it helps to look at one of the pinout sketches you can find on the net, for example http://marcusjenkins.com/wp-content/uploads/2014/06/ARDUINO_V2.png. So the pins are, as Martin said, (A0-A2) which must be exactly as you had them wired before if you had a "previous version" working MQTT Ethernet gateway. I have such a gateway (nowadays used as a spare) but built with a nano and a W5100 unit and it compiled and worked right away using the GatewayW5100MQTTClient.ino sketch from the development branch.

As you see nothing in the serial monitor there must be something else. Even if you upload the sketch to a completely barebone UNO without radio or anything else attached you will get a message in the serial monitor, in this case:

0;255;3;0;9;Starting gateway (RNNGA-, 2.0.0-beta)
0;255;3;0;9;Radio init failed. Check wiring.

So if you see nothing at all there is something really basic wrong.
Presumably your sketch compiles OK so I assume you have exchanged all Arduino libraries to the libraries which belong to the Mysensors development branch.

[DirkB19]

@mbj

Aha !! Finally, it WORKS
mbj gave me the right hint about using the development libraries !!!

So for others to learn from my mistakes, to get a UNO&W5100 shield to work with this MQTT Client Gateway sketch, you need your nRF24 radio connected to these pins nrs. on the W5100 shield ;
SCK -- Pin 13
MISO -- Pin 12
MOSI -- Pin 11
CS -- Pin 6
CE -- Pin 5

AND (!) you need to install the libraries from the development branch.
(I first removed the 1.5 libraries)

I just hope that installing the libraries from development branch will not get me into trouble with my the rest of my nodes (upladed with 1.5).

Is there an easy way to switch between 1.5 & development branch libraries when uploading sketches ?

Anyway, thanks to all those who responded and special thanks to mbj.

Some things are so obvious, but for newbies (like me) it can be a struggle ...

[martinhjelmare]

@DirkB19

You can use git to easily change between different branches with libraries and sketches.

The take home message from the first sentence in the top post of this thread, is that dev branch libs are required with this sketch.

You should take care not to mix old style sketches with dev branch library and vice versa. Node to node to gateway comm should be OK as long as all nodes and gateway know the message types being used.

[moskovskiy82]

In most examples for the openhab i see the examples being put like

Dimmer Light_R {mqtt=">[server:/mySensors/118/0/V_DIMMER:state:*:default]"}

Is there a way with the MQTT client gateway to have the messages defined as
V_DIMMER e.t.c?

[27maximilian]

@moskovskiy82

I believe these examples are mostly referring to the older MQTT (Broker) Gateway. As @hek said in post #1 the client gateway resembles the serial protocol. The difference between the two is on the one hand a slightly modified topic structure and on the other hand that instead of names the topics make use of the number representation of the message types (see the Serial API).

In your example you would have to use 3 instead of V_DIMMER as CMD-TYPE in your topic-configuration.

If you are wondering whether there is a library which you can included in your items-file in openhab such that you can use the name representation instead of the values for the message type, then I am afraid I don't have a solution for you. I am, moreover, pretty sure that this won't be possible with the mqtt-binding but would require a specific mysensors binding for openhab.

[moskovskiy82]

And for the better understanding - why were the inbound and outbound topics separated? Are there any drawbacks in unifiing when using openhab as a controller?

[hek]

If they weren't separated all outgoing messages would be looped back to the gateway.

[27maximilian]

Update: I guess I was answering while @hek also posted his answer.

I didn't do it, so I can only guess. But I believe this was done because otherwise the MQTT-topic would not include any information who the recipient is. This would on the one hand (1) make the GW send every message from a sensor back to the sensor and on the other hand (2) could cause infinite loops with controllers such as openhab.

Regarding (1), imagine the topic-strucutre would be like:

mysensors/6/1/1/0/0

for a temperature sensors with node id 6 and child id 1 without the in/out separation. If the sensor now sends a message with the temperature, e.g. 23, to the GW, then the GW posts a message with the payload 23 to the topic. However, because there is no in/out separation, the GW listens to all messages on the topic

mysensors/#

and hence receives it own message, which it then sends back to the originating sensor. Not only would this double the traffic in your radio network, it might also be problematic for bidirectional sensors, such as a relay, because it would also cause an infinite loop.

Regarding the infinite loop, I will use (2) as an example but the same would happen for a bidirectional sensor. Consider you would have a relay sensors, that can be locally or remotely trigged, as for example a lights relay with switch. In openhab this relay could be defined like this:

Switch LIGHT_test {mqtt=">[mosqMQTT:mysensors/2/1/1/0/2:command:*:default],>[mosqMQTT:mysensors/2/1/1/0/2:command:default]"}

Imagine the local switch is trigged on the relay, then the sensor sends a message to the GW, which posts this message on the mqtt-topic:

mysensors/2/1/1/0/2

The OH receives this message, updates the state of this item but also posts a message on the topic because of the item configuration, which would generate an infinite loop. If the sensor would do the same then we would have an infinite loop, in which the number of messages would double each iteration.

[moskovskiy82]

@27maximilian
Ok got the point. But in your example with the local button on the relay. How would one define the item in openhab?
Seems two items need to be defined - buton with the gateway-in topic and a relay with gateway-out topic? And than implement a rule to update the relay status when the button state was changed? As otherwise the controller wouldn't know the state of the relay?

Seems i will face exact same problem as in this topic when i finish the code MQTT Dimmer

[27maximilian]

I am not sure because I never did this myself but I would try to not only use the command class but also the state class in OH.

Something like this could work:

Switch LIGHT_test {mqtt=">[mosqMQTT:mysensors-in/2/1/1/0/2:command:*:default],<[mosqMQTT:mysensors-out/2/1/1/0/2:state:default]"}

Please be aware that this is just an idea. I never implemented this so far, neither did I test the code. The syntax could be wrong or it might conceptually not work.

[Martin Tellblom]

I use this gateway 2.0 and I'm very pleased with it, but...... I would like to be able to use it as a network gateway as well.
Is this possible?

[Samuel235]

Sorry if i shouldn't be bringing an old thread back to life, however it is a sticky and therefor I assume you guys would be okay with this being brought up to date.

I have a pretty simple question really;

My W5100GatewayMQTTClient is in a loop, as shown below on the serial monitor, it seems to be reconnecting to the MQTT server every 8-10 seconds. Is this normal behaviour or not? I'm by no means asking for a solution (if i get one i will be sure to post here for others to revert back to if needed), but if you would like to suggest something, then it would be appreciated. I'm looking more for just a confirmation of if this is normal behaviour as of MySensors 2.1 or not.

I used to run this client gateway back when MySensors V1.6 was in the development branch back in late 2015 and It didn't do anything like this back then.