MQTT Broker gateway

[Zeph]

At first, I thought MQTT would be a great extension to the functionality, in particular because it might systematize the idea of routing messages to more than one destination - eg: to both a HA controller and to a cloud storage service.

So I saw these possible advantages of MQTT:

• Route to multiple destinations, dynamically updated
• Subscription wildcards to filter for desired subsets
• Possibly easier to build plug-ins for some HA controllers

As I look deeper into it, I'm coming to wonder about that.

• There are lighter weight ways to send to more than one destination
• Does subscription wildcarding meet the filtering needs well (below)?
• Is it easier or not?

I'm wondering what subscription wildcards really get used - how valuable is the type of filtering functionality offered by MQTT for our purposes? The meaingful patterns I would imagine are:

• everything
• everything from one node
• everything from one child of one node
• everything of type V_RAIN from one child of one node (ie: one variable)
• everything of type V_RAIN from anywhere

Are any of these (or other options) in use except "everything"? Do you anticipate that they will be?

Just as "route to multiple destinations" could be lighter, filtering could be done differently as well. For example, suppose you wanted to receive all report of inside temperature (7 sensors in your network) or humidity (5 sensors), but not outside temp or humidity and not other v codes. That might take 12 subscriptions - and they would have to be manually set up, or make use of manually entered metadata tables in the controller plug-in (eg: the plug-in might be configured to know which nodes had temp and/or humidity, and which ones were considered "inside", if it was written to handled that).

I could imagine that even if the subscription filtering wasn't highly useful, MQTT might make it easier to create plug-ins for some HA controllers, than using semicolon separated values (serial text format). Is that the case?

I'm very open to hearing other advantages of MQTT, I'm not against it, just trying to understand the benefits vs cost aspect better.

[Zeph]

The other extension that occurs to me, in the gateway-as-transparent-switch model, is that if we ever wanted to encode binary values OTA from the gateway to the node, the plug-in would have to tell the gateway which payload_type to use. In the Semicolon Separted Values format, that could be another parameter, eg:

<node id>;<child id>;<command>;<ack>;<v_code>;<payload_type>;<value>

Or it could be added to one of the other fields with a comma sub-delimiter, etc.

How would a controller plug in tell the MQTT gateway to use payload type P_UNIT16 to send to node 7?

I'm thinking that it would be done in the value string, rather than encoded into the topic. So maybe the value string goes from "321" to "4,321" or something like that.

[John]

@Damme
What happens if a node does not receive it's address when it requests one (because of failed send). It is not registered within the broker?

[ToSa]

@Damme said:

@ToSa I'll update mqtt later with option to not translate V_TYPE into name string but to keep it as a byte, that would save a lot of memory and might solve problem you have. I'll keep you posted

I could not follow the discussions the last few days - will definitely have a look over the weekend and let you know if that worked. Thanks!

[John]

@Damme
When the MQTT gateway prints information about itself like "started", is this done with node id 0 or 255?

[Damme]

@John 'started' is only a internal log message, I've changed how that part works.

I've also fixed a couple of bugs and will test the changes tomorrow before I publish them.
New functionallity there user can choose of MQTT should translate ID to V_TYPE or not too.

[John]

@Damme
Ok, my plugin will then just try to reconnect.

Ok, i will check that then probably the day after tomorrow, i will make my code take no translation as first and if not found try the naming (internally), because the server takes over the semantics anyway because it also is going to include a MQTT broker service.

Currently the MQTT implementation is being tested with an end user. Maybe he will help test the new possibility without V_TYPE translation when your changes are implemented.

[John]

@Damme
When does exactly the messages like "MyMQTT/[NODEID]/255/V_SKETCH_NAME" and/or "MyMQTT/[NODEID]/255/V_SKETCH_VERSION" appear? is this only when an address is assigned or always when a node is started?

[Damme]

@John then node is started, There was a bug not processing that message which is fixed now.
hasn't been pulled to official yet but can be found at https://github.com/Damme/MQTTGateway/commit/f10843f5efd9c37900624fe57275be11c69e3942

[John]

@Damme
Thnx, was struggling with this with a user. Will need to wait for the next release :).

[Francois]

@Damme when will the new release be available?

[adrianmihai83]

Dear colleagues,

I have been using OpenHAB for a couple of days now, everything is going the right way, I've been using it with mosquitto broker for Windows, everything is solid, working well an fast. Decided to go a step further and use the MQTT Gateway presented here.

I've made a MQTT Gateway but get the "Not connected to broker" message from OpenHAB. First of all, the Gateway is an arduino uno with a w5100 shield on top in which I inserted the RF module (NRF24L01). Configured my ip in the gateway sketch (192.168.1.11) and uploaded to the arduino.

The first test was to connect with a phone (using MQTT from Playstore - used it many times before) and with a PC (using WMQTT Utility - used it many times before) and things don't look that promising, the problem beeing that client somethimes disconnects, and I cannot receive the message published in a topic (topic that I am subscribed to on the phone, pc and tablet using MQTT clients). I somethimes get the message in the serial monitor of the arduino, somethimes not.

Configured openhab.cfg to use the mqtt broker at 192.168.1.11 (before I was using mosquitto on pc 192.168.1.10 and working without problems) and everythime I change state of something in the dashboard of openhab I get this Broker Connection not Started even if after starting openhab I get serial output from arduino for some time, after this everything drops. Here is the serial output:serial_output.txt

Last two entries are: I opened the MQTT client on my phone, connected to MQTT Gateway and subscribed to topic home/# and the last one is me publishing from the phone to topic home/test the message "test message". None of the messages come back to the clients on the phone, tablet, pc, all of them subscribed to home/#

And here is the output from openhab: openhab_output.txt Ignore the samsung connect stuff, I have pull it off the network, i have managed to get it working well.

Best regards,

Adrian

[Damme]

@adrianmihai83 "home/openHAB/out/Temperature_GF_Corridor/state" is probobly long string, 47 chars. I have not investigated exacly how long it can be but keep it short. And the adress layout is also wrong, is should be something like MyMQTT/21/1/V_LIGHT . MQTTGateway is not compatible with the mosquitto broker you're using.

[adrianmihai83]

@Damme , your instructions were highly appreciated, got it working the first time. I will start creating the network now, I guess that everything will be ok. Will follow this forum for info and knowledge.

Now I will migrate openHAB to Rasberry Pi and add all the sensors one by one, there are a few.

A question though: I am using the humidity sketch and get MyMQTT/0/1/V_TEMP and MyMQTT/0/0/V_HUM reading fine but I cannot get sketch name... I don't know what I am doing wrong... Read above and I guess I have to wait for new version?

Thank you.

Adrian

Edit: Read more about Rasbery Pi capabilities regarding openHAB and I think I will use a cubietruck... Does anybody use one, cubieboard 2 or cubietruck?

[Damme]

@adrianmihai83 Sketch-name was a bug that I fixed in development branch, thought @hek updated master branch also. I can post the fix later on in here.

[adrianmihai83]

Maybe a dumb question but I don't figure things, here is my problem:

Uploaded the relay sketch to the same arduino that I used for humidity/temperature node (I don't know if this is relevant, the network has only one node, this one). Modified the pin that the relay is connected to: #define RELAY_1 4.

I am using the MQTT Gateway openHAB and cannot activate the relay...

Serial output of relay sketch is:
repeater started, id 0
send: 0-0-0-0 s=255,c=0,t=18,pt=0,l=3,st=ok:1.4
send: 0-0-0-0 s=255,c=3,t=6,pt=1,l=1,st=ok:0
send: 0-0-0-0 s=255,c=3,t=11,pt=0,l=5,st=ok:Relay
send: 0-0-0-0 s=255,c=3,t=12,pt=0,l=3,st=ok:1.0
send: 0-0-0-0 s=1,c=0,t=3,pt=0,l=3,st=ok:1.4

Serial output of gateway is:
Started!
0;0;3;0;9;read: 0-0-0 s=255,c=0,t=18,pt=0,l=3:1.4
0;0;3;0;9;read: 0-0-0 s=255,c=3,t=6,pt=1,l=1:0
0;0;3;0;9;read: 0-0-0 s=255,c=3,t=11,pt=0,l=5:Relay
0;0;3;0;9;read: 0-0-0 s=255,c=3,t=12,pt=0,l=3:1.0
0;0;3;0;9;read: 0-0-0 s=1,c=0,t=3,pt=0,l=3:1.4

My openHAB configuration is:

Switch node2_sw1 "sw2" (node2,all) {mqtt=">mysensor:MyMQTT/0/1/V_LIGHT:command:ON:1],>[mysensor:MyMQTT/0/1/V_LIGHT:command:OFF:0]"}

What am I doing wrong?

Thank you!

[Francois]

If I run the MQTTGateway my temperature shows in fahrenheit when I run the same nodes on a SerialGateway it is showing Celsius why will this be the case?

[adrianmihai83]

Downloaded from https://github.com/Damme/MQTTGateway latest MQTT Gateway (the one that responds on serial with "Ok!" not "Started!", with the modifications to MySensors.cpp) but no difference, still no sketch name and with the above settings, openHAB cannot control the relay sensor...

I used the same arduino that is working with Temp_Humidity sensor so the setup is good, the relay pins are specified in the sketch. I used the Clean_Eeprom_Config before uploading, I even used other Arduino Uno, no diffrerence, still cannot control the relay sensor from openHAB.

Anyone has a clue, a hint or something? The serial output from both the sensor and the gateway, together with openHAB configuration are two posts up.

Edit: 3 hours later and managed to make many of the sensors (Door, Distance, Dust, Gas, Humidity, Light, Motion) everyone working in conjunction with MQTT Gateway and openHAB except the Relay one...

Any help is appreciated.

Regards,
Adrian

[adrianmihai83]

Callback for incomming messages does not occur so I guess that the message does not get to the relay sensor. Is there someone that is using this with MQTT Gateway and openHAB?

[Francois]

@adrianmihai83 I was testing this today and it was working with the MQTTGateway. Here is my config file same as above:

Switch node2_sw2 "Study Lights" (node2,all) { mqtt=">[mysensor:MyMQTT/2/1/V_LIGHT:command:ON:1],>[mysensor:MyMQTT/2/1/V_LIGHT:command:OFF:0],<[mysensor:MyMQTT/2/1/V_LIGHT:command:MAP(1on0off.map)]" }

Here is the info from the serial port monitor.