Getting MySensors MQTT Gateway working on OpenHAB 2.2 (Stable)

[TimO]

@jogant There already is a working version (2.4 SNAPSHOT) available on the marketplace and on my server. Please let me know if you expierence any problems.

[jogant]

@timo unfortunately this also does not work for me. I followed your manual very well but I always get errors. It starts after installing feature:install esh-io-transport-mqtt and mysensors binding:

16:59:57.772 [ERROR] [protocol.mqtt.MySensorsMqttConnection] - MQTT connection offline - Reason unknown
16:59:57.806 [INFO ] [o.transport.mqtt.MqttBrokerConnection] - Starting MQTT broker connection to '192.168.96.8' with clientid paho1135997459230242 and file store '/var/lib/openhab2/mqtt/192.168.96.8'
16:59:57.867 [ERROR] [protocol.mqtt.MySensorsMqttConnection] - MQTT connection offline - {}
org.eclipse.paho.client.mqttv3.MqttException: MqttException
        at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:38) [229:org.eclipse.paho.client.mqttv3:1.0.2]
        at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:604) [229:org.eclipse.paho.client.mqttv3:1.0.2]
        at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:210) ~[?:?]
        at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:?]
        at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) ~[?:?]
        at sun.security.ssl.InputRecord.read(InputRecord.java:503) ~[?:?]
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983) ~[?:?]
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385) ~[?:?]
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413) ~[?:?]
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397) ~[?:?]
        at org.eclipse.paho.client.mqttv3.internal.SSLNetworkModule.start(SSLNetworkModule.java:89) ~[?:?]
        at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:590) ~[?:?]
        ... 1 more

and reconection starts every 10 seconds.
It must be interference with mqtt1 bindung that I use for eventbus.
The mqtt-bridge remains Offline.
What is wrong?

[TimO]

@jogant Indeed this looks like an error in your configuration. I suspect the MQTT configuration. Did you change the MQTT configuration according my posting above? I've also added a text to the Wiki. The name and content of the MQTT configuration differs between OpenHAB 2.2 and 2.3.

[jogant]

@timo with 2.2 I had two configurations which were very similar.
I have one mosquitto mqtt broker on localhost, version 1.5.
The mqtt1 binding is for my mqtt-eventbus connection is called mosquitto and the config file (mqtt.cfg) is as follows:

mosquitto.url=tcp://192.168.96.8:1883
mosquitto.retain=false
mosquitto.clientId=openhab
mosquitto.user=user
mosquitto.pwd=pwd

With OH2.2 this configuration works. It also works with 2.3 as I can see all openhab events on my mosquitto.
My mysonsors binding is configured by org.eclipse.smarthome.mqttbroker.cfg config file which looks as follows:

name=mosquittomys
host=192.168.96.8
port=1883
retain=false
username=username
password=password
clientId=openhabMys

I also tried without authentication and also without clientId, all to no succuess.
I cleaned /var/lib/openhab2/cache/ folder and /var/lib/openhab2/tmp/ folder.
I looked at /var/lib/openhab2/config folder for resulting runtime configuration, but all looks fine. Do you have any further idea?

[jogant]

@timo I found the fault in my configuration: it was the missing secure=false config flag in the org.eclipse.smarthome.mqttbroker.cfg config file. I have no idea what happens when the flag is missing but without it one doesn't achieve a valid connection.

name=mosquittomys
host=192.168.96.8
port=1883
secure=false  <-- important !

So now I have a working 2.3 installation! Thanks alot for your support.

[TimO]

@jogant Good catch! I'm glad it's working now! :-)

[MasMat]

I'm still unsuccesful.
Do I need to reinstall all the files as well? I didn't find a new set of instructions.

In my /etc/openhab2/services-folder I have:
mqtt-eventbus.cfg containing just

broker=mosquitto

and org.eclipse.smarthome.mqttbroker.cfg:

name=mosquitto
host=127.0.0.1
secure=false
port=1883
secure=false
username=XXXX
password=YYYYY
retain=false
clientId=openhabMys

openhab.log everything mysensors related:

2018-07-09 06:40:36.782 [INFO ] [.transport.mqtt.MqttBrokerConnection] - Starting MQTT broker connection to '127.0.0.1' with clientid paho175068223000 and file store '/var/lib/openhab2/mqtt/127.0.0.1'
2018-07-09 06:40:37.050 [DEBUG] [org.openhab.binding.mysensors       ] - BundleEvent STARTING - org.openhab.binding.mysensors
2018-07-09 06:40:37.075 [DEBUG] [org.openhab.binding.mysensors       ] - BundleEvent STARTED - org.openhab.binding.mysensors
2018-07-09 06:40:37.206 [DEBUG] [org.openhab.binding.mysensors       ] - ServiceEvent REGISTERED - {org.eclipse.smarthome.core.thing.binding.ThingHandlerFactory}={service.id=119, service.bundleid=13, service.scope=bundle, component.name=org.openhab.binding.mysensors.factory.MySensorsHandlerFactory, component.id=17} - org.openhab.binding.mysensors
2018-07-09 06:42:36.466 [WARN ] [lipse.smarthome.io.net.exec.ExecUtil] - Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "arping" (in directory "."): error=2, Tiedostoa tai hakemistoa ei ole)
2018-07-09 06:42:36.685 [INFO ] [ternal.dhcp.DHCPPacketListenerServer] - DHCP request packet listener online
2018-07-09 06:42:37.484 [DEBUG] [nsors.handler.MySensorsBridgeHandler] - Initialization of the MySensors bridge
2018-07-09 06:42:38.136 [DEBUG] [ensors.factory.MySensorsCacheFactory] - Cache file: given_ids.cached exist.
2018-07-09 06:42:38.178 [DEBUG] [ensors.factory.MySensorsCacheFactory] - Cache (given_ids) content: [0, 1, 99, 4, 5]
2018-07-09 06:42:38.608 [DEBUG] [rotocol.mqtt.MySensorsMqttConnection] - Enabling connection watchdog
2018-07-09 06:42:38.830 [DEBUG] [org.openhab.binding.mysensors       ] - ServiceEvent REGISTERED - {org.eclipse.smarthome.config.discovery.DiscoveryService}={service.id=281, service.bundleid=13, service.scope=singleton} - org.openhab.binding.mysensors
2018-07-09 06:42:38.897 [DEBUG] [.discovery.MySensorsDiscoveryService] - Starting MySensors discovery scan
2018-07-09 06:42:38.926 [DEBUG] [nsors.handler.MySensorsBridgeHandler] - Initialization of the MySensors bridge DONE!

Anyone spot what's still wrong?

P.S. I remember seeing someone got MyS gateway to log to a different file. Does anyone know how to implement that in the ./configure -command? Or how can I do that?

[jogant]

@masmat two things come to my mind: I have two different broker-names for eventbus connection and mysensors connection. For eventbus binding you need classic mqtt1 binding with its own configuration like this:

mosquitto.url=tcp://127.0.0.1:1883
mosquitto.retain=false
mosquitto.clientId=openhab
mosquitto.user=user
mosquitto.pwd=pwd

And second: can you show your mysensors bridge configuration? Not shure but debug log looks like an ethernet bridge and not a mqtt bridge.

[MasMat]

Not sure how to answer some of your questions...but here goes.
configuration:

sudo ./configure --my-transport=nrf24 --my-rf24-irq-pin=15 --my-signing-debug --my-signing=password --my-signing-password=XXXXXXXX --my-gateway=mqtt --my-controller-ip-address=127.0.0.1 --my-mqtt-user=YYYY --my-mqtt-password=ZZZZZ --my-mqtt-publish-topic-prefix=mysensors-out --my-mqtt-subscribe-topic-prefix=mysensors-in --my-mqtt-client-id=mygateway1 --my-leds-err-pin=12 --my-leds-rx-pin=16 --my-leds-tx-pin=18

I'm not sure what "eventbus"-binding is. Something I need to install in Paper UI?
Is the configuration file mqtt-eventbus.cfg or something else? Should I try to install it? I have not installed the MQTT binding in OpenHab because at least originally the instructions said not to.

[jogant]

@masmat I just wanted to understand your configuration. So I read through whole thread and saw your problems began with upgrade to OH2.3.
I mentioned eventbus as you wrote something about eventbus.cfg.
You need eventbus only if you want to send all oh events to mqtt broker.
I wanted to see your mqtt bridge configuration. You can either configure dynamicly via paper ui or you set it up via things file. The later is what I did. So here is my things file with bridge configuration.

Bridge mysensors:bridge-mqtt:gateway [  brokerName="mosquittomys",
                                        topicPublish="mysensors1/in",
                                        topicSubscribe="mysensors1/out",
                                        startupCheckEnabled=true ] {

}

Here the brokername has to be the same as in mqttbroker.cfg. Fix topic names to match your mysensors gateway.
Hope this is of any help.

[shadowbreed]

This post is deleted!

[MasMat]

@jogant Finally had time to double check this:
Paper UI:

Broker Name 
mosquitto
Subscription Topic 
mygateway1-out
Publish Topic 
mygateway1-in
Send Delay 100
Startup check enabled
Metric answer enabled
Network sanity check enabled
Interval to launch network sanity check 3
Attempts before disconnecting the bridge 3
Heartbeat disabled
Attempts before disconnecting nodes 10

EDIT: I noticed I have different topics in MQTT, corrected AND I did:

cd /usr/share/openhab2/addons/
wget http://www.oberfoell.com/openhab2/org.openhab.binding.mysensors-2.4.0-SNAPSHOT.jar   <== THIS probably  fixed it!
chown for the same file

rm org.openhab.binding.mysensors-2.3.0-SNAPSHOT.jar

Now my MQTT Gw is ONLINE, in Paper UI I just selected the bridge for every individual Thing and now it works.
Hope this helps people strugling with this.

[Euromir]

Hi

I want to set this up, but have openHAB already setup and running MQTT using the binding installed in paperui, whereas you say dont use that?

Suggestions? With devices already working well on MQTT currently i want to follow this great, but dont wish to break what i have already working?

Thanks

[itbeyond]

@TimO Thanks for the excellent work with the binding.

I have just got this all running on OH2.3 using mosquito and mqtt mysensors using a Mega 2560. I am migration from a Vera to openHab and have around 20+ mysensors nodes from 1 to 8 devices on each. So this is a significant task. Anyway I am also newish to OH so have a few questions if I may.

• Is it better to go via MQTT or Ethernet Bridge - on Vera I am using Ethernet my new Gateway using the Mega (as pro mini not enough ram to run security) can be either so is there a preference for either of these?
• Is it worth using PaperUI vs Items files for configuration. I am leaning to items files as I am working with Persistance and some complex rules for device control. I have the binding learning nodes during a reboot in PaperUI these are working but I am confused how to build everything outside of the json DB files or even if I should?
• Can MYSController be used somehow in this configuration either MQTT or Ethernet for OTA. At the moment I run this on a PC and connect to the Ethernet gateway do my updates and then let Vera have the gateway back. Is there a better way to link this using openHab?

Sorry for the questions but if you never ask you never learn - thanks.
David

[fredswed]

@itbeyond As always your mileage will vary depending on experience and goals.
Disclaimer: I only have a couple of MYS-sensors and a small OpenHAB installation for sensing and remote light control. OpenHAB is a complex system so it takes time to master. Fortunately it also has great support and I really like it for its flexiblity, power and structure.

• MQTT or ethernet bridge? I've tried both but never got MQTT to work. Based on the questions in the MYS forums I believe it is easier to use the ethernet connection. I find it easier to troubleshoot. You can start with the ethernet bridge to ensure everything works. Then you can easily switch to the MQTT setup if there are advantages for you. But unless you have other applications that also listen to the MYS messages, I can't really see any obvious reasons to use MQTT. (Of course, it may give you a warm, fuzzy feeling of having an all-MQTT sensor network.)

• Definitely go for config files. Autodiscovery is neat but there are many things you can't do in PaperUI or that are very difficult to do there. In config files you can also add comments to explain to your future self why you did something.
  You can try autodiscovery and PaperUI first if you like and then move to config files. Then you may want to check out the OpenHAB forums on how to manually clear the cache. Sometimes the OH cache is a bit too clever for itself.

• As I understand it, OTA is not supported by any MQTT controller. It is also not supported by OpenHAB. If you use the ethernet bridge you can keep your current ways of working - sensing and control in OpenHAB and update OTA via MYSController. If I'm not mistaken you only have to configure the MYS gw to accept at least two connections.

You seem to know your way around OH already, but I will add a few general comments for beginners, based on my own experience with OpenHAB (as a non-developer):

• First step is to set up backup and restore. Then test that it really works the way you intended.
• Use the production release of OpenHAB unless you really need the bleeding edge. It can be very time consuming to revert from a beta/snapshot release to the production release.
• Spend some time setting up OpenHAB and software around it so you can easily edit the config files. I use Openhabian which automatically shares all interesting files using Samba. It is then trivial to connect from my desktop and edit with my favourite editor. (MS Code has great support for OpenHAB config files.)
• Spend some time on how to properly backup and restore OpenHAB so you can experiment freely and still easily revert if it crashes.
• The OH forum is your friend. It is even more true than for MYS.
  E.g. check out the advice on how to name (and not name) your config files, things and items.
  and the OH patterns that provides beatiful solutions for many "impossible" problems.
• Did I mention backup and restore? At least archive your config directory regularely.

/Fredrik

[itbeyond]

@fredswed thanks for your comments. I have gone with Ethernet gateway as it works well and at this stage I do not need MQTT - maybe if I play with node-red or something down the track but for the moment happy with Ethernet - it just works. I did have MQTT working but the MYSController connection is a possible problem.

I have also gone to the config files directly and have worked out the bridge setup in the things file and have a few test items running.

Yes backups - done that and have my system backing up to a local NAS box.
Cheers David

[MasMat]

I need a fresh set of eyes on my problem.
Gateway is online, but things are not discovered. I have tried adding them manually (my syntax could be wrong), but it doesn't help.

There are no errors in mysgw.log-file.
Mosquitto mysensors-out:

mysensors-out/1/14/1/0/23 35
mysensors-out/1/15/1/0/0 49
mysensors-out/1/14/1/0/23 35
mysensors-out/1/15/1/0/0 49
mysensors-out/1/14/1/0/23 35
mysensors-out/1/15/1/0/0 49
mysensors-out/1/14/1/0/23 35
mysensors-out/1/15/1/0/0 49

mysgw.things:

Bridge mysensors:bridge-mqtt:mygateway1 [brokerName="mosquitto",
                                      topicPublish="mysensors-in",
                                      topicSubscribe="mysensors-out",
                                      startupCheckEnabled=true ]
 {
    /** define things connected to that bridge here */

  temperature BoilerTemp [nodeID=1, childId=15]
  motion PukuhLiike [nodeID=1, childId=3]

  }

default.items:

Number BoilerTemp "Lämminvesivaraaja [%.1f °C]" <heating> {channel="mysensors:temperature:mygateway1:temperature_1_15:temp"}
Contact PukuhLiike "Pukuhuone Liike" <motion> {channel="mysensors:motion:mygateway1:motion_1_3:tripped"}

OpenHab2 frontail:

2018-08-13 22:34:03.284 [INFO ] [.transport.mqtt.MqttBrokerConnection] - Starting MQTT broker connection to '127.0.0.1' with clientid paho111686078117 and file store '/var/lib/openhab2/mqtt/127.0.0.1'

2018-08-13 22:34:05.499 [INFO ] [er.internal.HomeBuilderDashboardTile] - Started Home Builder at /homebuilder

2018-08-13 22:34:11.928 [INFO ] [.core.internal.i18n.I18nProviderImpl] - Time zone set to 'Europe/Helsinki'.

2018-08-13 22:34:11.999 [INFO ] [.core.internal.i18n.I18nProviderImpl] - Locale set to 'fi'.

2018-08-13 22:34:20.211 [INFO ] [.dashboard.internal.DashboardService] - Started Dashboard at http://192.168.1.51:8080

2018-08-13 22:34:20.317 [INFO ] [.dashboard.internal.DashboardService] - Started Dashboard at https://192.168.1.51:8443

2018-08-13 22:35:16.502 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'default.items'

2018-08-13 22:35:34.984 [INFO ] [thome.model.lsp.internal.ModelServer] - Started Language Server Protocol (LSP) service on port 5007

2018-08-13 22:35:38.657 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'mys.things'

2018-08-13 22:35:39.245 [.ItemChannelLinkAddedEvent] - Link 'PukuhLiike-mysensors:motion:mygateway1:motion_1_3:tripped' has been added.

2018-08-13 22:35:39.309 [.ItemChannelLinkAddedEvent] - Link 'BoilerTemp-mysensors:temperature:mygateway1:temperature_1_15:temp' has been added.

2018-08-13 22:36:14.848 [INFO ] [basic.internal.servlet.WebAppServlet] - Started Basic UI at /basicui/app

2018-08-13 22:36:18.741 [INFO ] [arthome.ui.paper.internal.PaperUIApp] - Started Paper UI at /paperui

2018-08-13 22:36:22.407 [INFO ] [panel.internal.HABPanelDashboardTile] - Started HABPanel at /habpanel

2018-08-13 22:36:25.610 [hingStatusInfoChangedEvent] - 'mysensors:bridge-mqtt:mygateway1' changed from UNINITIALIZED to INITIALIZING

2018-08-13 22:36:26.660 [hingStatusInfoChangedEvent] - 'mysensors:temperature:mygateway1:BoilerTemp' changed from UNINITIALIZED to UNINITIALIZED (BRIDGE_UNINITIALIZED)

2018-08-13 22:36:26.861 [hingStatusInfoChangedEvent] - 'mysensors:motion:mygateway1:PukuhLiike' changed from UNINITIALIZED to UNINITIALIZED (BRIDGE_UNINITIALIZED)

2018-08-13 22:36:27.453 [hingStatusInfoChangedEvent] - 'mysensors:bridge-mqtt:mygateway1' changed from INITIALIZING to ONLINE


2018-08-13 22:36:27.546 [INFO ] [rotocol.mqtt.MySensorsMqttConnection] - Successfully connected to MySensors Bridge.


2018-08-13 22:36:28.075 [hingStatusInfoChangedEvent] - 'mysensors:temperature:mygateway1:BoilerTemp' changed from UNINITIALIZED (BRIDGE_UNINITIALIZED) to UNINITIALIZED (HANDLER_CONFIGURATION_PENDING)

2018-08-13 22:36:28.120 [hingStatusInfoChangedEvent] - 'mysensors:motion:mygateway1:PukuhLiike' changed from UNINITIALIZED (BRIDGE_UNINITIALIZED) to UNINITIALIZED (HANDLER_CONFIGURATION_PENDING)

In my addons I have the 2.4.0 Snapshot.
org.eclipse.smarthome.mqttbroker.cfg:

name=mosquitto
host=127.0.0.1
secure=false
port=1883
username=XXXX
password=YYYYY
retain=false
clientId=openhabian

EDIT: After a week of trying things. This worked: https://forum.mysensors.org/post/73600
No autodiscovery, but this switches my sensors ONLINE (even ones that are not powered up :( ), but it's not updating the values.

[TimO]

@masmat said in Getting MySensors MQTT Gateway working on OpenHAB 2.2 (Stable):

No autodiscovery, but this switches my sensors ONLINE (even ones that are not powered up ), but it's not updating the values.

Autodiscovery should work. :confused:
Does the temperature thing in OH2 get updates? Is it working?

The sensors will show up ONLINE if the connection to the broker is established, because the sensors are just a topic. There is at the moment no implementation to check if the sensors/things are really available. This could be done, but it is not implemented yet.

[MasMat]

@timo Before I did the re-install of everything autodiscovery was working, but now it isn't. Current sys: RpiZeroW, Openhabian 2.3.0-1release, mosquitto.
The temperature is not updating in OH2. But the mysensors-out topic does show the gateway sending the values out. OH2 is not listening for some reason...

Re-install some (which?) parts or bindings? Openhab2 mqtt-binding?
Permissions?

[TimO]

@masmat said in Getting MySensors MQTT Gateway working on OpenHAB 2.2 (Stable):

Number BoilerTemp "Lämminvesivaraaja [%.1f °C]" <heating> {channel="mysensors:temperature:mygateway1:temperature_1_15:temp"}

Needs to be changed to:

Number BoilerTemp "Lämminvesivaraaja [%.1f °C]" <heating> {channel="mysensors:temperature:mygateway1:BoilerTemp:temp"}

But there is something additionally amiss, because discovery is not working.
Please enable debug in the OH2 console and post the output of the initialization.
Don't spend time reinstalling, this looks like a configuration issue.