OH3 - MySensors Binding

[TimO]

@kerberos Interesting find. Haven't seen something like this till now. I suggest to delete the jar file, delete the cache and start over again.

I haven't tested textual configuration, but this should work just fine and your problem occurred before the textual configuration is initialized.

[Andreas Kräuchi]

hmm maybe I go back to the binding as well. Just spent several hours migrating all to the mqtt binding. It's working now but much slower.... So when I press a button the mqtt is slower to react as with the mysensor binding is. Why so?

[kerberos]

@TimO

many thanks for quick answer. I already remove the jar file and delete the cache, tmp and mysensors folder in the userdata folder several times. The textual configuration is used for many years, with some small updates. I don't try the gui configuration.

If you need more data's for debugging, I will provide.

[gjongerh]

Great work TimO your thing is working for me. Didn't do that much with it yet, so will keep testing.

[Dustin Hagstrom]

@TimO What a fantastic build, thank you so much!
My build:
OH2 -> 3.0.0 upgraded via raspberry pi update (in OH3 instructions)
Battery Nodes, Color Lights, Buttons, Moisture sensors
My Sensors Eth Gateway Serial bridge

Side note for any newbies like me out there. Follow the instructions for OH2 installation for my sensors. Follow the directions for manually adding in the addon. And using TimO's jar instead.

I've been monitoring all my logs and no issues reported, everything seems to work just like in OH2.

[syntacrsc]

@TimO
Thanks a lot for the updated version of the openHAB Binding (jar file) for openHAB 3.
It seems to be working well on OH 3- just perfomed some initial testing on my OH3 (3.0.0) test system.

Also I feel it might be a good idea adding a link to the new 3.1.0 jar file to the "official" Install Instruction on https://www.mysensors.org/controller/openhab - don't you agree?

[ncollins]

@TimO Great work.

I've had a merge request https://github.com/tobof/openhab2-addons/pull/141 floating around for months to add a wind direction channel to the my sensors binding.

Is your code in a shared repo? I'd love help add in my enhancement into your OH3 mysensors build.

[ncollins]

Well, after a rather painful experience of manually upgrading and cherrypicking SerialPort changes from the decompiled OH3 mysensors jar posted above, I managed to get it a build working of the MySensors binding 3.1-SNAPSHOT.

This is the source: https://github.com/nikolac/openhab2-addons/tree/mysensors-oh3-migration/bundles/org.openhab.binding.mysensors

This fork has my addition, a direction channel on wind variables.

[fabyte]

@ncollins Any chance this rebased work will be pushed upstream to the openhab/openhab-addons repo? Mainline support could help this binding with future framework changes and avoid reverse engineering of code changes and simplifies the binding installation.

Also any ideas on the MQTT bridge problem? Somehow the broker name cannot be resolved as also discussed here:
https://forum.mysensors.org/topic/10184/help-openhab-2-4-binding-mqtt

[ncollins]

@fabyte I'd love to see this get pulled into the main repo. Not sure if @TimO plans to continue support, but I have no problem helping or even picking up where he leaves off.

I didn't contribute any MQTT specific fixes, and I don't recall manually migrating any MQTT fixes, but let me take a look.

[fabyte]

@ncollins Thanks, really appreciate it :) Let me know if you need more info, here's the basic config I did:

• created a mqtt broker (not system broker) This thing is ONLINE

UID: mqtt:broker:mosquitto
label: mosquitto
thingTypeUID: mqtt:broker
configuration:
  lwtQos: 0
  publickeypin: true
  keepAlive: 60
  clientid: 566e1af2-19c2-4596-904f-18b89c502e53
  qos: 0
  reconnectTime: 60000
  host: localhost
  secure: false
  certificatepin: true
  lwtRetain: true
  enableDiscovery: true

• created a MySensors MQTT Bridge with default settings (brokername = mosquitto)

UID: mysensors:bridge-mqtt:01d212b118
label: MySensors MQTT Gateway
thingTypeUID: mysensors:bridge-mqtt
configuration:
  networkSanCheckSendHeartbeat: false
  topicPublish: mygateway1-in
  networkSanCheckInterval: 3
  networkSanCheckSendHeartbeatFailAttempts: 10
  topicSubscribe: mygateway1-out
  networkSanCheckEnabled: false
  networkSanCheckConnectionFailAttempts: 3
  startupCheckEnabled: false
  brokerName: mosquitto
  imperial: false
  sendDelay: 100

Here's the logs:

09:59:32.973 [INFO ] [hab.event.ThingStatusInfoChangedEvent] - Thing 'mysensors:bridge-mqtt:01d212b118' changed from UNINITIALIZED to INITIALIZING
09:59:32.980 [DEBUG] [ensors.handler.MySensorsBridgeHandler] - Initialization of the MySensors bridge
09:59:32.994 [DEBUG] [sensors.factory.MySensorsCacheFactory] - Cache file: given_ids.cached not exist.
09:59:32.995 [DEBUG] [sensors.factory.MySensorsCacheFactory] - Cache file: given_ids.cached not exist. Default passed, creating it...
09:59:32.997 [DEBUG] [sensors.factory.MySensorsCacheFactory] - Writing on cache given_ids, content: []
09:59:32.998 [DEBUG] [sensors.factory.MySensorsCacheFactory] - Cache (given_ids) content: []
09:59:33.011 [DEBUG] [protocol.mqtt.MySensorsMqttConnection] - Enabling connection watchdog
09:59:33.015 [DEBUG] [ensors.handler.MySensorsBridgeHandler] - Initialization of the MySensors bridge DONE!
09:59:33.016 [DEBUG] [s.discovery.MySensorsDiscoveryService] - Starting MySensors discovery scan
09:59:33.022 [ERROR] [protocol.mqtt.MySensorsMqttConnection] - No connection to broker: mosquitto
09:59:33.027 [INFO ] [hab.event.ThingStatusInfoChangedEvent] - Thing 'mysensors:bridge-mqtt:01d212b118' changed from INITIALIZING to OFFLINE
09:59:33.027 [DEBUG] [sensors.factory.MySensorsCacheFactory] - Writing on cache given_ids, content: []
09:59:33.029 [ERROR] [protocol.mqtt.MySensorsMqttConnection] - Failed connecting to bridge...next retry in 10 seconds (Retry No.:0)

I tried different broker names (like mqtt:broker:mosquitto, the clientid, different uid names, but the log always says the broker cannot be found. It maps to the following lines: link

I can provide more info if needed and try out debug versions of the binding if required.

[fabyte]

Okay, now i tried to configure the MQTT system broker connection and this seems to work.

• I set up the system broker and named it 'mosquitto'.
• in the MySensors MQTT bridge, as before, I used mosquitto as Broker Name. Now the Broker Name could be resolved and the Thing is online.

10:12:55.086 [DEBUG] [ensors.handler.MySensorsBridgeHandler] - Initialization of the MySensors bridge
10:12:55.089 [DEBUG] [sensors.factory.MySensorsCacheFactory] - Cache file: given_ids.cached not exist.
10:12:55.091 [DEBUG] [sensors.factory.MySensorsCacheFactory] - Cache file: given_ids.cached not exist. Default passed, creating it...
10:12:55.094 [DEBUG] [sensors.factory.MySensorsCacheFactory] - Writing on cache given_ids, content: []
10:12:55.100 [DEBUG] [sensors.factory.MySensorsCacheFactory] - Cache (given_ids) content: []
10:12:55.103 [DEBUG] [protocol.mqtt.MySensorsMqttConnection] - Enabling connection watchdog
10:12:55.106 [DEBUG] [ensors.handler.MySensorsBridgeHandler] - Initialization of the MySensors bridge DONE!
10:12:55.108 [ERROR] [protocol.mqtt.MySensorsMqttConnection] - MQTT connection offline - Reason unknown
10:12:55.113 [DEBUG] [s.discovery.MySensorsDiscoveryService] - Starting MySensors discovery scan
10:12:55.117 [DEBUG] [protocol.mqtt.MySensorsMqttConnection] - Adding consumer for topic: mygateway1-out/+/+/+/+/+
10:12:55.127 [WARN ] [protocol.mqtt.MySensorsMqttConnection] - Skipping I_VERSION connection test, not recommended...
10:12:55.133 [INFO ] [hab.event.ThingStatusInfoChangedEvent] - Thing 'mysensors:bridge-mqtt:84dd4a387f' changed from INITIALIZING to ONLINE
10:12:55.133 [DEBUG] [sensors.factory.MySensorsCacheFactory] - Writing on cache given_ids, content: []
10:12:55.139 [INFO ] [protocol.mqtt.MySensorsMqttConnection] - Successfully connected to MySensors Bridge.

But
I'm having troubles with the MQTT system broker, it can't connect the broker instance.

com.hivemq.client.mqtt.exceptions.ConnectionClosedException: java.io.IOException: Connection reset by peer
Caused by: java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[?:?]
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[?:?]
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276) ~[?:?]
        at sun.nio.ch.IOUtil.read(IOUtil.java:233) ~[?:?]
        at sun.nio.ch.IOUtil.read(IOUtil.java:223) ~[?:?]
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:358) ~[?:?]
        at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:247) ~[bundleFile:4.1.42.Final]
        at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1147) ~[bundleFile:4.1.42.Final]
        at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:347) ~[bundleFile:4.1.42.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148) [bundleFile:4.1.42.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700) [bundleFile:4.1.42.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635) [bundleFile:4.1.42.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552) [bundleFile:4.1.42.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514) [bundleFile:4.1.42.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044) [bundleFile:4.1.42.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [bundleFile:4.1.42.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [bundleFile:4.1.42.Final]
        at java.lang.Thread.run(Thread.java:834) [?:?]

Since this seems more a OpenHAB/mosquitto issue I will dig into it myself.

Edit
The MQTT System Broker seems to be a concept from OH2, where a MQTT broker was shipped internally with OpenHAB and it seems to be broken in OH3 (OH Forum).

This leaves me with the following conclusions:

1. The MySensors binding configuration references the MQTT system broker connection
2. With OH3, this MQTT System Broker is not working anymore.

I suppose the correct way of handling this issue is to make the MySensors binding reference a normal MQTT Broker bridge, not a System Broker bridge.
Anyone who knows what changes this would require? My guess is that it should be manageable -> Source Code

[ncollins]

@fabyte this is helpful.

My intuition says the embedded mqtt broker and the generic mqtt binding create the same "mqtt" connection service behind the scenes. I'm guessing the generic mqtt binding isn't registering its service by name, or it's registering it by some other key.

I'd like to get the mysensors addon to work with both mqtt brokers if possible.

[fabyte]

@ncollins Yes, it looks like some resources are blocked or not resolved correctly when using the system broker.
But the system broker functionality comes with the mqtt binding itself. Here is a screen of the settings when I deleted the mqtt binding:

Also I never got the system broker working, even without defining a generic mqtt broker thing.

[fabyte]

After reinstalling the mqtt bindung the MQTT System Broker connection now could be established - strange, that a reinstall was required since I did several OpenHAB system restarts and those didn't fix it.

Well, now the MySensors binding is working with the system broker:

Here's a MySensors temperature item:

To sum up, here's what I did

1. (manually) installed MySensors binding
2. uninstalled mqtt binding
3. installed mqtt binding
4. set up the MQTT system broker connection
   4.1 set Connection Name to systembroker
   4.2 set Broker Host to localhost (it's running on the same device)
   4.3 Show advanced -> set Broker Port to 1883 since it was always using 8883 but I don't use a secure connection
   (4.4 I didn't set a clientid, no user and no password)
5. set up the MySensors MQTT Gateway thing
   5.1 set Broker Name to systembroker

[ncollins]

@fabyte Nice work. I don't use mysensors-mqtt, but I do have a lot of tasmota devices using the generic mqtt binding.

I followed your directions to create system broker, then changed my existing mqtt broker connection to a "systemBroker" and referenced it by brokerid (Broker Name).

I had one other issue, the new system broker wouldn't start up. I had to manually add secure="false" to /opt/openhab/userdata/config/org/openhab/mqttbroker.config .

Maybe you can get away with toggling "Secure Connection?" a couple of times.

[TimO]

Hey @fabyte and @ncollins!

I just want you to know, that I'm still there, but out of time at the moment. My third child was born just recently and I don't have any time for the binding at the moment.
I'm really sorry you had to reverse engineer the binding to add your code. That's awfully wrong.
We need to create a new common base to continue the work on the binding.
I was not able to add the binding to the official repo because of the formal requirements from the Openhab Team. I'm sure these requirements are needed, but I'm no professional in java. I struggled mainly at the nonnull annotations that are needed.

[fabyte]

@TimO Congratulations on your child :clap:
I know it's an exciting time and definitely more important than developing a OH binding :grin:
Spending time with your family should always be number 1 priority and you can't blame anyone for taking time off from a internet community :)

[ncollins]

@TimO Congrats!

I've been looking some of the other addon-ons and your code actually looks a lot better than most. I think the base is pretty solid.

I'll take a look at the annotations, and maybe getting some more traditional unit tests in place.

If we get to a point where a merge is acceptable, maybe we can move the maintenance fork to the MySensors repo.

[ncollins]

@TimO & @fabyte I took a stab at making all of the classes/interfaces @NonNullByDefault and adding/overriding null checks throughout. It builds fine, seems to be working well with my ethernet gateway.

I'm going to look at unit testing next.

https://github.com/nikolac/openhab2-addons/tree/mysensors-oh3-migration/bundles/org.openhab.binding.mysensors