Getting MySensors MQTT Gateway working on OpenHAB 2.2 (Stable)



  • Here are step by step instructions for taking a vanilla OpenHabian 2.2 and adding MQTT Gateway and MySensors to it. This may change, but at the date of this post and with the versions / snapshots below, this currently works:

    Pre-requisites / configuation:

    GatewayW1500MQTTClient I have setup as follows:


    #define MY_RADIO_NRF24
    #define MY_RF24_PA_LEVEL RF24_PA_MIN // ONLY FOR nrf24L01+ Enhanced, it reduces the output of the nrf24L01 otherwise rfaxis power amp overloads and starts corrupting messages
    #define MY_RF24_CHANNEL (109) // 0-126: Note specifying a channel >= 108 clears most of the wifi interference.

    // Security (Signing)
    #define MY_SIGNING_SOFT
    //For the software backed signing backend, an unconnected analog pin is required on boards that does not provide a hardware based random generator unit to set a
    //random seed for the pseudo-random generator. It is important that the pin is floating, or the output of the pseudo-random generator will be predictable, and thus compromise the signatures.
    #define MY_SIGNING_SOFT_RANDOMSEED_PIN A12
    #define MY_SIGNING_REQUEST_SIGNATURES

    #define MY_DEFAULT_ERR_LED_PIN 7 // Error led pin
    #define MY_DEFAULT_RX_LED_PIN 8 // Receive led pin
    #define MY_DEFAULT_TX_LED_PIN 9 // the PCB, on board L

    #define MY_GATEWAY_MQTT_CLIENT

    // W5100 Ethernet module SPI enable (optional if using a shield/module that manages SPI_EN signal)
    //#define MY_W5100_SPI_EN 4

    // Enable Soft SPI for NRF radio (note different radio wiring is required)
    // The W5100 ethernet module seems to have a hard time co-operate with
    // radio on the same spi bus.
    #if !defined(MY_W5100_SPI_EN) && !defined(ARDUINO_ARCH_SAMD)
    #define MY_SOFTSPI
    #define MY_SOFT_SPI_SCK_PIN A0
    #define MY_SOFT_SPI_MISO_PIN A2
    #define MY_SOFT_SPI_MOSI_PIN A1
    #endif

    // When W5100 is connected we have to move CE/CSN pins for NRF radio
    #ifndef MY_RF24_CE_PIN
    #define MY_RF24_CE_PIN 5
    #endif
    #ifndef MY_RF24_CS_PIN
    #define MY_RF24_CS_PIN 6
    #endif

    #define MY_MAC_ADDRESS 0xXX, 0xXX, 0xXX, 0xXX, 0xXX, 0xXX <--- CHANGE TO YOUR MAC ADDRESS

    // Static ip setup, otherwise if not defined, dhcp is used
    //#define MY_IP_ADDRESS 192,168,178,87
    //#define MY_IP_SUBNET_ADDRESS 255,255,255,0
    //#define MY_IP_GATEWAY_ADDRESS 192,168,178,1

    // MQTT broker ip address or url. Define one or the other.
    //#define MY_CONTROLLER_URL_ADDRESS "m20.cloudmqtt.com"
    #define MY_CONTROLLER_IP_ADDRESS 192, 168, XXX, XXX <--- SET THIS TO THE ADDRESS OF YOUR RASPBERRY PI

    // The MQTT broker port to to open
    #define MY_PORT 1883

    // Enable these if your MQTT broker requires usenrame/password
    #define MY_MQTT_USER "openhabian" <--- MUST MATCH THE USERNAME FOR MOSQUITTO ON THE PI
    #define MY_MQTT_PASSWORD "XXXXX" <--- SET THIS TO YOUR A PASSWORD - MUST MATCH THE ONE FOR MOSQUITTO ON THE PI"

    // Set MQTT client id
    #define MY_MQTT_CLIENT_ID "mysensors-1"

    // Set this node's subscribe and publish topic prefix
    #define MY_MQTT_PUBLISH_TOPIC_PREFIX "mygateway1-out"
    #define MY_MQTT_SUBSCRIBE_TOPIC_PREFIX "mygateway1-in"


    Steps:

    1. Boot Raspberry pi from SD Card connected to ethernet (if you have any wifi connectivity problems, parts of the installation fail, and you get an incomplete install, that doesn't necessarily fail to complete, hence use ethernet till built)

    2. After about 2-3 mins, connect to the Pi with: ssh openhabian@IP_ADDRESS (password: openhabian) and you will see the install in progress

    3. Watch until installation has finished (approx 25mins depending on connection speed), the last entry will be "First time setup successfully finished". There should be no errors (except: sed: -e expression #1, char 5: unterminated address regex), if you're getting other errors, retry it (you need to make up the SD Card with the image again), if errors are randomly happening, suspect the SD Card.

    4. Ctrl-C and type sudo reboot

    5. Verify Installation: After about 5 mins you should be able to connect to: http://IP_ADDRESS:8080, select Standard Setup and wait until PaperUI and HABPanel have appeared
      Also: http://IP_ADDRESS:9001 should show no errors

    6. ssh openhabian@IP_ADDRESS
      sudo openhabian-config
      Change openhab-cli password:

      System Settings / Change Passwords / openHAB Console: and set to something, we'll need this when logging into openhab-cli, we'll call this password_openhab-cli

      Add Mosquitto:

      Optional Components / Mosquitto, set password to something (we'll call this password_mosquitto), needs to match the one in your MySensors gateway and the one in the next step.

      Exit

    7. Configuring MQTT:
      IMPORTANT: DO NOT INSTALL THE MQTT BINDING IN THE PAPER UI, IT CONFLICTS WITH MYSENSORS

    cd /srv/openhab2-conf/services
    nano org.eclipse.smarthome.mqtt.cfg   (or use your favorite editor)
    

    add:

    mosquitto.url=tcp://localhost:1883	
    mosquitto.user=openhabian
    mosquitto.pwd=YOUR_PASSWORD_GOES_HERE   (This is password_mosquitto referenced previously)
    mosquitto.qos=1
    mosquitto.retain=true
    mosquitto.async=false
    (NOTE: We don't add clientId here, it causes problems and prevents both OpenHab and MySensors connecting due to multiple id's (persistance in use errors).  Allowing the id's to be auto and uniquely assigned, will prevent this clash)
    

    save the file and exit

    nano mqtt-eventbus.cfg
    

    Add:

    broker=mosquitto
    

    save the file and exit

    sudo chown openhab:openhabian org.eclipse.smarthome.mqtt.cfg mqtt-eventbus.cfg

    Remove the backup files:

    cd /var/lib/openhab2/config/org/openhab
    rm mqtt-eventbus.config
    cd ../eclipse/smarthome
    rm mqtt.config
    
    sudo reboot
    

    Power cycle or switch on your already configured MySensors MQTT Gateway

    1. ssh openhabian@IP_ADDRESS
      After a few minutes, visit: http://IP_ADDRESS:9001
      You should have no errors.

    2. Installing MySensors:
      openhab-cli console
      (if password prompt doesn’t appear after a while, ctrl-c and try again, it can take a while or timeout the first few times after installation, this is password_openhab-cli)

    feature:install openhab-transport-serial
    feature:install esh-io-transport-mqtt
    CTRL-D
    cd /srv/openhab2-addons
    wget http://www.oberfoell.com/openhab2/org.openhab.binding.mysensors-2.3.0-SNAPSHOT.jar
    sudo chown openhab:openhabian org.openhab.binding.mysensors-2.3.0-SNAPSHOT.jar
    openhab-cli console
    
    Check the features and services we need are active:
    
    feature:list | grep openhab-transport-serial	(Should say Started)
    feature:list | grep esh-io-transport-mqtt	(Should say Started)
    bundle:list | grep MySensors			(Should say Active)
    bundle:list | grep nrjavaserial			(Should say Active)
    
    CTRL-D
    sudo reboot
    
    1. ssh openhabian@IP_ADDRESS
      After a few minutes, visit: http://IP_ADDRESS:9001
      You should have no errors.

    Check mosquitto is running and gateway is connected:

    cd /var/log/mosquitto
    tail -f mosquitto.log
       526088756: mosquitto version 1.4.10 (build date Fri, 22 Dec 2017 08:19:25 +0000) starting
       1526088756: Config loaded from /etc/mosquitto/mosquitto.conf.
       1526088756: Opening ipv4 listen socket on port 1883.
       1526088756: Opening ipv6 listen socket on port 1883.
       1526088764: New connection from 192.168.20.143 on port 1883.
       1526088764: New client connected from 192.168.20.143 as mysensors-1 (c1, k15, u'openhabian'). <-- This is the MySensors MQTT Gateway connecting)
       1526088783: New connection from 127.0.0.1 on port 1883.						<-- This is the MQTT Service on the Raspberry Pi Connecting
       (If you're not seeing the above, power cycle the MySensors MQTT Gateway)
    CTRL-C
    
    1. Adding MySensors Binding in PaperUI:
      Visit: http://IP_ADDRESS:8080 and click on PaperUI
    Choose Inbox / + / MySensors Binding / MySensors MQTT Gateway
       Enable: "Enabled the startup check"
       Enable: "Enabled network sanity check"
       (all other settings remain at the default, but you might want to change Imperial/Metric for your preference)
       Click the "Blue" Checkmark at the top to save.
    
    Choose Configuration / Things (you may have to reload the page)
       MySensors MQTT Gateway should be ONLINE, if not, check your Gateway is configured correctly.
    
    sudo reboot
    
    1. After a few minutes, visit: http://IP_ADDRESS:9001
      You should have no errors.
      Visit: http://IP_ADDRESS:8080 and click on PaperUI
      Visit Configuration / Things, MySensors MQTT Gateway should still be ONLINE

      ssh openhabian@IP_ADDRESS

      Check mosquitto is running and gateway is connected:
      cd /var/log/mosquitto
      tail mosquitto.log

      It should look like this:

    1526089285: mosquitto version 1.4.10 (build date Fri, 22 Dec 2017 08:19:25 +0000) starting
    1526089285: Config loaded from /etc/mosquitto/mosquitto.conf.
    1526089285: Opening ipv4 listen socket on port 1883.
    1526089285: Opening ipv6 listen socket on port 1883.
    1526089294: New connection from 192.168.20.143 on port 1883.
    1526089294: New client connected from 192.168.20.143 as mysensors-1 (c1, k15, u'openhabian').
    1526089312: New connection from 127.0.0.1 on port 1883.
    

    CONGRATULATIONS, gateway and mysensors are now setup

    1. Now you want to add some mysensors things into PaperUI. Easiest way is to power cycle the sensor, this should result in the sensor appear in the Inbox in PaperUI. For example, I built an RGB Light, when I power cycle it, I see in the Inbox:
      MySensors Device (1;1) RGB Light Sensor: mysensors:rgbLight:a6e1bcda:rgblight_1_1

      Click on the Blue checkmark, and click "Add As Thing"
      Go to Configuration / Things, and you should see the MySensors Device(1;1) show up
      Click the bolded title of the sensor (MySensors Device (1;1) and check the blue circles you want to see the data for one by one. Each one pops up a "Link Channel" message, change item to "Create new item" and select "Link"

      That newly added item will now show up in Configuration / Items and also in "Control" so you can change it's status

    2. If you're new to this, checkout HABPanel, use PaperUI to setup items, but HABPanel to present a UI for use.



  • Great instructions, thanks!
    My problems started yesterday with my OpenHab2 updating to 2.3.0.

    I'm using Rpi Zero W / DietPi - conf to MQTT gw. Mosquitto and mysgw service work OK. Everything else doesn't.
    My problems briefly from the logs:

    1527624943: New client connected from 127.0.0.1 as mygateway1 (c1, k15, u'masi').
    1527626485: New connection from 127.0.0.1 on port 1883.
    1527626485: New client connected from 127.0.0.1 as mosqsub/3760-DietPi (c1, k60, u'masi').
    1527626499: Socket error on client mosqsub/3760-DietPi, disconnecting.
    

    OpenHab (edited):

    2018-05-29 23:17:17.655 [DEBUG] [org.openhab.binding.mysensors       ] - BundleEvent STARTING - org.openhab.binding.mysensors
    2018-05-29 23:17:17.880 [DEBUG] [org.openhab.binding.mysensors       ] - BundleEvent STARTED - org.openhab.binding.mysensors
    2018-05-29 23:17:18.019 [DEBUG] [org.openhab.binding.mysensors       ] - ServiceEvent REGISTERED - {org.eclipse.smarthome.core.thing.binding.ThingHandlerFactory}={service.id=116, service.bundleid=13, service.scope=bundle, component.name=org.openhab.binding.mysensors.factory.MySensorsHandlerFactory, component.id=15} - org.openhab.binding.mysensors
    2018-05-29 23:19:24.908 [WARN ] [lipse.smarthome.io.net.exec.ExecUtil] - Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "arping" (in directory "."): error=2, Tiedostoa tai hakemistoa ei ole)
    2018-05-29 23:19:25.184 [INFO ] [ternal.dhcp.DHCPPacketListenerServer] - DHCP request packet listener online
    2018-05-29 23:19:25.985 [DEBUG] [nsors.handler.MySensorsBridgeHandler] - Initialization of the MySensors bridge
    2018-05-29 23:19:26.257 [DEBUG] [ensors.factory.MySensorsCacheFactory] - Cache file: given_ids.cached exist.
    2018-05-29 23:19:26.315 [DEBUG] [ensors.factory.MySensorsCacheFactory] - Cache (given_ids) content: [0, 1, 99, 4, 5]
    2018-05-29 23:19:26.594 [DEBUG] [rotocol.mqtt.MySensorsMqttConnection] - Enabling connection watchdog
    2018-05-29 23:19:26.795 [ERROR] [rotocol.mqtt.MySensorsMqttConnection] - No connection to broker: mosquitto
    2018-05-29 23:19:26.834 [DEBUG] [org.openhab.binding.mysensors       ] - ServiceEvent REGISTERED - {org.eclipse.smarthome.config.discovery.DiscoveryService}={service.id=280, service.bundleid=13, service.scope=singleton} - org.openhab.binding.mysensors
    2018-05-29 23:19:26.925 [DEBUG] [.discovery.MySensorsDiscoveryService] - Starting MySensors discovery scan
    2018-05-29 23:19:27.032 [INFO ] [ateway.MySensorsNetworkSanityChecker] - Network Sanity Checker thread stopped
    2018-05-29 23:19:27.191 [DEBUG] [ensors.factory.MySensorsCacheFactory] - Writing on cache given_ids, content: [0,1,99,4,5]
    2018-05-29 23:19:27.242 [DEBUG] [nsors.handler.MySensorsBridgeHandler] - Initialization of the MySensors bridge DONE!
    2018-05-29 23:19:27.250 [ERROR] [rotocol.mqtt.MySensorsMqttConnection] - Failed connecting to bridge...next retry in 10 seconds (Retry No.:0)
    

    I've gone through your steps, Mysensors MQTT gateway remain offline.
    What I'm wondering (since the directory structure is different), what is the total content of file: mqtt-eventbus.cfg? Can you post?

    Ideas?



  • Hey,

    same Problem here. After upgrading to OpenHAB 2.3 i receive the same errors in my /var/log/openhab2/openhab.log

    2018-05-31 16:28:42.776 [ERROR] [rotocol.mqtt.MySensorsMqttConnection] - No connection to broker: mosquitto
    2018-05-31 16:28:42.918 [ERROR] [rotocol.mqtt.MySensorsMqttConnection] - Failed connecting to bridge...next retry in 10 seconds (Retry No.:0)
    

  • Hero Member

    @andre_m87 & @MasMat : I have the same problem here. Looks like a change in the OpenHAB MQTT implementation.
    I'm working on it.


  • Hero Member

    @andre_m87 & @MasMat : I found a solution and indeed the MQTT implementation in OH2 (ESH) changed. This was a hard nut to crack, so it took me a while.

    Here are the steps you need to do:

    1. Rename: org.eclipse.smarthome.mqtt.cfg to org.eclipse.smarthome.mqttbroker.cfg
    2. Change the content to:
    name=mosquitto
    host=192.168.2.3
    secure=false
    port=1883
    username=USERNAME
    password=PASSWORD
    retain=false
    

    If you're using a secure connection (SSL): change secure to true and use the correct port.

    Another intersting feature in OpenHAB 2.3 is the embedded broker, so there is no need to have a separate MQTT server up and running.



  • @timo unfortunately this solution doesn't work for me.
    After update oh to version 2.3 I don't get mqtt to work. Prior I used mqtt for mysensors and for eventbus. I configured two broker connections and set up a mysensors-mqtt-bridge via things-file. In the broker all events from openhab arrived and all mysensors-messages arrived in openhab.
    After update mysensors doesn't work at all, allthough I followed your instructions. If I setup mqtt1 binding I can see eventbus messages in mqtt broker but my mysensors-mqtt-gateway remains initializing.

    What I'm doing wrong?


  • Hero Member

    @jogant Yeah. 😞 There are still some problems with 2.3 to fix. The binding works fine within the development IDE, but only there. I'm still looking into it.



  • Did that, MQTT bridge stuck at "initializing". Not the full answer yet, but thanks @timo for your effort!

    Openhab.log when I checked the Services - MQTT in Paper UI

    2018-06-08 00:59:19.162 [ERROR] [rotocol.mqtt.MySensorsMqttConnection] - MQTT connection offline - Reason unknown
    2018-06-08 00:59:19.221 [WARN ] [me.config.core.internal.ConfigMapper] - Could not set field value for field 'port': For input string: "1883.0"
    java.lang.NumberFormatException: For input string: "1883.0"
    	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:?]
    	at java.lang.Integer.parseInt(Integer.java:580) [?:?]
    	at java.lang.Integer.valueOf(Integer.java:766) [?:?]
    	at org.eclipse.smarthome.config.core.internal.ConfigMapper.objectConvert(ConfigMapper.java:159) [98:org.eclipse.smarthome.config.core:0.10.0.oh230]
    	at org.eclipse.smarthome.config.core.internal.ConfigMapper.as(ConfigMapper.java:98) [98:org.eclipse.smarthome.config.core:0.10.0.oh230]
    	at org.eclipse.smarthome.config.core.Configuration.as(Configuration.java:54) [98:org.eclipse.smarthome.config.core:0.10.0.oh230]
    	at org.eclipse.smarthome.io.transport.mqtt.internal.MqttBrokerConnectionServiceInstance.modified(MqttBrokerConnectionServiceInstance.java:77) [207:org.eclipse.smarthome.io.transport.mqtt:0.10.0.oh230]
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
    	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:?]
    	at org.apache.felix.scr.impl.inject.BaseMethod.invokeMethod(BaseMethod.java:229) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.scr.impl.inject.BaseMethod.access$500(BaseMethod.java:39) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.scr.impl.inject.BaseMethod$Resolved.invoke(BaseMethod.java:650) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.scr.impl.inject.BaseMethod.invoke(BaseMethod.java:506) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.scr.impl.inject.ActivateMethod.invoke(ActivateMethod.java:307) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.scr.impl.inject.ActivateMethod.invoke(ActivateMethod.java:299) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.scr.impl.manager.SingleComponentManager.invokeModifiedMethod(SingleComponentManager.java:772) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.scr.impl.manager.SingleComponentManager.modify(SingleComponentManager.java:727) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:645) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.scr.impl.manager.SingleComponentManager.reconfigure(SingleComponentManager.java:609) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.scr.impl.manager.ConfigurableComponentHolder.configurationUpdated(ConfigurableComponentHolder.java:426) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.scr.impl.manager.RegionConfigurationSupport.configurationEvent(RegionConfigurationSupport.java:284) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.scr.impl.manager.RegionConfigurationSupport$1.configurationEvent(RegionConfigurationSupport.java:89) [41:org.apache.felix.scr:2.0.12]
    	at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.sendEvent(ConfigurationManager.java:2090) [8:org.apache.felix.configadmin:1.8.16]
    	at org.apache.felix.cm.impl.ConfigurationManager$FireConfigurationEvent.run(ConfigurationManager.java:2058) [8:org.apache.felix.configadmin:1.8.16]
    	at org.apache.felix.cm.impl.UpdateThread.run0(UpdateThread.java:141) [8:org.apache.felix.configadmin:1.8.16]
    	at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:109) [8:org.apache.felix.configadmin:1.8.16]
    	at java.lang.Thread.run(Thread.java:748) [?:?]
    2018-06-08 00:59:19.340 [WARN ] [.MqttBrokerConnectionServiceInstance] - Ignore existing broker connection configuration for: mosquitto```

  • Hero Member

    @masmat please check your MQTT configuration. You're receiving an exception because it's trying to set "1883.0" for the value port.



  • @timo I did. I can't figure out where it's finding that .0 it adds to the 1883 port I have in the cfg-file



  • How close is this for 2.3? Worth a try? or should I hold? Thanks for the work so far on it!



  • @crankycoder Unfortunately there are still issues with the mqtt implementation. I reverted back to 2.2 yesterday as all my fiddling didn't help.



  • my setup is still just using the direct mqtt subscription so i will hold off.

    is there any discovery working (in 2.2 that is) how about once you configure the things. do you get all the properties of that item from when it presents?



  • @crankycoder in 2.2 with appropriate binding mysensors is working pretty well. All variables are presented with mysensors lib 2.2.
    I only updated openhab to 2.3 as this release was presented on my debian system. But I didn't get 2.3 working with mysensors mqtt.



  • I did an upgrade the other week from 1.8 to 2.2. Then right about the time I had some stuff working 2.3 came out. So I figured since I have to do all the learning curve anyways I might as well stick with 2.3. The variables being presented in mysensors would be HUGE for me. But hopefully it's not too far out.


 

430
Online

7.5k
Users

8.3k
Topics

89.8k
Posts