OpenHAB/MQTT Tips & Hints

[Yveaux]

This topic was split off http://forum.mysensors.org/topic/301/the-best-way-to-connect-to-openhab to discuss tips and hints specific to using MySensors together with MQTT middleware and OpenHAB.
If you made something nice using this combination or have some questions/issues please post them in here!

---

@kolaf I just started experimenting with openhab and find it quite hard to get started. There's some documentation ( far from complete, especially when you're just starting) but the general impression is that it's very powerful, mainly due to all the programming options. I'm quite sure you could directly talk to a mysensors gateway using the serial protocol, using e.g. rules, if you want.
I doubt however if you want to write your own protocol handler...
Anyway, I'm currently using an Ethernet gateway which tasks to a Perl script I wrote (https://github.com/Yveaux/MySensors_MQTTGateway) that does the conversion to and from MQTT. This script is a MQTT client that runs on the same server I run openhab and the mosquitto broker on. This is the only difference compared to @Damme solution who runs a broker on the gateway.
I like the flexibility of storing & accessing all data through an MQTT broker which makes up for the apparent overkill going through MQTT just to connect mysensors to openhab.
As long as your server has enough resources to run it all its not really worth the effort to directly talk to my sensors or create an openhab binding for it.

[Yveaux]

@kolaf said:

how do you use the mqtt binding to talk to the sensors?

I'm not at my PC right now so I can't add any actual code, but let's take the switch as an example.
A sensor (with say, id 100) which has the switch connected sends a 1 when pressed and a 0 when released to the gateway, with child I'd 7. This comes out the gateway and is picked up by the perl script. The script publishes this to /mysensors/100/7/V_LIGHT.
You define a switch item in openhab which subscribes to this topic. To read the state of the switch openhab expects either ON or OFF from MQTT instead of 1 or 0 sent by the sensor. I could have changed this on the sensor side, but I choose to implement a map-transformation on the openhab-side. Now the state of the switch changes to ON when pressed on the sensor.

I also implemented a dimmer which can be controlled by 1or 2 momentary switches (on different nodes) and an rgb dimmer so far. Maybe we can start a separate topic on MQTT & openhab usage for different sensors so we can all benefit from it.

[kolaf]

@Yveaux said:

I also implemented a dimmer which can be controlled by 1or 2 momentary switches (on different nodes) and an rgb dimmer so far. Maybe we can start a separate topic on MQTT & openhab usage for different sensors so we can all benefit from it.

Good idea. If someone made a good, or at least workable and well-documented, solution than I am sure this project would be much utilised by the openHAB crowd as well. Maybe explaining it to me could be a good start for a how to/guide which could also be utilised in their wiki?

I hope I'm not stepping on any Vera toes when saying this?

[Yveaux]

@kolaf said:

I hope I'm not stepping on any Vera toes when saying this?

Not everybody owns a vera and IMHO it's only beneficial for this project when multiple home automation systems are supported. Finding a greatest common divider for e.g. the types of variables will only improve things.

[kolaf]

@Yveaux good, I agree, and I am glad to hear it.

How would you solve a toggle switch? The sensor does not know whether the light is on or off since it may be toggled through the controller separately. This means that in my head it will simply send "on" every time and it is up to the item/map to figure out whether it should be turned on or off depending on its current state. I guess this may be required as a rule. Or does your server support sending items dated back to the sensor so that it knows whether the light is off or on currently? In that case it can make the decision itself and send the correct on or off command. Maybe I should ask this question in the openhab group...

[tboha]

@kolaf : I´m not very experienced in Java, so the code reflects my missing knowledge and is much longer than it should be.

The purpose of "Split Message" is to split multilpe messages into single messages and afterward split this single messages into useful informations to set the OpenHAB items accordingly.

First part: common declarations

import org.openhab.core.library.types.*
import org.openhab.core.persistence.*
import org.openhab.model.script.actions.*

var String[] buffer
var String[] linebuffer
var int SensorID
var int transmissions_old = 0
var int transmissions_new = 0  
var int transmissions_missed = 0 
var int RadioID

Second part: Splitting multiple Messages

rule SplitMessage
when 
	Item Arduino changed
 then
 /* Split messages separated with NEWLINE */
linebuffer=  Arduino.state.toString.split("\n")

Third part: Splitting messages according to the serial protocol
buffer(0) = NODE_ID
buffer(1) = CHILD_ID
buffer(2) = MESSAGE_TYPE
buffer(3) = SUB_TYPE
buffer(4) = Message

 for (String element : linebuffer) {
	buffer = element.split(";")
	RadioID = new Integer(buffer.get(0))
	
	switch (RadioID) {
	case 7: {
			SensorID = new Integer(buffer.get(1))
			switch (SensorID) {
				case 0 : postUpdate (MySensorsT0, buffer.get(4))
				case 1 : postUpdate (MySensorsT1, buffer.get(4))
				case 2 : postUpdate (MySensorsT2, buffer.get(4))
				case 3 : postUpdate (MySensorsT3, buffer.get(4))
				} /*switch (SensorID) */
				
			}  /* case 7 */
	case 6: {													 /* ExperimentalNode 6 - soll mal NODE 1 werden */
			if (buffer.get(1)== "10") {							 /* child 10 ist der Homematic Anschluss */
				postUpdate (HMSerialOut, buffer.get(4))	
			} /* if */
	} /* case 6 */
	case 9: {                                                      /* eigentlich war das mal NODE 6 */
			if (buffer.get(1) == "2") {                            /* Child 2 ist der Schrittmacher      */
				transmissions_new = new Integer(buffer.get(4))
				logInfo ("TRANS","Transmissions new " + transmissions_new.toString() )
				logInfo ("TRANS","Transmissions old " + transmissions_old.toString() )
				if (transmissions_old == 0) {transmissions_old = transmissions_new -1 } /* beim ersten mal passiert nichts */
				transmissions_missed = transmissions_missed + (transmissions_new - transmissions_old - 1)
				transmissions_old = transmissions_new 
				postUpdate (Missed_Transmissions, transmissions_missed.intValue.toString)
				postUpdate (Transmission_Count, transmissions_new.toString())
				}
			} /* case 9 */
	case 5: {
				if (buffer.get(1) == "0") { 						/* Child 0 ist die Luftfeuchte */
					postUpdate (MySensorsMobHum, buffer.get(4))
					}
				if (buffer.get(1) == "1") { 						/* Child 1 ist die Temperatur */
					postUpdate (MySensorsMobTemp, buffer.get(4))
					}
							
			} /*case 5: */
	default:	{
			postUpdate (MySensorsNode, buffer.get(0))
			postUpdate (MySensorsChild, buffer.get(1))
			postUpdate (MySensorsMtype, buffer.get(2))
			postUpdate (MySensorsStype, buffer.get(3))
			postUpdate (MySensorsMessage, buffer.get(4)) 
			}
			
	  }   /*switch(RadioID) */ 		   	
	} /*for (String element) */
	
end

So the drawback of the serial binding becomes obvious - every action has to be coded separately. On the other hand it offers enormous controlling possibilities (eg scene configurations),

Fourth part: some actions triggerd from OpenHAB GUI/Webinterface:

rule ArdSwon
when 
	Item ArduinoSwitch changed from OFF to ON
then
	sendCommand(Arduino, "4;2;1;0;2;1\n")
end

rule ArdSwoff
when 
	Item ArduinoSwitch changed from ON to OFF
then
	sendCommand(Arduino, "4;2;1;0;2;0\n")
end

rule HmArdon
when
 Item ArduinoHMSw1	changed from ON to OFF
then
	sendCommand(Arduino, "1;10;1;0;24;HD01004000000\n")
end

rule HmArdoff
when
 Item ArduinoHMSw1	changed from OFF to ON
then
	sendCommand(Arduino, "1;10;1;0;24;HD01004010000\n")
end

To send commands to the MySensors Network you have to use the same "Arduino"Item. In my oppinion another drawback. A separate way out would be nicer. Despite of my oppinion there is no interference between input and output - so i can live with this.

The above example illustrates controlling a LED connected to an Arduino-UNO with the original Relais-Sketch, the second part controls some of my Homematic devices via another MySensors node (connected to Homematic via USB) .

So at last I got a 2.4Ghz network to communicate with my Homematic and an ethernet connection via OpenHAB. In combination with direct interaction between certain MySensor nodes this results in a very redundant and stress-resistant home controlling network.

[Yveaux]

@kolaf said:

Or does your server support sending items dated back to the sensor so that it knows whether the light is off or on currently?

To make sure I understand you right, are both the switch & light located at the same node or do you have a sensor node (switch) and actuator node (light) ?

[kolaf]

@Yveaux said:

@kolaf said:

Or does your server support sending items dated back to the sensor so that it knows whether the light is off or on currently?

To make sure I understand you right, are both the switch & light located at the same node or do you have a sensor node (switch) and actuator node (light) ?

The light is entirely separate from the switch. In my specific case it will typically be a Z-wave relay controlling the light, so I will have to map the toggle switch to the light relay in openhab. This is why I assume I have to use a rule to achieve this.

[Yveaux]

@kolaf Here's my code to use one or two switches to dimm. I hope it gives you some inspiration to connect your zwave switch.

MQTT topics (exposed by the MQTT perl script. Node 120 = switch sensor, node 119 = dimmable light):

/mySensors/120/0/V_LIGHT           switch, reporting 1 when pressed, 0 when released
/mySensors/120/1/V_LIGHT           switch, reporting 1 when pressed, 0 when released
/mySensors/120/2/V_LIGHT           switch, reporting 1 when pressed, 0 when released
/mySensors/119/0/V_DIMMER          dimmable light, accepting integer value between 0 and 100

Items:

Switch Switch_Up               {mqtt="<[server:/mySensors/120/0/V_LIGHT:state:MAP(switchFromMqtt.map)]"}
Switch Switch_Down             {mqtt="<[server:/mySensors/120/1/V_LIGHT:state:MAP(switchFromMqtt.map)]"}
Switch Switch_UpDown           {mqtt="<[server:/mySensors/120/2/V_LIGHT:state:MAP(switchFromMqtt.map)]"}
Dimmer Light_Dimmed   (Lights) {mqtt=">[server:/mySensors/119/0/V_DIMMER:state:*:default]"}

switchFromMqtt.map: (in transform folder)

0=OFF
1=ON 

Rules for 2-switch dimmer control (short press Switch_Up switches light on, short press Switch_Down switches light off, keep pressed to increase/decrease light level):

val Long DimmerDelayMs = 333L 

rule "DimUp"
when
  Item Switch_Up received update ON
then
  var Boolean dimmed = false
  do
  {
    Thread::sleep(DimmerDelayMs) 
    if (Switch_Up.state == ON)
    {
      sendCommand( Light_Dimmed, INCREASE )
      dimmed = true;
    }
  } while (dimmed)
  if (!dimmed) sendCommand( Light_Dimmed, ON )
end

rule "DimDown"
when
  Item Switch_Down received update ON
then
  var Boolean dimmed = false
  do
  {
    Thread::sleep(DimmerDelayMs) 
    if (Switch_Down.state == ON)
    {
      sendCommand( Light_Dimmed, DECREASE )
      dimmed = true;
    }
  } while (dimmed)
  if (!dimmed) sendCommand( Light_Dimmed, OFF )
end

Rules for 1-switch dimmer control (short press switches light on/off (depending on current state), keep pressed to increase/decrease light level):

rule "DimUpDown"
when
  Item Switch_UpDown received update ON
then
  var Boolean dimmed = false
  var Number percent = 0
  if (Light_Dimmed.state instanceof DecimalType) percent = Light_Dimmed.state as DecimalType 
  do
  {
    Thread::sleep(DimmerDelayMs) 
    if (Switch_UpDown.state == ON)
    {
      // Start cycling up/down until released
      var Boolean dimmUp = percent < 100
      do
      {
        if (Light_Dimmed.state instanceof DecimalType) percent = Light_Dimmed.state as DecimalType 
        if (dimmUp) sendCommand( Light_Dimmed, INCREASE )
        else		sendCommand( Light_Dimmed, DECREASE )
        if (percent >= 100)	dimmUp = false;
        if (percent <= 0)	dimmUp = true;
        dimmed = true;
        Thread::sleep(DimmerDelayMs) 
      } while (Switch_UpDown.state == ON)
    }
  } while (dimmed)
  if (!dimmed)
  {
    // Short press: switch on or off, depending on current state
    if (percent > 0)	sendCommand( Light_Dimmed, OFF )
    else				sendCommand( Light_Dimmed, ON )
  }
end

Rule for the dimmable light:

rule "MyDimmer0"
  when
    Item Light_Dimmed received command
  then
    var Number percent = 0
  
    if(Light_Dimmed.state instanceof DecimalType) percent = Light_Dimmed.state as DecimalType 
    if(receivedCommand==INCREASE) percent = percent + 5
    if(receivedCommand==DECREASE) percent = percent - 5
    if(receivedCommand==ON) percent = 100		
    if(receivedCommand==OFF) percent = 0		
    if(percent<0)   percent = 0
    if(percent>100) percent = 100
    postUpdate(Light_Dimmed, percent)
end

Some stuff I'm still struggling with (any help/ideas appreciated):

• Not sure if I can wrap (parts of) rules in a function to make re-use easier
• Getting the current value of an item seems complicated first testing for DecimalType, then getting it. Maybe this can be done more efficient?
• I use Thread::sleep to determine the timing of the buttons on the sensor node (pressed short/long). This will be influenced by the jitter on the sensor values received, but currently it seems to work fine. Furthermore Thread::sleep (probably) blocks whole rule processing, so this isn't a nice solution. The short/long presses can also be dermined on the sensor node and reported with different values, but then the sensor node is no longer a dumb switch and has to have knowledge of short/long presses....

[kolaf]

@Yveaux Excellent, this is just what I needed, thank you.

I started looking at your Perl script since this seems like the most active solution for openhab integration. I have a couple questions if you don't mind.

Why do you have two versions of the gateway script, and which version should I use as a basis when adding serial support?

Does your gateway take care of node ID assignments? I guess I will figure out that by reading more of code

I'm thinking that I will fork your project to add a serial option to your script, controlled by some kind of commandline argument.

[Yveaux]

@kolaf said:

@Yveaux Excellent, this is just what I needed, thank you.

Glad I could help!

Why do you have two versions of the gateway script, and which version should I use as a basis when adding serial support?

The serial format changed from MySensors 1.3 (protocol 1) to 1.4beta (protocol 2). I was lazy and just cloned the 1.3 version to add 1.4 support (the 2-version). Wouldn't be hard to support both, but I really wrote this script for my own usage. It's not documented and I tried to help Damme in the past to build MQTT support. I'm primarily on 1.4b btw, with some 1.3 setup for backwards sniffer support...
If more people start using it I should do some things the nice way

Does your gateway take care of node ID assignments? I guess I will figure out that by reading more of code

Not at the moment. Shouldn't be hard to implement though, but when you implement it you run into another issue of mapping the node-ID's onto the MQTT topic tree, or accessing the right topic from OpenHAB.
Sticking to fixed Node ID's seemed to make more sense to me.

I'm thinking that I will fork your project to add a serial option to your script, controlled by some kind of commandline argument.

Feel free to fork, but the current code is based on AnyEvent. Not that I like it, but it seemed to have the best MQTT support. There is some hacking in the script to get things working, for which I don't know how they behave when switching to serial.

[Yveaux]

Btw. there's one important thing to understand when using the Perl script. When it receives values from a sensor node through the gateway it's easy to publish the value in the topic-tree.
When a value has to be sent to an actuator node, the story is different as the script has to know which topic to subscribe to at the MQTT broker.
Currently when a mysensor node requests a value from the gateway the script automagically translates this into a subscription of the corresponding topic with the MQTT broker. The dimmer node for example, calls gw.request(childID, V_DIMMER) from setup to subscribe itself to dimmer messages.
Note that sometimes this request gets lost (CRC error or so) and the subscription fails. While not supported by the MySensors library, a robust implementation should wait for a response after requesting this value and retry when it doesn't come.
The Perl script stores all current subscriptions persistent (file subscriptions.storage) so restarting the Perl script will not force you to restart all nodes to subscribe again.

[Yveaux]

And another small one; an RGB dimmer!

MQTT topics (exposed by the MQTT perl script. Node 118 = 3-channel dimmable light):

/mySensors/118/0/V_DIMMER          dimmable light RED, accepting integer value between 0 and 100
/mySensors/118/1/V_DIMMER          dimmable light GREEN, accepting integer value between 0 and 100
/mySensors/118/2/V_DIMMER          dimmable light BLUE, accepting integer value between 0 and 100

Items:

Dimmer Light_R      {mqtt=">[server:/mySensors/118/0/V_DIMMER:state:*:default]"}
Dimmer Light_G      {mqtt=">[server:/mySensors/118/1/V_DIMMER:state:*:default]"}
Dimmer Light_B      {mqtt=">[server:/mySensors/118/2/V_DIMMER:state:*:default]"}
Color Light_RGB 	"RGB Dimmer" 	(Lights)

Rules to distribute colorwheel value over R,G,B dimmers:

rule "Set RGB value"
when
        Item Light_RGB changed
then
        var HSBType hsbValue = Light_RGB.state as HSBType
        postUpdate( Light_R, hsbValue.red.intValue )
        postUpdate( Light_G, hsbValue.green.intValue )
        postUpdate( Light_B, hsbValue.blue.intValue )
end

No button control here; just using the colorwheel and on/off buttons in the GUI.

Enjoy!

[Damme]

rule to calculate absolute humidity and dew point from degree Celsius and rH%

import java.lang.Math
import java.lang.Integer
import java.lang.Double


rule "Calculate absolute humidity (g h2o / m3 air) and dew point"
when
    Item temp1 changed or
    Item hum1 changed
then
    var temp = temp1.state as DecimalType
    var hum = hum1.state as DecimalType
	
    var t1 = (17.271*temp.floatValue) / (237.7+temp.floatValue) + Math::log(hum.floatValue*0.01)
    var dew = (237.7 * t1) / (17.271 - t1)
    var Number c1 = ((17.67*temp.floatValue)/(temp.floatValue+243.5))
    var abs = (Math::pow(Math::E,c1.doubleValue)*6.112*2.1674*hum.floatValue) /(273.15+temp.floatValue)
	
    Dewpoint1.postUpdate(dew)
    AbsHum1.postUpdate(abs)
end

[Yveaux]

@Damme Nice calculation example! Thanks!

[Dany]

Hi everyone!

I'm new in the topic. I find it very interesting the world of mysensors.
I would like to ask whether there is a description of someone that I can set my mqtt openHAB bindings eg .: an Arduino LED dimmer ?

I already read http://forum.mysensors.org/topic/303/mqtt broker-gateway
I made a Humidity sensor node, and Relay node, too.

It would be good if we had a basic description or example project for beginners from all sensor type.

[pgo]

It is not rocket science to get the openHAB running w/MQTT gateway, see for example my post with DS/Light/Relay in http://forum.mysensors.org/topic/115/implementing-multiple-sensors/60

But sure, it would be great to put a wiki with all sensor settings for openHAB together on one page. I needed to read/search for some days to put the knowledge together...

Example of the openhab screenshots on mobile https://github.com/pgo-sk/mysensors/wiki/Home-automation-using-mysensors-and-openHAB
There you see also the mapping of the sensor

[adi32k]

How can I make openhad respod to gw.request(sensor, V_HEATER_SW,0);

I have a relay actuator sketch and in setup() I have this

for (int sensor=1 ; sensor<=NUMBER_OF_RELAYS;sensor++)
{
gw.present(sensor, S_HEATER);
gw.request(sensor, V_HEATER_SW,0);
}

practically I would like openhab to respond to the gw.request with the actual state of the relay.

My item definition is the follwing. I am able to ON and OFF the relay, but I need to find a way to get the values from openhab of the relays when the relay actuator arduino reboots.

Switch Incalzire_Releu_GF_Living2 "Incalzire Releu Living 2" <heating> (Incalzire) {mqtt=">[mysensor:MyMQTT/3/2/V_HEATER_SW:command:ON:1],>[mysensor:MyMQTT/3/2/V_HEATER_SW:command:OFF:0]"}

[raditv]

is there any sample code for controlling RGB led here?

[Chaotic]

Sorry to Necro this thread but had a question,

I believe I read somewhere that you can use a serial gateway connected to a pi and have openhab/mqtt run on the pi?

How would one go about setting this up? Currently I have 2 Unos one running as the serial gateway. I plan to replace the serial gateway with a nano I'm still waiting on that.

I just got the pi last night so I'm still working on getting everything up and running on that but I'd rather just connect the gateway directly to the pi rather than have to get a wifi/ethernet module for one of the arduinos.

Also probably outside the scope of this thread (and thats fine) but anyone have a good guide for setting up the pi for openhab/MQTT?

[mhortman]

@Chaotic , Try this..
https://github.com/abouillot/HomeAutomation/tree/master/piGateway
Worked great for me

[Sweebee]

@mhortman Didn't work for me:

sudo wget http://repo.mosquitto.org/debian/mosquitto-wheezy.list~
--2015-03-27 10:46:51--  http://repo.mosquitto.org/debian/mosquitto-wheezy.list~
Resolving repo.mosquitto.org (repo.mosquitto.org)... 85.119.83.194, 2001:ba8:1f1:f271::2
Connecting to repo.mosquitto.org (repo.mosquitto.org)|85.119.83.194|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2015-03-27 10:46:51 ERROR 403: Forbidden.'

[quocanhcgd]

My house have 4 floor, i have plan build one gateway for each floor. Each floor has 4-5 sensors (temp, hum, relay, light, door, RF light). Each sensors use NRF24 to connect with gateway. Gateway connect to RAS by ethernet.
My question:

• Can i build 2 gateway mqtt connect to openhab?
  if not, what my solution to solve ?
  Thanks

[rachmat aditiya27]

you can build more than 1 gateway for that, but rather than mqtt. I think it's better to use ethernet or serial gateway because mqtt gateway has lot of problem, I tried it for months. You can forward mqtt from fhem to mosquito.

[siod]

@Yveaux
could you please post your arduino code of controlling your RGB (I guess it´s a RGB LED Strip!?)?

Thanks for your help!!

[siod]

I build up a mqtt gateway and configured it in openhab.cfg. Now I would like to set up a mosquitto server on a raspberry pi, too. Is it possible to add this new MQTT server to the openhab.cfg beside the already set up MQTT gateway?

[tomkxy]

Why not setup the Mosquitto broker connect OpenHab to it and use the MQTTClientGateway which connects as a client to the Mosquitto broker. Has been discussed at several places in the forum already. You can find the MQTTClientGateway in the development branch.

[John Connolly]

This post is deleted!

[siod]

@tomkxy
I´m reading in the forums for weeks now, but I don´t get it to work. I´ve succesfully set up Mosquitto on my raspi, also openHAB is running there. MQTTEthernetGateway (from Startpage) is pingable. One sensor is running and working.

I don´t know how to get the MQTTEthernetGateway communicate with the Mosquitto Broker. I´ve read about "bridging" what made me more confused as I already was.

So, right now, when I subscribe to MyMQTT/# using "mosquitto_sub -t MyMQTT/#" on my Raspi I don´t see any communication coming in.

What do I have to do to get the MQTTEthernetGateway (=MQTTClientGateway I suppose!?!) connecting to Mosquitto?

Or: must I setup the MQTTEthernetGateway like this: http://forum.mysensors.org/topic/524/mqtt-client-gateway instead of using the sketch from http://www.mysensors.org/build/mqtt_gateway?

Thanks for your help!!

[tomkxy]

@siod I am not sure to what code you are exactly referring to if you say MQTTEthernetGateway.
In the mysensor Github he cleaned up the code I merged from my Github. I did not test that code so far.
So I can only comment on the MQTTClientGateway you can find in my Github https://github.com/tomkxy/Arduino.git in branch MQTTClientGateway.

I suggest you start off ignoring Openhab at the moment. Just test with clients using mosquitto_sub and mosquito_pub commands from mosquito.

In the MQTTClientGateway sketch you need to adapt the IP related infos. See excerpt from my config.

//replace with ip of server you want to connect to, comment out if using 'remote_host'
uint8_t remote_ip[] =  { 192, 168, 178, 74 };  // Mosquitto broker

//replace with hostname of server you want to connect to, comment out if using 'remote_ip'
//char* remote_ip = "server.local";
//replace with the port that your server is listening on
#define remote_port 1883
//replace with arduinos ip-address. Comment out if Ethernet-startup should use dhcp. Is ignored on Yun
uint8_t local_ip[] = {192, 168, 178, 11};
//replace with ethernet shield mac. It's mandatory every device is assigned a unique mac. Is ignored on Yun
uint8_t mac[] = { 0xA2, 0xAE, 0xAD, 0xA0, 0xA0, 0xA2 };

Depending on whether you use the signing feature or not you have to comment the define MY_SIGNING_FEATURE in my_config.h.

If your sketch is running you should see one additional client connected on your mosquito broker.
You can subscribe with mosquito_sub to $SYS/# which gives you all sorts of statistics.

In order to check whether your Arduino connected to the broker you can also put a debug statement in the loop function. Just check and print the return code from the client.connect() call.

If the connect did not work double check that you configured the correct IP address and port, and make also sure that you fulfill the authentication requirements configured in the broker.

If the connection works publish a message to the broker for a topic like MyMQTT/27/5/V_PRESSURE
The MQTTClientGateway sketch should receive that and you should see that in the log.

Next reset your sensor node. You should see a message being processed by MQTTClientGateway and then being published to the MQTT broker.

Only after all that works you start dealing with OpenHab.

Hope that helps.

[siod]

Hi tomkxy,

thanks for your detailed answer, I will test all your suggestions after I got my GW work on LAN, right now it´s not answering to pings and I don´t get it why....

I´ve uploaded the ntruchsees sketch from http://forum.mysensors.org/topic/524/mqtt-client-gateway now and configured IP address, port and MAC.
When I was talking about MQTTEthernetGateway I was talking about the MySensors Sketch at http://forum.mysensors.org/topic/524/mqtt-client-gateway.

I will get back to you and give your more info as soon as the Gateway in it´s actual state works in my network!

[tomkxy]

@siod I adapted the ntruchsees for the MySensors library version 1.5 which you find here
https://github.com/tomkxy/Arduino.git

[siod]

@tomkxy

I could compile your code now, unfortunately I still cannot ping the gw.

BTW: In ntruchsess`code I had to enter the Mosquitto broker IP, why isn´t this part in your code?

//replace with ip of server you want to connect to, comment out if using 'remote_host'
uint8_t remote_ip[] = { 192, 168, 1, 50 };

[tomkxy]

It is the same in my code. See the code extract in the previous reply above. The variable is remote_ip[].

[siod]

sorry, but I can´t fin it in your code:

/**
 * The MySensors Arduino library handles the wireless radio link and protocol
 * between your home built sensors/actuators and HA controller of choice.
 * The sensors forms a self healing radio network with optional repeaters. Each
 * repeater and gateway builds a routing tables in EEPROM which keeps track of the
 * network topology allowing messages to be routed to nodes.
 *
 * Created by Henrik Ekblad <henrik.ekblad@mysensors.org>
 * Copyright (C) 2013-2015 Sensnology AB
 * Full contributor list: https://github.com/mysensors/Arduino/graphs/contributors
 *
 * Documentation: http://www.mysensors.org
 * Support Forum: http://forum.mysensors.org
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * version 2 as published by the Free Software Foundation.
 *
 *******************************
 *
 * REVISION HISTORY
 * Version 1.0 - Created by Daniel Wiegert <daniel.wiegert@gmail.com>
 * 
 * DESCRIPTION
 * MyMQTT Broker Gateway 0.1b
 * Latest instructions found here:
 * http://www.mysensors.org/build/mqtt_gateway
 * http://www.mysensors.org/build/ethernet_gateway
 * 
 * Change below; TCP_IP, TCP_PORT, TCP_MAC
 * This will listen on your selected TCP_IP:TCP_PORT below, Please change TCP_MAC your liking also.
 * 1 -> NOTE: Keep first byte at x2, x6, xA or xE (replace x with any hex value) for using Local Ranges.
 * 2 You can use standard pin set-up as MySensors recommends or if you own a IBOARD you may change
 *	the radio-pins below if you hardware mod your iBoard. see [URL BELOW] for more details.
 *	http://forum.mysensors.org/topic/224/iboard-cheap-single-board-ethernet-arduino-with-radio/5
 *
 * Don't forget to look at the definitions in MyMQTT.h!
 *
 *	define TCPDUMP and connect serial interface if you have problems, please write on
 *	http://forum.mysensors.org/ and explain your problem, include serial output. Don't forget to
 *	turn on DEBUG in libraries\MySensors\MyConfig.h also.
 *
 *	MQTT_FIRST_SENSORID is for 'DHCP' server in MyMQTT. You may limit the ID's with FIRST and LAST definition.
 *	If you want your manually configured below 20 set MQTT_FIRST_SENSORID to 20.
 *	To disable: set MQTT_FIRST_SENSORID to 255.
 *
 *	MQTT_BROKER_PREFIX is the leading prefix for your nodes. This can be only one char if like.
 *
 *	MQTT_SEND_SUBSCRIPTION is if you want the MyMQTT to send a empty payload message to your nodes.
 *	This can be useful if you want to send latest state back to the MQTT client. Just check if incoming
 *	message has any length or not.
 *	Example: if (msg.type==V_LIGHT && strlen(msg.getString())>0) otherwise the code might do strange things.
 *
 * (*) Address-layout is : [MQTT_BROKER_PREFIX]/[NodeID]/[SensorID]/V_[SensorType]
 *	NodeID and SensorID is uint8 (0-255) number.
 *	Last segment is translation of the sensor type, look inside MyMQTT.cpp for the definitions.
 *	User can change this to their needs. We have also left some space for custom types.
 *
 * Special: (sensor 255 reserved for special commands)
 * You can receive a node sketch name with MyMQTT/20/255/V_Sketch_name (or version with _version)
 *
 * To-do:
 * Special commands : clear or set EEPROM Values, Send REBOOT and Receive reboot for MyMQTT itself.
 * Be able to send ACK so client returns the data being sent.
 * ... Please come with ideas!
 * What to do with publish messages.
 *
 * Test in more MQTT clients, So far tested in openhab and MyMQTT for Android (Not my creation)
 * - http://www.openhab.org/
 * - https://play.google.com/store/apps/details?id=at.tripwire.mqtt.client&hl=en
 * ... Please notify me if you use this broker with other software.
 * 
 *  How to set-up Openhab and MQTTGateway:
 * http://forum.mysensors.org/topic/303/mqtt-broker-gateway
 */

#include <DigitalIO.h>
#include <SPI.h>

#include <MySigningNone.h>
#include <MyTransportRFM69.h>
#include <MyTransportNRF24.h>
#include <MyHwATMega328.h>
#include <MySigningAtsha204Soft.h>
#include <MySigningAtsha204.h>

#include <MySensor.h>
#include <MsTimer2.h>
#include <Ethernet.h>
#include "MyMQTT.h"


#define INCLUSION_MODE_TIME 1 // Number of minutes inclusion mode is enabled
#define INCLUSION_MODE_PIN  3 // Digital pin used for inclusion mode button

// * Use this for IBOARD modded to use standard MISO/MOSI/SCK, see note *1 above!
/*
#define RADIO_CE_PIN        3			// radio chip enable
#define RADIO_SPI_SS_PIN    8			// radio SPI serial select
#define RADIO_ERROR_LED_PIN A2  		// Error led pin
#define RADIO_RX_LED_PIN    A1  		// Receive led pin
#define RADIO_TX_LED_PIN    A0  		// the PCB, on board LED
*/

// * Use this for default configured pro mini / nano etc :
///*

#define RADIO_CE_PIN        5		// radio chip enable
#define RADIO_SPI_SS_PIN    6		// radio SPI serial select
#define RADIO_ERROR_LED_PIN 7		// Error led pin
#define RADIO_RX_LED_PIN    8		// Receive led pin
#define RADIO_TX_LED_PIN    9		// the PCB, on board LED*/

#define TCP_PORT 1883						// Set your MQTT Broker Listening port.
IPAddress TCP_IP ( 192, 168, 1, 51 );				// Configure your static ip-address here
byte TCP_MAC[] = { 0x02, 0xDE, 0xAD, 0x00, 0x00, 0x42 };	// Mac-address - You should change this! see note *2 above!

//////////////////////////////////////////////////////////////////

// NRFRF24L01 radio driver (set low transmit power by default) 
MyTransportNRF24 transport(RADIO_CE_PIN, RADIO_SPI_SS_PIN, RF24_PA_LEVEL_GW);  
//MyTransportRFM69 transport;

// Message signing driver (signer needed if MY_SIGNING_FEATURE is turned on in MyConfig.h)
//MySigningNone signer;
//MySigningAtsha204Soft signer;
//MySigningAtsha204 signer;

// Hardware profile 
MyHwATMega328 hw;

// Construct MySensors library (signer needed if MY_SIGNING_FEATURE is turned on in MyConfig.h)
MySensor gw(transport, hw /*, signer*/);


EthernetServer server = EthernetServer(TCP_PORT);
EthernetClient *currentClient = NULL;
MyMessage msg;
char convBuf[MAX_PAYLOAD*2+1];
char broker[] PROGMEM = MQTT_BROKER_PREFIX;
bool MQTTClientConnected = false;
uint8_t buffsize;
char buffer[MQTT_MAX_PACKET_SIZE];
volatile uint8_t countRx;
volatile uint8_t countTx;
volatile uint8_t countErr;


void writeEthernet(const char *writeBuffer, uint8_t *writeSize) {
#ifdef TCPDUMP
	Serial.print(">>");
	char buf[4];
	for (uint8_t a=0; a<*writeSize; a++) { sprintf(buf,"%02X ",(uint8_t)writeBuffer[a]);  Serial.print(buf); } Serial.println("");
#endif
	server.write((const uint8_t *)writeBuffer, *writeSize);
}


void processEthernetMessages() {
  char inputString[MQTT_MAX_PACKET_SIZE] = "";
  byte inputSize = 0;
  byte readCnt = 0;
  byte length = 0;

  EthernetClient client = server.available();
  if (client) {
    while (client.available()) {
      // Save the current client we are talking with
      currentClient = &client;
      byte inByte = client.read();
      readCnt++;

      if (inputSize < MQTT_MAX_PACKET_SIZE) {
        inputString[inputSize] = (char)inByte;
        inputSize++;
      }

      if (readCnt == 2) {
        length = (inByte & 127) * 1;
      }
      if (readCnt == (length+2)) {
        break;
      }
    }
#ifdef TCPDUMP
    Serial.print("<<");
    char buf[4];
    for (byte a=0; a<inputSize; a++) { sprintf(buf, "%02X ", (byte)inputString[a]); Serial.print(buf); } Serial.println();
#endif
    processMQTTMessage(inputString, inputSize);
    currentClient = NULL;
  }
}




void incomingMessage(const MyMessage &message) {
   rxBlink(1);
   sendMQTT(message);
} 



void setup()  
{ 
  Ethernet.begin(TCP_MAC, TCP_IP);

  
  countRx = 0;
  countTx = 0;
  countErr = 0;

  // Setup led pins
  pinMode(RADIO_RX_LED_PIN, OUTPUT);
  pinMode(RADIO_TX_LED_PIN, OUTPUT);
  pinMode(RADIO_ERROR_LED_PIN, OUTPUT);
  digitalWrite(RADIO_RX_LED_PIN, LOW);
  digitalWrite(RADIO_TX_LED_PIN, LOW);
  digitalWrite(RADIO_ERROR_LED_PIN, LOW);

 
  // Set initial state of leds
  digitalWrite(RADIO_RX_LED_PIN, HIGH);
  digitalWrite(RADIO_TX_LED_PIN, HIGH);
  digitalWrite(RADIO_ERROR_LED_PIN, HIGH);


  // Add led timer interrupt
  MsTimer2::set(300, ledTimersInterrupt);
  MsTimer2::start();


  // give the Ethernet interface a second to initialize
  delay(1000);


  // Initialize gateway at maximum PA level, channel 70 and callback for write operations 
  gw.begin(incomingMessage, 0, true, 0);  

  // start listening for clients
  server.begin();
  
  Serial.println("Ok!");
  Serial.println(TCP_IP);
}

void loop()
{
  gw.process();  
  
  processEthernetMessages();
}





inline MyMessage& build (MyMessage &msg, uint8_t destination, uint8_t sensor, uint8_t command, uint8_t type, bool enableAck) {
	msg.destination = destination;
	msg.sender = GATEWAY_ADDRESS;
	msg.sensor = sensor;
	msg.type = type;
	mSetCommand(msg,command);
	mSetRequestAck(msg,enableAck);
	mSetAck(msg,false);
	return msg;
}

char *getType(char *b, const char **index) {
	char *q = b;
	char *p = (char *)pgm_read_word(index);
	while (*q++ = pgm_read_byte(p++));
	*q=0;
	return b;
}

void processMQTTMessage(char *inputString, uint8_t inputPos) {
	char *str, *p;
	uint8_t i = 0;
	buffer[0]= 0;
	buffsize = 0;
	(void)inputPos;

	if ((uint8_t)inputString[0] >> 4 == MQTTCONNECT) {
		buffer[buffsize++] = MQTTCONNACK << 4;
		buffer[buffsize++] = 0x02;			// Remaining length
		buffer[buffsize++] = 0x00;			// Connection accepted
		buffer[buffsize++] = 0x00;			// Reserved
		MQTTClientConnected=true;			// We have connection!
	}
	if ((uint8_t)inputString[0] >> 4 == MQTTPINGREQ) {
		buffer[buffsize++] = MQTTPINGRESP << 4;
		buffer[buffsize++] = 0x00;
	}
	if ((uint8_t)inputString[0] >> 4 == MQTTSUBSCRIBE) {
		buffer[buffsize++] = MQTTSUBACK << 4;		// Just ack everything, we actually dont really care!
		buffer[buffsize++] = 0x03;			// Remaining length
		buffer[buffsize++] = (uint8_t)inputString[2];	// Message ID MSB
		buffer[buffsize++] = (uint8_t)inputString[3];	// Message ID LSB
		buffer[buffsize++] = MQTTQOS0;			// QOS level
	}
	if ((uint8_t)inputString[0] >> 4 == MQTTUNSUBSCRIBE) {
		buffer[buffsize++] = MQTTUNSUBACK << 4;
		buffer[buffsize++] = 0x02;			// Remaining length
		buffer[buffsize++] = (uint8_t)inputString[2];	// Message ID MSB
		buffer[buffsize++] = (uint8_t)inputString[3];	// Message ID LSB
	}
	if ((uint8_t)inputString[0] >> 4 == MQTTDISCONNECT) {
		MQTTClientConnected=false;			// We lost connection!
	}
	if (buffsize > 0) {
		writeEthernet(buffer,&buffsize);
	}

	// We publish everything we get, we dont care if its subscribed or not!
	if ((uint8_t)inputString[0] >> 4 == MQTTPUBLISH || (MQTT_SEND_SUBSCRIPTION && (uint8_t)inputString[0] >> 4 == MQTTSUBSCRIBE)) {
		buffer[0]= 0;
		buffsize = 0;
		// Cut out address and payload depending on message type.
		if ((uint8_t)inputString[0] >> 4 == MQTTSUBSCRIBE) {
			strncat(buffer,inputString+6,inputString[5]);
		} else {
			strncat(buffer,inputString+4,inputString[3]);
		}

#ifdef DEBUG
		Serial.println(buffer);
#endif
		// TODO: Check if we should send ack or not.
		for (str = strtok_r(buffer,"/",&p) ; str && i<4 ; str = strtok_r(NULL,"/",&p)) {
			if (i == 0) {
				if (strcmp_P(str,broker)!=0) {	//look for MQTT_BROKER_PREFIX
					return;			//Message not for us or malformatted!
				}
			} else if (i==1) {
				msg.destination = atoi(str);	//NodeID
			} else if (i==2) {
				msg.sensor = atoi(str);		//SensorID
			} else if (i==3) {
				unsigned char match=255;			//SensorType
#ifdef MQTT_TRANSLATE_TYPES				

				for (uint8_t j=0; strcpy_P(convBuf, (char*)pgm_read_word(&(vType[j]))) ; j++) {
					if (strcmp((char*)&str[2],convBuf)==0) { //Strip V_ and compare
						match=j;
						break;
					}
					if (j >= V_TOTAL)  break;
				}

#endif
                                if ( atoi(str)!=0 || (str[0]=='0' && str[1] =='\0') ) {
					match=atoi(str);
				}

				if (match==255) {
					match=V_UNKNOWN;
 				}
				msg.type = match;
			}
			i++;
		}						//Check if packge has payload
		if ((uint8_t)inputString[1] > (uint8_t)(inputString[3]+2) && !((uint8_t)inputString[0] >> 4 == MQTTSUBSCRIBE)) {
			strcpy(convBuf,inputString+(inputString[3]+4));
			msg.set(convBuf);			//Payload
		} else {
			msg.set("");				//No payload
		}
		txBlink(1);
		if (!gw.sendRoute(build(msg, msg.destination, msg.sensor, C_SET, msg.type, 0))) errBlink(1);

	}
}

void sendMQTT(const MyMessage &inMsg) {
        MyMessage msg = inMsg;
	buffsize = 0;
	if (!MQTTClientConnected) return;			//We have no connections - return
	if (msg.isAck()) {
//		if (msg.sender==255 && mGetCommand(msg)==C_INTERNAL && msg.type==I_ID_REQUEST) {
// TODO: sending ACK request on id_response fucks node up. doesn't work.
// The idea was to confirm id and save to EEPROM_LATEST_NODE_ADDRESS.
//  }
	} else {
		// we have to check every message if its a newly assigned id or not.
		// Ack on I_ID_RESPONSE does not work, and checking on C_PRESENTATION isn't reliable.
		uint8_t newNodeID = gw.loadState(EEPROM_LATEST_NODE_ADDRESS)+1;
		if (newNodeID <= MQTT_FIRST_SENSORID) newNodeID = MQTT_FIRST_SENSORID;
		if (msg.sender==newNodeID) {
			gw.saveState(EEPROM_LATEST_NODE_ADDRESS,newNodeID);
		}
		if (mGetCommand(msg)==C_INTERNAL) {
			if (msg.type==I_CONFIG) {
				txBlink(1);
				if (!gw.sendRoute(build(msg, msg.sender, 255, C_INTERNAL, I_CONFIG, 0).set(MQTT_UNIT))) errBlink(1);
				return;
			} else if (msg.type==I_ID_REQUEST && msg.sender==255) {
				uint8_t newNodeID = gw.loadState(EEPROM_LATEST_NODE_ADDRESS)+1;
				if (newNodeID <= MQTT_FIRST_SENSORID) newNodeID = MQTT_FIRST_SENSORID;
				if (newNodeID >= MQTT_LAST_SENSORID) return; // Sorry no more id's left :(
				txBlink(1);
				if (!gw.sendRoute(build(msg, msg.sender, 255, C_INTERNAL, I_ID_RESPONSE, 0).set(newNodeID))) errBlink(1);
				return;
			}
		}
		if (mGetCommand(msg)!=C_PRESENTATION) {
			if (mGetCommand(msg)==C_INTERNAL) msg.type=msg.type+(S_FIRSTCUSTOM-10);	//Special message
			buffer[buffsize++] = MQTTPUBLISH << 4;	// 0:
			buffer[buffsize++] = 0x09;		// 1: Remaining length with no payload, we'll set this later to correct value, buffsize -2
			buffer[buffsize++] = 0x00;		// 2: Length MSB (Remaing length can never exceed ff,so MSB must be 0!)
			buffer[buffsize++] = 0x08;		// 3: Length LSB (ADDR), We'll set this later
			strcpy_P(buffer+4, broker);
			buffsize+=strlen_P(broker);
#ifdef MQTT_TRANSLATE_TYPES
			if (msg.type > V_TOTAL) msg.type=V_UNKNOWN;// If type > defined types set to unknown.
 			buffsize+=sprintf(&buffer[buffsize],"/%i/%i/V_%s",msg.sender,msg.sensor,getType(convBuf, &vType[msg.type]));
#else
			buffsize+=sprintf(&buffer[buffsize],"/%i/%i/%i",msg.sender,msg.sensor,msg.type);
#endif
			buffer[3]=buffsize-4;			// Set correct address length on byte 4.
#ifdef DEBUG
			Serial.println((char*)&buffer[4]);
#endif
			msg.getString(convBuf);
			for (uint8_t a=0; a<strlen(convBuf); a++) {// Payload
				buffer[buffsize++] = convBuf[a];
			}
			buffer[1]=buffsize-2;			// Set correct Remaining length on byte 2.
			writeEthernet(buffer,&buffsize);
		}
	}
}


void ledTimersInterrupt() {
  if(countRx && countRx != 255) {
    // switch led on
    digitalWrite(RADIO_RX_LED_PIN, LOW);
  } else if(!countRx) {
     // switching off
     digitalWrite(RADIO_RX_LED_PIN, HIGH);
   }
   if(countRx != 255) { countRx--; }

  if(countTx && countTx != 255) {
    // switch led on
    digitalWrite(RADIO_TX_LED_PIN, LOW);
  } else if(!countTx) {
     // switching off
     digitalWrite(RADIO_TX_LED_PIN, HIGH);
   }
   if(countTx != 255) { countTx--; }

  if(countErr && countErr != 255) {
    // switch led on
    digitalWrite(RADIO_ERROR_LED_PIN, LOW);
  } else if(!countErr) {
     // switching off
     digitalWrite(RADIO_ERROR_LED_PIN, HIGH);
   }
   if(countErr != 255) { countErr--; }

}

void rxBlink(uint8_t cnt) {
  if(countRx == 255) { countRx = cnt; }
}
void txBlink(uint8_t cnt) {
  if(countTx == 255) { countTx = cnt; }
}
void errBlink(uint8_t cnt) {
  if(countErr == 255) { countErr = cnt; }
}

Also I must mention I am using a ENC28J60 chip on my Ethernet module. Could that be the problem of not answering to pings?

edit: Ah, sure, got the network problem:

//#include <Ethernet.h>
#include <UIPEthernet.h>

But still, remote_ip is not in the code I copied from your github library. Maybe you can post the correct code here!?

edit:
Ok, finally got it. I must admit I am not good in using Github, it´s very new to me...
Question: Do I always have to download the complete branch and overwrite my arduino file with it or ist it enough to only download your MQTTClientGateway example? Just for me to understand this...

Of course, when using the UIPEthernet library, the sketch is too big now...Is there a workaround?

[tomkxy]

@siod Sorry, i don't know whether that makes sense here. I do not know where this code came from.
I pointed you to the the contribution I made which is here: https://github.com/tomkxy/Arduino.git
in the MQTTClient branch (follow the link, select the branch mqttclient and click through the directories: libraries/MySensors/examples/MQTTClientGateway/ ).

As I wrote before the current development branch of the official MySensors library contains a cleaned up and re-factored version I didn't test and can say nothing about.

[siod]

I found a W5100 Ethernet Module et voila, everything works fine now!!! Thanks for your help @tomkxy !!

[tomkxy]

@siod If it works now you probably can give it a try with the latest version in the development branch of the Mysensor library.

[ericsko]

@Yveaux said:

And another small one; an RGB dimmer!

I tried this 3-dimmer approach, but I found it not responsive enough. Sometimes the arduino can't be fast enough to process 3 consecutive requests, and the result is lame. (it changes 2 of 3 colors only)
I think this would be more efficient with transmitting just one cummulative message with all 3 colors and decoding it in arduino.

[Yveaux]

@ericsko sounds to me that you're missing a message instead of the Arduino being too 'slow'.
But go ahead, use a single message to send the rgb value and post your findings in here!