Serial Gateway and MQTT Broker

[Diego Baravalle]

Hi everybody!

I need to setup two or more "controllers" and a requeriment is to use serial gateway (Arduino as serial gateway connected to raspberry).

Is it possible?

Can I setup two controllers listening to same serial port?
Can I use MyController configured with serial gateway and publish all messages to MQTT?
Is there another alternative?

Thanks!

[jkandasa]

@Diego-Baravalle for now MyController will not push serial gateway messages to MQTT

[barduino]

@Diego-Baravalle

In theory you could use nodeJS to read from serial and push to MQTT.

From there you should be able to have 2 MQTT enabled controllers subscribing/publishing to broker.

That should be an interesting experiment

Cheers

[Diego Baravalle]

@jkandasa, @barduino

Thanks for your answers.

Use NodeJS is an interesting option.

[Frank C]

@jkandasa is there a future feature to push serial-gateway messages with MyController's build in MQTT? It would a great feature.

[jkandasa]

@Frank-C For now no plan

[gohan]

@Frank-C
It would be quite a redundant function because you can publish directly from the gateway if you use an ethernet shield, but it's not your case if I understood right

[jkandasa]

@Frank-C But I can write simple library to run independently to convert serial gateway messages to MQTT type. If you want.

[Frank C]

@jkandasa That would be fantastic if you could ? I think it would be a great feature

[jkandasa]

@Frank-C Yes, I can. but you have to run it as separate service. I will create a separate project for this work. I am not going to include this on MyController project.

[Frank C]

@jkandasa As a seperate service is okay, it would be very helpful. Thank you

[Diego Baravalle]

@jkandasa Perfect!

[FotoFieber]

Maybe this is enough for your requirement:
https://forum.mysensors.org/topic/4599/how-to-add-a-serial-device-to-use-in-node-red/10

[Frank C]

@FotoFieber Thanks for the suggestion, I got that working. I have had to change MyController to read an MQTT gateway. Problem now is MyController can't read the MQTT format of this script. Any suggestions or tips ?
MyController does such a great job with handling MySensors' data, it would be a pity to loose that functionality.

[FotoFieber]

@Frank-C
The script uses the serial protocol and should be compatible with the mysesnosrs mqtt-gateway. Are the pub/sub-topics set correct?

[Frank C]

@FotoFieber MyController is getting the MQTT msg, but doesn't like it's format, i've had continuous error msg like below:
2017-02-16 18:17:34,657 ERROR [Thread-4] [org.mycontroller.standalone.provider.mysensors.MySensorsProviderBridge:60] Unable to process this rawMessage:RawMessage(gatewayId=2, data=2;0;1;0;23;36, subData=mysensors/1/out/, isTxMessage=false, networkType=MY_SENSORS)
org.mycontroller.standalone.message.RawMessageException: Unknown message format, Topic:mysensors/1/out/, PayLoad:2;0;1;0;23;36
at org.mycontroller.standalone.provider.mysensors.MySensorsRawMessage.updateMQTTMessage(MySensorsRawMessage.java:114) ~[mycontroller-dist-standalone-0.0.3.Alpha2-single.jar:0.0.3.Alpha2]

MyController is set up with the correct topic, as seen in above Topic:/mysensors/1/out/ (i've also tried without the ending /).

[FotoFieber]

2;0;1;0;23;36 Is a correct message according to the serial API. I don't know anything about your controller...sorry

Maybe the controller needs the ; translated to / in the topic?

Have you tried to switch pub/sub topic?

[jkandasa]

@Frank-C Current approach of MyController on MQTT see only payload on message. Other data(nodeId, sensorIds, etc) from topic. Kindly create a request on MyController issue page. I will address this. If we address this I am not going to create separate service to parse serial-to-real-mqtt service type.

[Frank C]

@jkandasa Thanks for the follow up. I can see MyController is modifying the serial message into a customized MQTT msg (as you have said).
MyController works great with the serial so it would be good to keep that as is, and if we had an independent service to publish to a separate MQTT broker would be very helpful?

[jkandasa]

@Frank-C cool, we will go with independent service.

[Frank C]

@jkandasa

[talhatec]

@jkandasa has the independent service been created? I would really love to see this feature. Im using Home Assistant for overall home automation. But I want to manage MySensors with MyController due to the better features to handle MySensor Nodes.

[jkandasa]

@talhatec Still I am working on this feature. I will update you once it gets ready.

[jkandasa]

@talhatec Thank you for your patience
Finally the first version of serial2mqtt adapter available and released for public. Kindly report issues if you find any on issues page

[Diego Baravalle]

@jkandasa Thanks very very much!

Awesome work!

[talhatec]

@jkandasa Thank you very much, will give it a go over the weekend.

[talhatec]

@jkandasa The serial2MQTT adapter worked for a while, but then it started to fail. Looking at the logs, its not receiving the data from serial properly

2017-08-07 18:04:00,942 ERROR [Thread-1] [org.mycontroller.serial2mqtt.gateway.MessageMonitorThread:66] Exception on processing [RawMessage(gatewayId=null, data=%ﾍﾲￌￒ￲▒ￍﾞﾉ<*▒oﾰﾲￋﾛ￤▒0;255;3;0;9;TSF:MSG:READ,254-254-0,s=255,c=3,t=1,pt=0,l=0,sg=0:, subData=null, isTxMessage=false, networkType=MY_SENSORS, timestamp=1502125439421)],
java.lang.NumberFormatException: For input string: "%ﾍﾲￌￒ￲▒ￍﾞﾉ<*▒oﾰﾲￋﾛ￤▒0"
        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
        at java.lang.Integer.parseInt(Integer.java:569)
        at java.lang.Integer.valueOf(Integer.java:766)
        at org.mycontroller.serial2mqtt.gateway.MessageEngine.postOnMqtt(MessageEngine.java:53)
        at org.mycontroller.serial2mqtt.gateway.MessageMonitorThread.processRawMessage(MessageMonitorThread.java:63)
        at org.mycontroller.serial2mqtt.gateway.MessageMonitorThread.run(MessageMonitorThread.java:85)
        at java.lang.Thread.run(Thread.java:745)

[talhatec]

@jkandasa Not sure what the issue is, but the adapter seems to be working fine now. Ill leave it running for a day and see if the data issue reappears. I was trying to get HomeAssistant and myController to work off the same MQTT topics. maybe thats what cause the data issue.

[jkandasa]

@talhatec Are you seeing this error often?

[talhatec]

@jkandasa It seems to be stable now. Think that it was a one off issue. Ive now got both HomeAssistant and MyController working on the same MQTT in/out topics.

However, when a sensor calls back to the controller, both HomeAssistant and myController send out replies, and due to this, i think the messages clash, and the controllers endup sending it multiple times.

Have to think of a way to make only one respond to requests.

[jkandasa]

@talhatec

However, when a sensor calls back to the controller, both HomeAssistant and myController send out replies, and due to this, i think the messages clash, and the controllers endup sending it multiple times.

I do not know your preference about, which controller wants to response for your sensors request. Suppose you want MyController only on listen mode, you have to change MQTT Topic publish to some dummy topic on MyController gateway settings.