'production' setup - how do you guys do it?

[Kurt]

Hi all,
How do you guys build your 'production' setup? What I mean is a setup that can run 24/7, headless, log all sensor events, maintainable.

I've been playing around with gateway sketches & sensor nodes. I've been able to log events in a database using the MQTTsketch and the Node.js script. I've connected MYSController to a gateway sketch ... but still I fail to see how I can combine all these applications while connecting to a single gateway/sensor-network.

I have feeling either I'm missing something or there's still a sort of central component of MYSensors missing:

• MYSController I love because it gives an overview of network & network events. I can also be used to upload new FW, great ...
• MQTT gives great possibilities to connect to a 'controller' application such as OpenHab, Domoticz, ... (I find the name 'controller' confusing here)
• ? to log all the sensor events in a database and pull graphs from them (could also be done by something like OpenHab but prefer separate tool here)

How do you combine these tools on a single headless PC without need to 'unplug' one application before being able to connect the next application.

I hope my drawing explains what I'm looking for. Need something in the middle where the question mark is?
Any hints or am I missing the obvious?

Kurt

[ahhk]

I think its the best thing, to move the line between "?" and "db" to "Openhab" and "db"...
One central entrance-point for all values.

I believe, that it does not exist something you mean with your "?"...it makes the system unnecessary complicated.

Mysensors => MQTT-Clientgateway => MQTT-host (Mosquitto) => Openhab runs great in my house...

[Yveaux]

@Kurt In my setup MQTT is in the place of your question mark, but I'm not using myscontroller.
MQTT data is converted, where applicable, using NodeRed, and it also stores the data in an InfluxDB database (way better for storing time oriented data then any other database I know).
Graphing is done by Grafana (Simply awesome! Seamless integration with InfluxDB).
MQTT gateway, NodeRed, InfluxDB, Grafana and OpenHab all run on a single Linux server.

Apart from OpenHab (which I'm not too happy with) I finally seem to have found a great solution that covers all my requirements!

[Fabien]

@Yveaux : same setup here on ODROID C1, with last influxDB and grafana. Works fine except opnehab limitation. I test domoticz and jeedom but it's not better.
Can you share some of your nodered flux ? Especialy for InfluxDB ...

[Yveaux]

@Fabien said:

Can you share some of your nodered flux ? Especialy for InfluxDB ...

I just use some function using curl with the http api of InfluxDB to store the data. I had problems using an Influx node I found on the net.
I'll share it later on, when I'm at my pc.

[Yveaux]

@Fabien Ok, here's the flow to store a single MQTT value in an InfluxDB:

[{"id":"fb1ad45d.04e528","type":"mqtt-broker","broker":"localhost","port":"1883","clientid":""},{"id":"2ad0959b.d52f6a","type":"mqtt in","name":"Keuken_Temp","topic":"/mySensors/126/0/V_TEMP","broker":"fb1ad45d.04e528","x":113,"y":96,"z":"5c2b04bf.a3d4fc","wires":[["3330671f.cccf98"]]},{"id":"27a5b713.d85a48","type":"exec","command":"curl -i -XPOST 'http://localhost:8086/write?db=test' --data-binary ","addpay":true,"append":"","useSpawn":"","name":"Wite InfluxDB","x":677,"y":83,"z":"5c2b04bf.a3d4fc","wires":[[],[],[]]},{"id":"725ce47c.8da31c","type":"function","name":"","func":"var value = msg.payload;\nmsg.payload = \"'temp,location=keuken value=\"+value+\"'\";\nreturn msg;","outputs":1,"noerr":0,"x":502,"y":87,"z":"5c2b04bf.a3d4fc","wires":[["27a5b713.d85a48"]]},{"id":"3330671f.cccf98","type":"rbe","name":"","func":"rbe","gap":"","x":298,"y":94,"z":"5c2b04bf.a3d4fc","wires":[["725ce47c.8da31c"]]}]

[Fabien]

Thank you I will test soon. Is it for the last version (0.9) of influxdb ?

[Yveaux]

@Fabien yes

[bpair]

I have been struggling with this as well. Have you played with openHAB2? If you are using NodeRed do you still need openHAB? The charting seems weak in openHAB so i will look at Grafana.

Does the MQTT interface provide everything the serial or ethernet does?

The last feature i am looking for is an easy way to create forms for manually entered data. Unfortunately I have some data that must still be manually collected and i want an easy UI form to enter it.

[bpair]

Also is it necessary to run mosquitto?

[Yveaux]

@bpair said:

Have you played with openHAB2?

No

If you are using NodeRed do you still need openHAB?

Depends. NodeRed is not really meant as a user interface for home automation. You can toggle some inputs etc. for development, but you probably don't want your familiy to wander around in NodeRed.

[Yveaux]

@bpair said:

Also is it necessary to run mosquitto?

If you're using an MQTT client gateway (like my Perl MQTT gateway which runs on Linux and requires a regular Ethernet gateway) you will also need an MQTT Broker (e.g. Mosquitto) to exchange data with.
When using the MQTT broker gateway (like @Damme 's implementation, included in MySensors library examples) the gateway itself will be a broker (running on an Arduino) and clients can directly connect to it. Another option is to have the Broker gateway mirror its messages on another Broker (e.g. Mosquitto).
It's a very flexible architecture and it depends on your infrastructure and personal preference what direction to go.

I personally prefer my client gateway
My Linux server is more than capable of handling the MQTT and conversion. I originally started off with an Arduino Client impementation but found it too limited to do the conversion to my liking.
I need a Linux server anyhow for OpenHab/NodeRed/InfluxDB/Grafana/etc so it's imperative to also run the MQTT gateway client on the same system.

[bpair]

I really like the concept of using MQTT and node-red however my setup is in a school greenhouse without internet or a router. I will have one older computer running linux with a serial gateway. Is it possible to use your perl MQTT with a serial connection?

If not maybe I just spend the $25 and get a cheap router.

[Yveaux]

@bpair said:

Is it possible to use your perl MQTT with a serial connection?

Some users of the gateway added serial support way back, but I'm afraid it might not be functioning very well at the moment as I don't use serial support and therefore won't test it.

[Fabien]

@Yveaux For your influx flow you can try this :

[{"id":"8347dab9.7cb828","type":"mqtt-broker","broker":"localhost","port":"1883","clientid":""},{"id":"d92d398d.26d2c8","type":"mqtt in","name":"Temperature Bureau","topic":"MySensors/4/1","broker":"8347dab9.7cb828","x":159,"y":103,"z":"183ecd0f.e7c133","wires":[["a929fd06.56d6"]]},{"id":"a929fd06.56d6","type":"function","name":"","func":"var newMsg={};\nnewMsg.payload = new Buffer(\"temp,location=Bureau value=\"+msg.payload,\"binary\");\nreturn newMsg;","outputs":1,"noerr":0,"x":354,"y":103,"z":"183ecd0f.e7c133","wires":[["4b2e850f.b4d17c"]]},{"id":"4b2e850f.b4d17c","type":"http request","name":"","method":"POST","ret":"bin","url":"http://localhost:8086/write?db=test","x":505,"y":103,"z":"183ecd0f.e7c133","wires":[[]]}]