Note: edited 2017-01-22. Key format on Zabbix Items has been modified to fit Auto-Discovery and automatic creation of sensors and nodes. Item-key = child-id. See part 2.
I wanted to evaluate Zabbix as a Controller, installing it on my Raspberry Pi and get it to work together with MySensors. Here's my experience.
- Introduction
Zabbix is a very powerful Open Source monitoring system used to monitor e.g. IT or Telecom Data Centers and Data Networks. It comes with a variety of functions, it's very flexible, has a strong Forum and a lot of documentation. It can be integrated via zabbix.api, zabbix-sender or itâs Low Level Discovery protocol zabbix LLD, among other stuff. There are several software libraries developed for a variety of program languages for integrations with external systems and software. Take a look at the web pages www.zabbix.org and www.zabbix.com.
So far Zabbix has been running smoothly on my Raspberry Pi 2 together with my MySensors integration.
I have it running with an MQTT broker, MQTT Gateway and some sensors. In fact, itâs running in parallel with a Domoticz Controller, both registering the same data from my sensors, installed on the same Raspberry.
- Here are a couple of screens from my setup, showing temperature and energy sensors in operation.



- Zabbix comes with a Plug and Play Linux Servers monitoring setup via preinstalled templates, like the Raspberry Pi. It sends alerts automatically if thereâs some issue. Here's a screen example from my setup.

- Setup procedure. MQTT Broker
First the Raspberry needs to have an MQTT broker running to capture and distribute MQTT messages, from a MySensors MQTT gateway client in this case. Mosquitto I guess is the best choice.
Follow the instructions on this page to install the MQTT Mosquitto Broker:
http://www.instructables.com/id/Installing-MQTT-BrokerMosquitto-on-Raspberry-Pi/
Or run directly with the following commands on your Raspberry Pi:
sudo apt-get update
sudo apt-get install mosquitto
sudo apt-get install mosquitto-clients
- Instructions on how to install zabbix on the Raspberry Pi, using MySQL as database
Zabbix needs a database to run. MySQL is selected as database in this setup.
There are two packages needed from Zabbix to get it working. zabbix-server-mysql and zabbix-frontend-php. As well itâs recommended you install zabbix-agent on the Raspberry. Zabbix can then monitor the Raspberry Pi performance automatically through the zabbix-agent tpgether with pre-installed templates.
If you install Zabbix directly via "apt-get install" the version you get is rather old (zabbix 2.x), and I got into some trouble trying to get it to work. The web page further down did work straight forward in my case. It installs zabbix v3.0. Have a look at the zabbix 3.0 manual: https://www.zabbix.com/documentation/3.0/manual
If you don't have mysql installed. Do this first on the Raspberry:
sudo apt-get install mysql-server mysql-client
- Installing zabbix 3.0
Follow the page, step by step, to install Zabbix on your Raspberry Pi. It installs the version zabbix 3.0:
http://devopsish.blogspot.com/2016/05/installing-zabbix-3-on-raspberry-pi.html
You should now have got installed
-
zabbix-server-mysql
-
zabbix-frontend-php
-
zabbix-agent
on your Raspberry Pi. You can now access Zabbix from a web browser pointing at the Raspberry IP address. Eg. http://192.168.1.70/zabbix
- You should get to the login screen.

Enter with User = Admin, Password = zabbix. Later on you can add and modify other users for access control.
- Setup zabbix-sender
Additionally weâll need to install the package zabbix-sender to be able to send the MQTT messages from a script to the Zabbix Server.
On your Raspberry Pi enter:
cd ~
cd zabbix3-rpi-master
sudo dpkg âi zabbix-sender_3.0.*-1+jessie_armhf.deb
- Zabbix setup concepts
To pass MQTT messages from the MQTT broker to Zabbix I wrote a small Python script. Please note, the MQTT format with MySensors is "topic/a/b/c/d/e", but Zabbix doesn't allow the letter "/" in an âitem keyâ, it's therefore changed in the Python script to the format "topic.a.b.c.d.e".
In Zabbix I have setup a hostgroup named âMySensorsâ. Added to the hostgroup are the sensor nodes (hosts). To the hosts are added the sensor children (items) following the MySensors MQTT format:
MY_MQTT_TOPIC_PREFIX/SENSOR-NODE-ID/SENSOR-CHILD-ID/CMD-TYPE/ACK-FLAG/SUB-TYPE
-
hostgroup = "MySensorsâ
-
host = MY_MQTT_TOPIC_PREFIX + "." + MY_NODE_ID
eg. mygateway1-out.5
-
item = CHILD_ID + "." + CMD-TYPE + "." + "ACK-FLAG" + "." + SUB-TYPE
eg. 1.1.0.0 (Child Id = 1, Cmdtype = set, Ack = 0, Subtype = V_TEMP)
The Zabbix items should be set to the type âzabbix trapperâ. You find the items at -configuration-, -host-, -item-. Zabbix trapper listens to messages sent from âzabbix-senderâ, which weâre going to use here.
Be aware. No automatic node assignment to the Sensor Nodes from the Controller would work with Zabbix in this setup.
So we need to preset the Node_ID in the MySensor arduino scripts e.g.:
#define MY_NODE_ID 1
- Zabbix setup
- Define a hostgroup and give it a name.

- Define some hosts (sensor nodes). The host name must have an exact match with your mqtt gateway topic name and node id separated with a dot (MY_MQTT_TOPIC_PREFIX + "." + MY_NODE_ID)
In this example I'm using âdomoticz.in.MyMQTT.11â with the node = 11.

- Define some items (children) to your sensor nodes
Give them a name of your choice. Select Type = Zabbix trapper. The key for each item is the Child-Id taken from the MySensors MQTT message. In this case Child-ID = 1.
Note: edited 2017-01.22. Key for items has been modified to only use Child-id, to fit integration of Auto-Discovery. See part 2.

- Zabbix-sender
There is an executable thatâs now available âzabbix_senderâ that you can use to test directly from the command line in the Raspberry. Try sending to an item thatâs been previously setup in Zabbix by executing the command:
zabbix_sender -v -z localhost -s domoticz.in.MyMQTT.11 -k 1 -o 10.5
A response should look something like:
info from server: "processed: 1; failed: 0; total: 1; seconds spent: 0.000176"
sent: 1; skipped: 0; total: 1
If failed = 0, sent = 1 everything should be ok in the setup.
You should now see the message show up in Zabbix. Look it up in Zabbix, in âMonitoringâ and select âLatest Dataâ. You should see an event with temperature 10.5 °C registered in the node = 11, Child = 1. That is, hostname = âdomoticz.in.MyMQTT.11â with key item = â1â and with the payload = 10.5 received.
Note: edited 2017-01-22. Key format modified to key=child-id to fit Auto-Discovery. Check on part 2.
- Python script
The Python script uses two libraries that we have to download first:
Mosquito client library. See: www.mosquitto.org and https://pypi.python.org/pypi/paho-mqtt
pip install paho-mqtt
Library that uses zabbix-sender / zabbix-api See: https://www.zabbix.org/wiki/Docs/api/libraries
And: https://github.com/blacked/py-zabbix
pip install py-zabbix
Prepare a script named e.g. âmqtt_zabbix.pyâ, with your favorite editor, like vi or nano.
nano mqtt_zabbix.py
Paste the Python script below and save the file. Run for example the script in the background, as a daemon, from the command line. The script stores the stdout and stderr printouts in the text file âzabbix.logâ:
python -u mqtt_zabbix.py > zabbix.log 2>&1 &
The file zabbix.log stores the sent MQTT events, with a time stamp and the result received from zabbix-sender. Useful for debugging.
Python script:
import paho.mqtt.client as mqtt
import time
from pyzabbix import ZabbixMetric, ZabbixSender
topic_sub = "domoticz/in/MyMQTT/"
# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
print("Connected with result code "+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe(topic_sub+"+/+/+/+/+")
# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
msg.topic = msg.topic.replace("/", ".")
myNode = msg.topic[:msg.topic.find(".", len(topic_sub)+1)]
myItem = msg.topic[len(myNode)+1:]
mySplit = myItem.split('.') # split topic to indexed list
if mySplit[1] == '1': # type = set
myItem = mySplit[0] # change item to child-id
metrics = [ZabbixMetric(myNode, myItem, msg.payload)]
result = ZabbixSender(use_config=True).send(metrics)
print(time.strftime("%c")+" "+str(result)+" : "+myNode+" "+myItem+" "+str(msg.payload))
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect("localhost", 1883, 60)
In the script, to change the MQTT format from topic/a/b/c/d/e to topic.a.b.c.d.e, required by zabbix, this Python line fixes it:
msg.topic = msg.topic.replace("/", ".")
These two lines make the deal to send the MQTT events to Zabbix:
metrics = [ZabbixMetric(myNode, myItem, msg.payload)]
result = ZabbixSender(use_config=True).send(metrics)
Itâs possible to add triggers to the items in Zabbix, to generate alerts from MQTT sensor events received. Alerts can as well be sent by e.g. SMS, Email or script (outside this scope).
My next steps will be to create sensor nodes automatically (hosts) and itâs children (items), by preparing some Templates in Zabbix together with zabbix.api or zabbix LLD. And as well have a look into how to send messages from Zabbix to MySensor relays and actuators, and for autonomous flow actions.
Enjoy and have fun!