You´re absolutely right.
After 4+ hours debugging the MQTT gateway + ~10h trying to get MySensors MQTT on ethernet working properly, I tried a different way:
installed NoteRed on the RPi and an asynchronous MQTT client on the ESP32, and within ~1 hour I had a fast and stable way to send messages from and to my nodes. Adding JSON parser on the ESP, and I can perfectly communicate with dummy Domoticz devices.
Also, except of you and one very helpful guy within a private chat, noone preplied to MQTT-questions and ESP-stuff. Seems working with ESPs is reluctantly seen here on the forum.?
So I assume a different way then using MySensors for everything except (battery-powered) Arduinos is the best I can do, to also have no problems in the future..
Posts made by Inso
RE: 💬 Building a WiFi Gateway using ESP8266
RE: 💬 Building a WiFi Gateway using ESP8266
If I only would set up 8266´s I´d give it a try - problem is, I´m in love with the ESP 32.
Bluetooth LE, multithreading, a second core only for the wifi stuff so no crashes on high wlan activity and no problems with time-critical events, lots of GPIO, awesome processor power, combined with the lower power consumption makes this the most-used chip here
As far as I can see, both solutions you mentioned are not made for the ESP32 .
So I´d like to stick to MySensors as I could use lots of the sketches almost 1:1 (can´t be that hard to get more then one MQTT-"Client" running -.-).
Only real other options I see is using MQTT-Clientsoftware from Basecamp or just set up an asynchronous http socket and code / decode the JSON-parameters send from and to Domoticz myself..
RE: 💬 Building a WiFi Gateway using ESP8266
"Don't define MY_RADIO_NRF24 and you're good to go. It will still be a gateway, but a gateway with local sensors and no radio."
"On one MySensors network there can only be one MySensors gateway. But on one IP network, there can be many MySensors gateways. Exactly how many depends on geography and chosen transports but at the very least a few hundred. Thousands if you really need and plan accordingly."
Let´s say I have multiple ESPs, which should all act like a sensor node. I have a RPi w Domoticz, Mosquitto and MySensors MQTT ethernet gateway.
I add the MQTT gateway to Domotics, listening on port 1883 (Mosquitto port). Now I set up a sketch, using
#define MY_IP_ADDRESS 192,168,178,151 #define MY_CONTROLLER_IP_ADDRESS 192,168,178,200 #define MY_PORT 1883
which sets the gateway to client / node mode (as read on the documentation).
Setting a node Id is ignored, ID is 0 and can´t be changed, so it is recognized as a gateway at domoticz.
If I now set a second node to a different IP address and name, it overwrites the old one (as you say, there can be only one gateway).
The question now is: how can I set up another ESP? Do I need another RPi or virtual machine with different IP address to install another gateway?
RE: Switching from ethernet gateway to MQTT
Update: rebuild from the bones. Raspi-OS, Domoticz, Mosquitto, MySensors.
I don´t get a version of my gateway nether.
Sensors run but have to be renamed complete within Domotics (database backup did not help), and all data is lost.
Switching from ethernet gateway to MQTT
I want to switch from an ethernet- to an MQTT-gateway, to be able to add some ESP8266 and ESP32 "natively", i.e. without extra nRF24-modules.
From all I´ve read, my "old" gateway (ethernet) uses the nRF24-module of my Raspi to communicate with the nodes. MySensors handels the messaging stuff, talks to Domoticz using the easy-to-use LAN interface.
Switching from ethernet to MQTT means I have the option to send also data per MQTT to and from the MS-gateway, besides the "usual nRF24-way". Only difference is to setup a MQTT gateway instead of the ethernet-gateway within domotics and to install a Mosquitto Broker (Server). Now the communication Domoticz <-> MySensors uses
MQTTthe usual ethernet gateway, nodes can connect the "usual way" using nRF24 or communicate using MQTT. MySensors uses the Mosquitto server for some implementation(?).Communication is then node <-> Mosquitto Broker <-> MySensors <-> Domoticz (?)
Sadly, the Domoticz tutorial is very unclear about that , it just said install Mosquitto, you need it. Also the Build-section here did not help much, and reading here on the forum did confuse more then it helped.
However, I managed it to let Domoticz and MySensors connect fine.
Nov 30 02:16:29 INFO Starting gateway... Nov 30 02:16:29 INFO Protocol version - 2.3.0 Nov 30 02:16:29 DEBUG MCO:BGN:INIT GW,CP=RNNGL---,VER=2.3.0 Nov 30 02:16:29 DEBUG TSF:LRT:OK Nov 30 02:16:29 DEBUG TSM:INIT Nov 30 02:16:29 DEBUG TSF:WUR:MS=0 Nov 30 02:16:29 DEBUG TSM:INIT:TSP OK Nov 30 02:16:29 DEBUG TSM:INIT:GW MODE Nov 30 02:16:29 DEBUG TSM:READY:ID=0,PAR=0,DIS=0 Nov 30 02:16:29 DEBUG MCO:REG:NOT NEEDED Nov 30 02:16:29 DEBUG MCO:BGN:STP Nov 30 02:16:29 DEBUG MCO:BGN:INIT OK,TSP=1 Nov 30 02:16:29 DEBUG GWT:RMQ:MQTT RECONNECT Nov 30 02:16:29 DEBUG connected to 127.0.0.1 Nov 30 02:16:29 DEBUG GWT:RMQ:MQTT CONNECTED Nov 30 02:16:29 DEBUG GWT:TPS:TOPIC=domoticz/in/MyMQTT/0/255/0/0/18,MSG SENT Nov 30 02:16:36 DEBUG TSF:MSG:READ,51-51-0,s=2,c=1,t=37,pt=3,l=2,sg=0:5 Nov 30 02:16:36 DEBUG GWT:TPS:TOPIC=domoticz/in/MyMQTT/51/2/1/0/37,MSG SENT Nov 30 02:16:56 DEBUG GWT:IMQ:TOPIC=domoticz/out/MyMQTT/0/0/3/0/18, MSG RECEIVED Nov 30 02:16:56 DEBUG GWT:TPS:TOPIC=domoticz/in/MyMQTT/0/255/3/0/22,MSG SENT
Problems starting with having no Version on hardware-tab (MySensors Gateway with MQTT interface
Version: ?), and the type of my "old" nodes is not recognized. I see the Node IDs I gave em, children type is unknown, values (f.e. #1.V_TEMP (24.3)) are received, but do not show up at the devices tab.
Do I have to rewrite all the Nodes to now use MQTT? As the nRF-nodes have no ethernet-interface, I only could add MY_MQTT_PUBLISH_TOPIC_PREFIX and MY_MQTT_SUBSCRIBE_TOPIC_PREFIX.
Sorry if I am asking noobish questions here, I´m only searching for a good way to let Arduinos and ESPs work side by side fine while using the hardware they provide.
From all I´ve read, a MQTT gateway is the best way (better then connect every ESP as a separate gateway). Or maybe I have missed something?
RE: 💬 Building a Raspberry Pi Gateway
make install before testing did the trick, now it is 2.3.0 .
Was just going step by step through tutorial, there it´s make -> test -> make install. Didn´t realized it would use old parts if it´s not first install
thought stop, disable and remove the service would be enough to ensure the service is completely "uninstalled" - could you give me a hint what I´ve missed?
RE: 💬 Building a Raspberry Pi Gateway
I am currently trying to update to 2.3
Have stopped service and deleted the file (systemctl status mysgw -> Unit mysgw.service could not be found.)
Also renamed the config file /etc/mysensors.dat
Renamed the old folder, then copied new files.
Checked readme.md in MySensors-folder: MySensors Library v2.3.0
Then I configured the radio and -> make, no errors
However, changing to bin folder and executing mysgw shows: Protocol 2.2.0
What am I missing?
RE: 💬 Battery Powered Sensors
To answer my own question: yes, it seems to work. Searched around and read a while, found this:
Best addition imho ,no need for step up / down.
RE: MySensors needs restart to communicate
Thanks for your reply
It seems it is the antenna on intialisation. I have switched form NRF24 with antenna as print on PCB to the ones with external antenna, plan was to have more range. Always supported by a 470μF condensator.
If I put my finger on top of the antenna while it initializes, I have a stable connection without restart of the gateway. From there on, I do not have to touch it anymore, communication runs perfect. If I do not touch the antenna, I have only errors until I restart the gateway on the RPI (restarting the node does not have an effect).
Btw , antennas were 20cm, then 3m away from each other, nothing in between
MySensors needs restart to communicate
I have updated MySensors to 2.2.0 on my RPi two weeks ago. Clean new install. If I try to connect from a node, I get error messages, as long as I do not restart the MySensors gateway. MySensors Libary on Arduino IDE is also 2.2.0, all sketches are updated.
2125 TSF:MSG:SEND,20-20-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 4133 !TSM:FPAR:NO REPLY 4135 TSM:FPAR
On the RPi MySensors runs atm in debug mode - I can not see any incoming communication. As soon as mysgw is restarted (kill process, then mysgw -d), mostly the node gets the connection and runs fine (without restart). As soon as I restart the node (soft reset or sketch update), there´s again no way it connects.
Tested with several notes and sketches, also example sketches from the page.
How to send serial commands (v_var) from Domoticz to serial gateway
I´m working on this for days now, with no conclusion.
I´d like to send commands to my Arduino Gateway from Domotics, using my DzVents script.
As the commands won´t fit predefinded types, I´d like to use the v_var1 - 5 (as shown on the list on https://www.mysensors.org/download/serial_api_20#set,-req)
I found the json url commands
and also an example
(on https://forum.mysensors.org/topic/4933/using-the-selector-switch-in-domoticz-tutorial/2 ) :
Set f.e. to a virtual dimmer switch, I can now read the command on the Arduino sketch
message.type == V_PERCENTAGE
and get the value sended.
My problem is, I can´t find out why "param=switchlight" on Domotics results in "V_PERCENTAGE" on MySensors. And what I have to send to get the "V_VAR1".
Mayber there is also an easier way to send commands to MySensors - maybe send serial commands directly?
Gateway (and also Domoticz server) is an RPi3.
RE: A few noob questions
Finally it works, thank you very much for your fast and detailed help!
If someone else has the same problem : I exchanged the capacitor (had two old ones without number on it, now 47uf best China quality :D) which solved the problem. Seems with the old ones the voltage collapsed before sending all data, so ping worked, communication didn't. I btw need the capacitor also to get usable lux values from my bh1750, without its always the maximum lux value.
RE: A few noob questions
Have changed a screenshot. The Arduino is recognized as Repeater, however, the motion sensor does not show up. Arduino is running code from
Connecting to serial port of the Arduino keeps showing:
18146892 TSF:MSG:SEND,255-255-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 18148900 !TSM:FPAR:NO REPLY 18148902 TSM:FPAR 18148905 TSF:MSG:SEND,255-255-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 18150914 !TSM:FPAR:NO REPLY 18150916 TSM:FPAR 18150919 TSF:MSG:SEND,255-255-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 18152927 !TSM:FPAR:NO REPLY 18152929 TSM:FPAR 18152932 TSF:MSG:SEND,255-255-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 18154940 !TSM:FPAR:FAIL 18154942 TSM:FAIL:CNT=7 18154944 TSM:FAIL:PDT
should occur every time the sensor is triggered. Does not show up, all I see is the quoted stuff. As soon as I run a simple trigger script on the Uno, I see the sensor works perfectly.
copy/paste libaries of BT1750 did the trick, compiling now works. However, I have the same problem as with the light sensor:
176452 TSM:FAIL:RE-INIT 176454 TSM:INIT 176461 TSM:INIT:TSP OK 176463 TSM:FPAR 176466 TSF:MSG:SEND,255-255-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 178475 !TSM:FPAR:NO REPLY 178477 TSM:FPAR 178480 TSF:MSG:SEND,255-255-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 180487 !TSM:FPAR:NO REPLY 180489 TSM:FPAR 180492 TSF:MSG:SEND,255-255-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 182499 !TSM:FPAR:NO REPLY 182501 TSM:FPAR 182504 TSF:MSG:SEND,255-255-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 184511 !TSM:FPAR:FAIL 184513 TSM:FAIL:CNT=7 184515 TSM:FAIL:PDT
I also have the debug of the Raspi, which tells me
Client 0: 0;0;3;0;18;PING
which should be Domoticz. There is nothing else showing up.
Domotics says it´s a repeater node, as on the motion sensor..
RE: A few noob questions
Thank you very much for your answer
There is no "connection" to sensors really. Sensors send messages whenever they are programmed to. The gateway forwards those messages to the chosen output (serial, ethernet , mqtt, etc). The format of the output is called "serial protocol" (because it originated from serial but the same protocol is used for ethernet, ...). The serial protocol is described at https://www.mysensors.org/download/serial_api_20
The log parser can be useful to get details on what the log is saying.
Very helpful, thank you
In your case, the node is asking the controller for a sensor node id. The gateway logs this "ask" message and forwards it to the output. But there is no reply from the controller. Have you added the gateway in Domoticz? If so, could you post your settings from the Domoticz hardware tab, the Domoticz log and the configure command you used for the raspi gateway?
2017-11-07 14:27:53.024 Domoticz V3.8153 (c)2012-2017 GizMoCuz 2017-11-07 14:27:53.025 Build Hash: 494fff7, Date: 2017-07-30 12:19:41 2017-11-07 14:27:53.026 Startup Path: /home/pi/domoticz/ 2017-11-07 14:27:54.028 PluginSystem: Started, Python version '3.5.3'. 2017-11-07 14:27:54.046 Active notification Subsystems: gcm, http (2/12) 2017-11-07 14:27:54.082 WebServer(HTTP) started on address: :: with port 8080 2017-11-07 14:27:54.100 WebServer(SSL) started on address: :: with port 443 2017-11-07 14:27:54.101 Proxymanager started. 2017-11-07 14:27:54.102 Starting shared server on: :::6144 2017-11-07 14:27:54.102 TCPServer: shared server started... 2017-11-07 14:27:54.103 RxQueue: queue worker started... 2017-11-07 14:27:56.197 EventSystem: reset all events... 2017-11-07 14:27:56.211 EventSystem: reset all device statuses... 2017-11-07 14:27:56.365 Python EventSystem: Module not found - Trying to initialize. 2017-11-07 14:27:56.366 Python EventSystem: Initalizing event module. 2017-11-07 14:27:56.367 EventSystem: Started 2017-11-07 14:27:56.529 PluginSystem: Entering work loop. 2017-11-07 14:27:57.196 MySensors: trying to connect to: 192.168.178.66:5003 2017-11-07 14:27:58.197 Error: MySensors: Network is unreachable 2017-11-07 14:27:58.197 Error: TCP: Error: Network is unreachable 2017-11-07 14:27:58.197 TCP: Reconnecting in 30 seconds... 2017-11-07 14:28:08.706 Incoming connection from: 192.168.178.37 2017-11-07 14:28:48.762 TCP: Reconnecting... 2017-11-07 14:28:48.762 MySensors: connected to: 192.168.178.66:5003 2017-11-07 14:28:49.762 MySensors: Gateway Ready... 2017-11-07 14:28:49.763 MySensors: Gateway Version: 2.2.0-rc.1 2017-11-07 14:28:50.763 MySensors: Gateway Version: 2.2.0-rc.1
./configure --my-gateway=ethernet --my-port=5003
Raspi is 192.168.178.66, running Domotics (standart ports) and MySensors gateway. It´s a test OS, all fresh installations, up to date. Domotics is not connected to my smart home atm to avoid any problems. All added is the MS-gateway.
Sensor is a motion sensor at the Arduino Uno.
The log parser and the serial protocol are the main sources. But if you haven't already, read the getting started guide. Reading that will give you the background needed to understand how things fit together.
It is more the understanding of how sensor reading and sending data to the gateway is handled. In case of the BH1750 I have code which won´t need a libary, it´s very effective, very small. As I plan to combine sensors, less code would be fine to let a nano handle as much as possible. The links you provided should be very helpful setting up my own code after everything runs stable.
There seems to be a lot of html code in that library. There shouldn't be any html code in a library. Where did you download it from and how did you download it? Maybe you accidentally saved the github html page instead of saving the (raw) library file?
Argh, possible, as there was no zip archive I just right-clicked and saved from Github. I will try copy / paste instead^^.Was my DL -.-
Have you seen the instructions and the example sketch at https://www.mysensors.org/build/light-bh1750 ? They are intended to provide all you need to get the sensor working, but we may have failed.
I copied 1:1, assume my fault with downloading the libary causes the problem.Compiling works now
I hope this helps. If you have further questions, just ask (but do read the instructions I linked first, they are intended to bring people up to speed as easily and quickly as possible so hopefully they'll save you some time)
I will definitely read through, already read the "getting started" section, will continue with the protocol wiki.
Btw, I just wanted to test if the problem of communicaton could be on the hardware side. I stopped the gateway on the RPI, and installed RF24 stuff from "http://tmrh20.github.io/RF24Installer/RPi/install.sh". Also used this before i found MySensors, to test the RPi/Arduino. With the getting started script, and RPi set to respond, I was not able to get a clean connection. The RPi was not able to send. The OS is, as I said, a fresh, updated install, only MS-gateway and Domotics are installed. I suspected the Rpi to cause the problem, switched the SD-card to the one with my running Domotics, installed also from "http://tmrh20.github.io/RF24Installer/RPi/install.sh" - and the Rpi was perfectly answering. Exact same hardware, only OS was switched. Maybe the problem is on the OS itself, If the information of Domotics do not give you a hint whats going wrong, I will reinstall the OS / MS / Domotics again..Reinstall also did not solve the problem.
A few noob questions
I´m absolutely new to MySensors. I have Domoticz connected to my Yeelight-System.
Now I wanted to expand with Arduinos, PIDs, light sensors..
First off I played around a bit with my Uno, Raspi and some sensors. My NRF24L01+ on Raspi and Arduino were able to ping and answer, all fine. Luxmeter and PIR gave the output expected.
Then I read about MySensors, installed on the Raspberry (dev channel, stable did not work).Gateway type ethernet. Log sais all fine.
- Question: as I read, there is no way just to see what is connected to the RPI, which sensor sends what.. right? All I have is the option to fully debug to a file. If I have a connection to a sensor, I have to set it up in Domotics to read the values?
Ok, Raspi runs fine, now I tried to set up my Uno. MySensors-Libary is included, I can compile just fine. Problem: I always get the following on my serial monitor (if debug is enabled):
0 MCO:BGN:INIT NODE,CP=RNNNA--,VER=2.1.1 3 TSM:INIT 4 TSF:WUR:MS=0 11 TSM:INIT:TSP OK 13 TSM:FPAR 15 TSF:MSG:SEND,255-255-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 2022 !TSM:FPAR:NO REPLY 2024 TSM:FPAR 2026 TSF:MSG:SEND,255-255-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 4034 !TSM:FPAR:NO REPLY 4036 TSM:FPAR 4038 TSF:MSG:SEND,255-255-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 6046 !TSM:FPAR:NO REPLY 6048 TSM:FPAR 6050 TSF:MSG:SEND,255-255-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK: 8058 !TSM:FPAR:FAIL 8059 TSM:FAIL:CNT=1 8061 TSM:FAIL:PDT 18065 TSM:FAIL:RE-INIT ...
I have no idea what it wants to tell me. It sends four times, gets no answer, so I get an error and it starts from the beginning.
2) Question: where can I get a good documentation, which tells me what is normal, what is sended. Would also be awesome if I could see what the Raspi receives. If I open syslog, it tells me just all is fine once in an hour.
I also downloaded the BT1750 libary, as linked on the tutorial page. Is not the first libary I added manually, so folder should be correct. If I now compile, I get a lot of errors:
In file included from C:\Users\Inso\AppData\Local\Temp\arduino_modified_sketch_59209\LightSensor.ino:50:0: C:\Users\Inso\Documents\Arduino\libraries\BH1750/BH1750.h:30:3: error: stray '\302' in program <title>MySensorsArduinoExamples/BH1750.h at master Â· mysensors/MySensorsArduinoExamples Â· GitHub</title> ^ C:\Users\Inso\Documents\Arduino\libraries\BH1750/BH1750.h:30:3: error: stray '\267' in program C:\Users\Inso\Documents\Arduino\libraries\BH1750/BH1750.h:30:3: error: stray '\302' in program C:\Users\Inso\Documents\Arduino\libraries\BH1750/BH1750.h:30:3: error: stray '\267' in program In file included from C:\Users\Inso\AppData\Local\Temp\arduino_modified_sketch_59209\LightSensor.ino:50:0: C:\Users\Inso\Documents\Arduino\libraries\BH1750/BH1750.h:151:8: warning: missing terminating ' character <!-- '"` --><!-- </textarea></xmp> --></option></form><form accept-charset="UTF-8" action="/mysensors/MySensorsArduinoExamples/search" class="js-site-search-form" data-scoped-search-url="/mysensors/MySensorsArduinoExamples/search" data-unscoped-search-url="/search" method="get"><div style="margin:0;padding:0;display:inline"><input name="utf8" type="hidden" value="✓" /></div> ^ C:\Users\Inso\Documents\Arduino\libraries\BH1750/BH1750.h:151:3: error: missing terminating ' character <!-- '"` --><!-- </textarea></xmp> --></option></form><form accept-charset="UTF-8" action="/mysensors/MySensorsArduinoExamples/search" class="js-site-search-form" data-scoped-search-url="/mysensors/MySensorsArduinoExamples/search" data-unscoped-search-url="/search" method="get"><div style="margin:0;padding:0;display:inline"><input name="utf8" type="hidden" value="✓" /></div> ^ In file included from C:\Users\Inso\AppData\Local\Temp\arduino_modified_sketch_59209\LightSensor.ino:50:0: C:\Users\Inso\Documents\Arduino\libraries\BH1750/BH1750.h:313:48: error: exponent has no digits <!-- blob contrib key: blob_contributors:v21:982ea3630030e14dff7f417b07c6caf3 --> ^ In file included from C:\Users\Inso\AppData\Local\Temp\arduino_modified_sketch_59209\LightSensor.ino:50:0: C:\Users\Inso\Documents\Arduino\libraries\BH1750/BH1750.h:532:70: error: stray '#' in program <td id="LC16" class="blob-code blob-code-inner js-file-line">#<span class="pl-k">ifndef</span> BH1750_h</td> ^ C:\Users\Inso\Documents\Arduino\libraries\BH1750/BH1750.h:536:70: error: stray '#' in program <td id="LC17" class="blob-code blob-code-inner js-file-line">#<span class="pl-k">define</span> <span class="pl-en">BH1750_h</span></td> ^ C:\Users\Inso\Documents\Arduino\libraries\BH1750/BH1750.h:545:70: error: stray '#' in program <td id="LC19" class="blob-code blob-code-inner js-file-line">#<span class="pl-k">if</span> (ARDUINO >= 100)</td> [...]
- Question: Any idea what wents wrong? Folder on Github says it´s all up to date, my Arduino IDE is 1.8.5
There is also a lot of good code on the net.
- Question: If I want to use that code, what is neccessary to add so MySensors can handle the data? I add
so it sends my data,
so it adds my Arduino as a Sensor, gives a unique ID.
is the data I send.
Here it starts - is there a wiki, hopefully with a few examples, where I can see which data has to be send, what is needed, how I can set up my own code..?
Thanks a lot for reading my wall of text, everything is appriciated
RE: 💬 Building a Raspberry Pi Gateway
For the ones having problems with the stable build (Version 2.1.1), always getting "mysgw: !TSM:INIT:TSP FAIL":
Seems there is a bug, you have to install "development" (Version 2.2.0)
Maybe someone adds an note on top of the tutorial, so others won´t also spend hours in searching for the problem on their Pi/Hardware.