For the ones having problems with the stable build (Version 2.1.1), always getting "mysgw: !TSM:INIT:TSP FAIL":
Seems there is a bug, you have to install "development" (Version 2.2.0)
Maybe someone adds an note on top of the tutorial, so others won´t also spend hours in searching for the problem on their Pi/Hardware.
Thank you very much for your answer 
@mfalkvidd said in A few noob questions:
There is no "connection" to sensors really. Sensors send messages whenever they are programmed to. The gateway forwards those messages to the chosen output (serial, ethernet , mqtt, etc). The format of the output is called "serial protocol" (because it originated from serial but the same protocol is used for ethernet, ...). The serial protocol is described at https://www.mysensors.org/download/serial_api_20
The log parser can be useful to get details on what the log is saying.
Very helpful, thank you
In your case, the node is asking the controller for a sensor node id. The gateway logs this "ask" message and forwards it to the output. But there is no reply from the controller. Have you added the gateway in Domoticz? If so, could you post your settings from the Domoticz hardware tab, the Domoticz log and the configure command you used for the raspi gateway?
2017-11-07 14:27:53.024 Domoticz V3.8153 (c)2012-2017 GizMoCuz
2017-11-07 14:27:53.025 Build Hash: 494fff7, Date: 2017-07-30 12:19:41
2017-11-07 14:27:53.026 Startup Path: /home/pi/domoticz/
2017-11-07 14:27:54.028 PluginSystem: Started, Python version '3.5.3'.
2017-11-07 14:27:54.046 Active notification Subsystems: gcm, http (2/12)
2017-11-07 14:27:54.082 WebServer(HTTP) started on address: :: with port 8080
2017-11-07 14:27:54.100 WebServer(SSL) started on address: :: with port 443
2017-11-07 14:27:54.101 Proxymanager started.
2017-11-07 14:27:54.102 Starting shared server on: :::6144
2017-11-07 14:27:54.102 TCPServer: shared server started...
2017-11-07 14:27:54.103 RxQueue: queue worker started...
2017-11-07 14:27:56.197 EventSystem: reset all events...
2017-11-07 14:27:56.211 EventSystem: reset all device statuses...
2017-11-07 14:27:56.365 Python EventSystem: Module not found - Trying to initialize.
2017-11-07 14:27:56.366 Python EventSystem: Initalizing event module.
2017-11-07 14:27:56.367 EventSystem: Started
2017-11-07 14:27:56.529 PluginSystem: Entering work loop.
2017-11-07 14:27:57.196 MySensors: trying to connect to: 192.168.178.66:5003
2017-11-07 14:27:58.197 Error: MySensors: Network is unreachable
2017-11-07 14:27:58.197 Error: TCP: Error: Network is unreachable
2017-11-07 14:27:58.197 TCP: Reconnecting in 30 seconds...
2017-11-07 14:28:08.706 Incoming connection from: 192.168.178.37
2017-11-07 14:28:48.762 TCP: Reconnecting...
2017-11-07 14:28:48.762 MySensors: connected to: 192.168.178.66:5003
2017-11-07 14:28:49.762 MySensors: Gateway Ready...
2017-11-07 14:28:49.763 MySensors: Gateway Version: 2.2.0-rc.1
2017-11-07 14:28:50.763 MySensors: Gateway Version: 2.2.0-rc.1
./configure --my-gateway=ethernet --my-port=5003
Raspi is 192.168.178.66, running Domotics (standart ports) and MySensors gateway. It´s a test OS, all fresh installations, up to date. Domotics is not connected to my smart home atm to avoid any problems. All added is the MS-gateway.
Sensor is a motion sensor at the Arduino Uno.
The log parser and the serial protocol are the main sources. But if you haven't already, read the getting started guide. Reading that will give you the background needed to understand how things fit together.
It is more the understanding of how sensor reading and sending data to the gateway is handled. In case of the BH1750 I have code which won´t need a libary, it´s very effective, very small. As I plan to combine sensors, less code would be fine to let a nano handle as much as possible. The links you provided should be very helpful setting up my own code after everything runs stable.
There seems to be a lot of html code in that library. There shouldn't be any html code in a library. Where did you download it from and how did you download it? Maybe you accidentally saved the github html page instead of saving the (raw) library file?
Argh, possible, as there was no zip archive I just right-clicked and saved from Github. I will try copy / paste instead^^. Was my DL -.-
Have you seen the instructions and the example sketch at https://www.mysensors.org/build/light-bh1750 ? They are intended to provide all you need to get the sensor working, but we may have failed.
I copied 1:1, assume my fault with downloading the libary causes the problem. Compiling works now
I hope this helps. If you have further questions, just ask (but do read the instructions I linked first, they are intended to bring people up to speed as easily and quickly as possible so hopefully they'll save you some time)
I will definitely read through, already read the "getting started" section, will continue with the protocol wiki.
Btw, I just wanted to test if the problem of communicaton could be on the hardware side. I stopped the gateway on the RPI, and installed RF24 stuff from "http://tmrh20.github.io/RF24Installer/RPi/install.sh". Also used this before i found MySensors, to test the RPi/Arduino. With the getting started script, and RPi set to respond, I was not able to get a clean connection. The RPi was not able to send. The OS is, as I said, a fresh, updated install, only MS-gateway and Domotics are installed. I suspected the Rpi to cause the problem, switched the SD-card to the one with my running Domotics, installed also from "http://tmrh20.github.io/RF24Installer/RPi/install.sh" - and the Rpi was perfectly answering. Exact same hardware, only OS was switched. Maybe the problem is on the OS itself, If the information of Domotics do not give you a hint whats going wrong, I will reinstall the OS / MS / Domotics again.. Reinstall also did not solve the problem.
@gohan ah okay wasn´t aware of it, as the dev channel worked I assumed the bug at MySensors.
However, would be awesome if there were a little hint in the tutorial. Took plenty of time to find out it was an software incompability 
@mfalkvidd
Finally it works, thank you very much for your fast and detailed help!
If someone else has the same problem : I exchanged the capacitor (had two old ones without number on it, now 47uf best China quality :D) which solved the problem. Seems with the old ones the voltage collapsed before sending all data, so ping worked, communication didn't. I btw need the capacitor also to get usable lux values from my bh1750, without its always the maximum lux value.
@mirodin
make install before testing did the trick, now it is 2.3.0 . 
Was just going step by step through tutorial, there it´s make -> test -> make install. Didn´t realized it would use old parts if it´s not first install 
@gohan
thought stop, disable and remove the service would be enough to ensure the service is completely "uninstalled" - could you give me a hint what I´ve missed? 
@rejoe2
You´re absolutely right.
After 4+ hours debugging the MQTT gateway + ~10h trying to get MySensors MQTT on ethernet working properly, I tried a different way:
installed NoteRed on the RPi and an asynchronous MQTT client on the ESP32, and within ~1 hour I had a fast and stable way to send messages from and to my nodes. Adding JSON parser on the ESP, and I can perfectly communicate with dummy Domoticz devices.
Also, except of you and one very helpful guy within a private chat, noone preplied to MQTT-questions and ESP-stuff. Seems working with ESPs is reluctantly seen here on the forum.?
So I assume a different way then using MySensors for everything except (battery-powered) Arduinos is the best I can do, to also have no problems in the future..
@rejoe2
If I only would set up 8266´s I´d give it a try - problem is, I´m in love with the ESP 32.
Bluetooth LE, multithreading, a second core only for the wifi stuff so no crashes on high wlan activity and no problems with time-critical events, lots of GPIO, awesome processor power, combined with the lower power consumption makes this the most-used chip here
As far as I can see, both solutions you mentioned are not made for the ESP32 
.
So I´d like to stick to MySensors as I could use lots of the sketches almost 1:1 (can´t be that hard to get more then one MQTT-"Client" running -.-).
Only real other options I see is using MQTT-Clientsoftware from Basecamp or just set up an asynchronous http socket and code / decode the JSON-parameters send from and to Domoticz myself..
@mfalkvidd
Quotes:
"Don't define MY_RADIO_NRF24 and you're good to go. It will still be a gateway, but a gateway with local sensors and no radio."
and
"On one MySensors network there can only be one MySensors gateway. But on one IP network, there can be many MySensors gateways. Exactly how many depends on geography and chosen transports but at the very least a few hundred. Thousands if you really need and plan accordingly."
Let´s say I have multiple ESPs, which should all act like a sensor node. I have a RPi w Domoticz, Mosquitto and MySensors MQTT ethernet gateway.
I add the MQTT gateway to Domotics, listening on port 1883 (Mosquitto port). Now I set up a sketch, using
#define MY_IP_ADDRESS 192,168,178,151
#define MY_CONTROLLER_IP_ADDRESS 192,168,178,200
#define MY_PORT 1883
which sets the gateway to client / node mode (as read on the documentation).
Setting a node Id is ignored, ID is 0 and can´t be changed, so it is recognized as a gateway at domoticz.
If I now set a second node to a different IP address and name, it overwrites the old one (as you say, there can be only one gateway).
The question now is: how can I set up another ESP? Do I need another RPi or virtual machine with different IP address to install another gateway?
Update: rebuild from the bones. Raspi-OS, Domoticz, Mosquitto, MySensors.
I don´t get a version of my gateway nether.
Sensors run but have to be renamed complete within Domotics (database backup did not help), and all data is lost.
Hi,
I want to switch from an ethernet- to an MQTT-gateway, to be able to add some ESP8266 and ESP32 "natively", i.e. without extra nRF24-modules.
From all I´ve read, my "old" gateway (ethernet) uses the nRF24-module of my Raspi to communicate with the nodes. MySensors handels the messaging stuff, talks to Domoticz using the easy-to-use LAN interface.
Switching from ethernet to MQTT means I have the option to send also data per MQTT to and from the MS-gateway, besides the "usual nRF24-way". Only difference is to setup a MQTT gateway instead of the ethernet-gateway within domotics and to install a Mosquitto Broker (Server). Now the communication Domoticz <-> MySensors uses MQTT the usual ethernet gateway, nodes can connect the "usual way" using nRF24 or communicate using MQTT. MySensors uses the Mosquitto server for some implementation(?). Communication is then node <-> Mosquitto Broker <-> MySensors <-> Domoticz (?)
Sadly, the Domoticz tutorial is very unclear about that , it just said install Mosquitto, you need it. Also the Build-section here did not help much, and reading here on the forum did confuse more then it helped.
However, I managed it to let Domoticz and MySensors connect fine.
Nov 30 02:16:29 INFO Starting gateway...
Nov 30 02:16:29 INFO Protocol version - 2.3.0
Nov 30 02:16:29 DEBUG MCO:BGN:INIT GW,CP=RNNGL---,VER=2.3.0
Nov 30 02:16:29 DEBUG TSF:LRT:OK
Nov 30 02:16:29 DEBUG TSM:INIT
Nov 30 02:16:29 DEBUG TSF:WUR:MS=0
Nov 30 02:16:29 DEBUG TSM:INIT:TSP OK
Nov 30 02:16:29 DEBUG TSM:INIT:GW MODE
Nov 30 02:16:29 DEBUG TSM:READY:ID=0,PAR=0,DIS=0
Nov 30 02:16:29 DEBUG MCO:REG:NOT NEEDED
Nov 30 02:16:29 DEBUG MCO:BGN:STP
Nov 30 02:16:29 DEBUG MCO:BGN:INIT OK,TSP=1
Nov 30 02:16:29 DEBUG GWT:RMQ:MQTT RECONNECT
Nov 30 02:16:29 DEBUG connected to 127.0.0.1
Nov 30 02:16:29 DEBUG GWT:RMQ:MQTT CONNECTED
Nov 30 02:16:29 DEBUG GWT:TPS:TOPIC=domoticz/in/MyMQTT/0/255/0/0/18,MSG SENT
Nov 30 02:16:36 DEBUG TSF:MSG:READ,51-51-0,s=2,c=1,t=37,pt=3,l=2,sg=0:5
Nov 30 02:16:36 DEBUG GWT:TPS:TOPIC=domoticz/in/MyMQTT/51/2/1/0/37,MSG SENT
Nov 30 02:16:56 DEBUG GWT:IMQ:TOPIC=domoticz/out/MyMQTT/0/0/3/0/18, MSG RECEIVED
Nov 30 02:16:56 DEBUG GWT:TPS:TOPIC=domoticz/in/MyMQTT/0/255/3/0/22,MSG SENT
Problems starting with having no Version on hardware-tab (MySensors Gateway with MQTT interface
Version: ?), and the type of my "old" nodes is not recognized. I see the Node IDs I gave em, children type is unknown, values (f.e. #1.V_TEMP (24.3)) are received, but do not show up at the devices tab.
Do I have to rewrite all the Nodes to now use MQTT? As the nRF-nodes have no ethernet-interface, I only could add MY_MQTT_PUBLISH_TOPIC_PREFIX and MY_MQTT_SUBSCRIBE_TOPIC_PREFIX.
Sorry if I am asking noobish questions here, I´m only searching for a good way to let Arduinos and ESPs work side by side fine while using the hardware they provide.
From all I´ve read, a MQTT gateway is the best way (better then connect every ESP as a separate gateway). Or maybe I have missed something?
@mirodin
make install before testing did the trick, now it is 2.3.0 .
Was just going step by step through tutorial, there it´s make -> test -> make install. Didn´t realized it would use old parts if it´s not first install
@gohan
thought stop, disable and remove the service would be enough to ensure the service is completely "uninstalled" - could you give me a hint what I´ve missed?
I am currently trying to update to 2.3
Have stopped service and deleted the file (systemctl status mysgw -> Unit mysgw.service could not be found.)
Also renamed the config file /etc/mysensors.dat
Renamed the old folder, then copied new files.
Checked readme.md in MySensors-folder: MySensors Library v2.3.0
Then I configured the radio and -> make, no errors
However, changing to bin folder and executing mysgw shows: Protocol 2.2.0
What am I missing?
To answer my own question: yes, it seems to work. Searched around and read a while, found this:
https://forum.mysensors.org/topic/6511/hc-sr501-3-3v-randomly-sends-tripped-when-radio-is-on/22
Best addition imho ,no need for step up / down. 
@gohan
Thanks for your reply
It seems it is the antenna on intialisation. I have switched form NRF24 with antenna as print on PCB to the ones with external antenna, plan was to have more range. Always supported by a 470μF condensator.
If I put my finger on top of the antenna while it initializes, I have a stable connection without restart of the gateway. From there on, I do not have to touch it anymore, communication runs perfect. If I do not touch the antenna, I have only errors until I restart the gateway on the RPI (restarting the node does not have an effect).
Btw , antennas were 20cm, then 3m away from each other, nothing in between
I have updated MySensors to 2.2.0 on my RPi two weeks ago. Clean new install. If I try to connect from a node, I get error messages, as long as I do not restart the MySensors gateway. MySensors Libary on Arduino IDE is also 2.2.0, all sketches are updated.
2125 TSF:MSG:SEND,20-20-255-255,s=255,c=3,t=7,pt=0,l=0,sg=0,ft=0,st=OK:
4133 !TSM:FPAR:NO REPLY
4135 TSM:FPAR
On the RPi MySensors runs atm in debug mode - I can not see any incoming communication. As soon as mysgw is restarted (kill process, then mysgw -d), mostly the node gets the connection and runs fine (without restart). As soon as I restart the node (soft reset or sketch update), there´s again no way it connects.
Tested with several notes and sketches, also example sketches from the page.