Home Assistant - How to recognize MySensors nodes

kielnino

Hi,
I'm trying to set up home assistant (http://home-assistant.io) with the MySensors infrastructure. I've configured the mysensor-part in the configuration.yaml. But I dont know how the proceed. How can my sensors now be recognized. Is there a way to debug the communication between HA and the serial gateway?

I've used the example scetches for the serial gateway and a motion sensor. At startup of the serial Gateway I start the inclusion mode and then power up the motion sensor. If the gateway is connected to my computer I can see the messages from the motion sensor. But if I plug the serial Gateway to the Raspberry on which home assistant is running on, nothing happens. Even on the console ouput and in the logs of HA are no errors or warnings.

martinhjelmare

@kielnino

Hi!
I've been working on adding functionality to the mysensors component of home assistant, in my github forks, pymysensors and home-assistant. My experience so far is that if you have the gateway connected and then start home assistant and then start your sensor, it should get connected automatically. You might need to send at least one value from the sensor, though, to have it show in the web gui.

There are some issues with the current release of pymysensors, though, which can make it less reliable sometimes. One problem is a bug in the pyserial version used. There's a fresh patch for this now, but I don't think it's been released for the version at pypi used by pymysensors and home assistant. Sometimes you also need to restart home assistant after the first start, due to the serial connection bugging out. But only one restart is needed.

Activate debug mode in the config for the mysensors component in home assistant. It's also easiest in my experience when debugging to start home assistant manually from the command line and not run it as a service. You will then see all the log output directly, and can follow what happens. Deactivate every other customization you might have done in the config, so you minimize clutter in the log output, until you know it's working.

kielnino

Thanks for the detailed response.
I noticed, that only version 1.4 is loaded in the mysensors-plugin so far. (https://github.com/balloob/home-assistant/blob/dev/homeassistant/components/sensor/mysensors.py#L37). I wonder if this may be a problem as I used the new 1.5.

martinhjelmare

No, that limitation (version 1.4 hardcoded) shouldn't be a problem. The motion sensor "S_VALUES" and "V_VALUES" have not changed between version 1.4 and 1.5. It's only the available serial API that's changed depending on which version is loaded in the mysensors component of home assistant.

You can also try assigning a set node id, preferably "1" in your sketch, if you were using auto before? I haven't tested the auto assignment that much, so to narrow down possible problems, it might be better.

kielnino

I've used the example sketches for both, the gateway and the sensor. I think the node id is set to 1 in the example.
The main problem is, that I dont know how to test the serial communication between home assistant and the gateway. I have the debug option enabled and I can see the inclusion of the mysensor plugin in the logs at start up. But after that nothing happens.

martinhjelmare

In the latest release of home assistant version 0.8 the log level is not hardcoded to info anymore. But this release is only four hours old and is not at pypi yet. The problem is that pymysensors has set the log level for the debug printout of commands to debug, while home assistant has the lowest allowed level at info, which is higher than debug. So until version 0.8 of home assistant you can fix this by changing debug to info here:
https://github.com/theolind/pymysensors/blob/master/mysensors/mysensors.py#L83

This file will be installed in ~/.homeassistant/lib/mysensors folder.

This will make the debug option in the config actually work and print out the commands sent/received by the gateway. For example:

15-11-16 11:33:43 INFO (Thread-9) [mysensors.mysensors] n:0 c:0 t:3 s:9 p:gateway started, id=0, parent=0, distance=0

humblehacker

Hello,
I've been trying to get a MSG set up with HA for a couple days now with no luck. I've tried the recommendations made above and tried turning persistence on and off and while Ive been able to confirm HASS has registered the component, I still can't get anything to show up in the log... Any other suggestions about how I can debug? Another user made a comment about there being a MySensors MQTT client gateway available for HA but that was the only time Ive seen it mentioned. Is this another possibility?

martinhjelmare

@humblehacker

Hi!

There is general MQTT support in home-assistant, not specific to mysensors, but I have too little experience using the MQTT features in mysensors, to say if it will work or not. I guess it depends on how the mysensors protocol is translated by the MQTT gateway and posted to the broker. If you want to try it, I suggest using the dev branch of mysensors and following this guide:

http://forum.mysensors.org/topic/2352/guide-setting-up-and-testing-mqtt-client-gateway

Here is the home-assistant docs:
https://home-assistant.io/components/mqtt/

Looking at the topic structure and looking at the docs at home-assistant, I'm optimistic. Maybe I'll even try it myself.

Here are some advice if you want to continue trying the serial gateway approach:
Are you using the latest release of home-assistant (0.8)? That release supports debug level logging as default, which should make the debug mode of the mysensors component work. That was not the case before.

First, make sure your gateway and test node is talking without the controller. Hook up serial monitors and check the output on both sides. I'd be happy to look at that if you want to post it here.

Then, hook up your serial gateway to the home-assistant computer. Start home-assistant from the command line: (hass). Check the log output. Post it here if you want.

Then, powercycle your node. Make sure you have the serial monitor up on the node, or just reconnect it, and the node should reset. Look at the serial output on the node side, and check the log from hass, if there is anything. Post it here if you want. You can't have the serial monitor on the gateway active when starting hass, as the serial connection is exclusive.

Edit:
Also post your home-assistant config if you want, which is a crucial piece of evidence.

humblehacker

Are you using the latest release of home-assistant (0.8)? That release supports debug level logging as default, which should make the debug mode of the mysensors component work. That was not the case before.

Yes I can confirm I am running version 0.8 and having just reread your above post, I realized changing debug to info in mysensors.py only applies to >v0.8 so I had to change it back and reboot again. I"ve been using the terminal output from hass -ui as my main debugger and through this was able to confirm that the mysensors component is recognized. Also troubleshooting on the Home Assistant gitter page over the past few days and tried enabling and disabling persistence from the config.yaml.

First, make sure your gateway and test node is talking without the controller. Hook up serial monitors and check the output on both sides. I'd be happy to look at that if you want to post it here.
Then, hook up your serial gateway to the home-assistant computer. Start home-assistant from the command line: (hass). Check the log output. Post it here if you want.
Then, powercycle your node. Make sure you have the serial monitor up on the node, or just reconnect it, and the node should reset. Look at the serial output on the node side, and check the log from hass, if there is anything. Post it here if you want. You can't have the serial monitor on the gateway active when starting hass, as the serial connection is exclusive.

OK yes. This is the part I've really needed help with as I can't seem to find any documentation from the MySensors end about debugging via serial monitor. I tried opening a serial monitor as I normally would with the Arduino IDE and nothing happened but I dont know if the MySensors library uses a custom protocol or baud rate. Is there a tutorial or guide on the subject that you can point me to? I can connect an OLED and/or LCD screen directly too if that is possible.

martinhjelmare

For debugging with serial monitor, just plug in both the gateway and the node to your computer, either through USB if your arduino supports that or through FTDI adapter. Then open two instances of the Arduino IDE and open one serial monitor per IDE. Make sure you set board and port correct first, in the two IDEs, one per arduino. Mysensors use baudrate 115200, so make sure that's selected in the serial monitors.

For an explanation of the mysensors' serial api, see:
http://www.mysensors.org/download/serial_api_15

In the end of that page there are some examples of serial output.

humblehacker

Yes, it was just the baud rate that had me thrown off I guess, since the default is always 9600 on Arduino. Anyway, the serial gateway seems to be functioning as expected printing 0;0;3;0;9;gateway started, id=0, parent=0, distance=0 0;0;3;0;14;Gateway startup complete.

However when I load up the motion sensor sketch and open the monitor, it still shows nothing. I then tried a couple different sensors incl. a PIR sensor with the distance example and another with the ds18b20 temp sensor with the same result. Finally I loaded up the MockMySensors sketch and again the serial monitor was blank.
I then loaded up the default DigitalSerialRead sketch from the Arduino IDE and the serial output printed as expected. Then lastly, I even tested the PhysicalPixel sketch with the standard NRF24 library and it transmitted as expected! So the issue is definitely with the MySensors library. But then I still couldnt even get it to work using Codebender and that should be using a completely different library file. SO I'm all out of ideas... In any case, there still could be an issue on the Controller end, but I have no way of knowing until I can fix the sender function.

martinhjelmare

Have you activated (uncommented) DEBUG in MyConfig.h?

humblehacker

As I keep my sketchbook synchronized through Github, I can confirm that debugging is enabled in my master here
However, I will need to confirm that the local sketchbook on the laptop I was using yesterday is up to date with the remote master. I'm pretty sure I enabled debugging when I first installed it.

EDIT- Isn't debugging enabled by default now?- https://github.com/mysensors/Arduino/blob/master/libraries/MySensors/MyConfig.h

acmbc68

Hi,

I've been following this thread for a few days, as I am in the same situation and kielnino - I can see the MySensors component get loaded and then....silence....I know the MySensors GW and nodes are working from testing with serial monitors and another controller app (MySController).

Will probably be spending some time tracing through the pymysensors code tonight to see what gives. I do have debug enabled on homeassistant, but still no joy.

One thought - shouldn't MySensors on homeassistant be implemented as a platform (and not a sensor) much like MQTT.

I have got readings to work through MQTT (in my case, patched a node-red flow together to read the serial line and publish to a local mosquitto), but this is like driving around the city to get to the store next door...

Any further thoughts/hints as to where to start looking?

Thanks!

martinhjelmare

@humblehacker

Yes, it should be enabled by default, that's why I didn't mention it in my first advice. But it's good to double check anyway.

Are you sure you have two different instances of the Arduino IDE running when you test? Only one serial monitor can be active at a time, per IDE. It's not enough to open a second sketch window. You have to start the IDE twice from the start menu, or however you start it. Also, check the ports. You should see different headings in the serial monitor window, specifying the port.

What hardware are you using?

martinhjelmare

@acmbc68

Hi!

Regarding the current mysensors implementation in home assistant, it's a sensor platform. But it's not a separate component.

I have added a mysensors component together with a switch platform, in my github fork of home assistant. I haven't made a pull request yet, cause I'm waiting to have my other pull request to pymysensors approved, where I implement support for setting child values from the controller, cause it's a dependency.

https://github.com/MartinHjelmare/home-assistant/tree/mysensors-component-switch

https://github.com/MartinHjelmare/pymysensors/tree/fifo_queue

I'm not completely finished with the home assistant switch branch. I want to add support for multiple gateways as well, and I haven't had time to test my solution properly yet.

I've added more debug logging in my fifo_queue branch of pymysensors, and made the serial connection more robust, by closing it when the thread quits. I've also downgraded the required pyserial version to 2.5, due to a bug in pyserial 2.6 - 3. This has been fixed in the master branch of pyserial, but no new release exists, I think.

If you know how to change installed components in home assistant, you can give my branches a try. They should be working in their current state on github, but more testing is always good.

When I first started testing connecting my node with the gateway and home assistant, I had problems in the beginning, When I set the node id to 1 and made sure at least one value was sent from the node, I got it working. It might have been a coincidence, that it started working then, cause I'm not sure that home assistant needs sequential entity ids, but when you're debugging it's good to narrow down possible variables/problems. You also have to make sure that the gateway receives all the presentation data, including sketch name, from the node. Otherwise the sensor won't show up in the GUI. That's currently how it's written in the code.

Edit:
One more thing. Even though debug level logging should be enabled by default in home assistant 0.8, you still have to enable the logger component in your config.
https://home-assistant.io/components/logger/

acmbc68

Thanks @martinhjelmare

As per https://home-assistant.io/blog/posts/2/, all the components should be under the homeassistant install, so it shouldn't be a brain-wrecker to change those out.

I'll start giving it a whirl tonight.

acmbc68

Ok.
First of all - still no joy. Can't get any communications from MySensors to homeassistant.

Trying to tackle this bottom up. Thought it might be an issue with PySerial, but that reads data OK from the serial port:

import serial as serial
print ("starting")
ser = serial.Serial (
    port='COM4',
    baudrate=115200,
    parity=serial.PARITY_NONE,
    stopbits=serial.STOPBITS_ONE,
    bytesize=serial.EIGHTBITS
)

ser.flushInput()
ser.flushOutput()

print("connected to: " + ser.portstr)
count=1

while True:
        try:
                data_raw = ser.readline()
                print(data_raw)
        except serial.SerialException:
                print ("SERIAL ERROR")
                exit(1)

This results in what is expected:

starting
connected to: COM4
b'0;0;3;0;14;Gateway startup complete.\n'
b'0;0;3;0;9;read: 1-1-0 s=2,c=1,t=23,pt=2,l=2:0\n'
b'1;2;1;0;23;0\n'
b'0;0;3;0;9;read: 1-1-0 s=1,c=1,t=0,pt=7,l=5:25.0\n'
b'1;1;1;0;0;25.0\n'
b'0;0;3;0;9;read: 1-1-0 s=0,c=1,t=1,pt=7,l=5:84.0\n'
b'1;0;1;0;1;84.0\n'
b'0;0;3;0;9;read: 1-1-0 s=3,c=1,t=16,pt=0,l=1:0\n'
b'1;3;1;0;16;0\n'

(this is a four sensor node - Temp, Humidity, light level and presence detection)

I am trying to test the pymysensors (mysensors.mysensors) and I believe that is where the situation may originate.

Basically, pymysensors just hangs there on calling gw.start() in the code below:


import mysensors.mysensors as mysensors
import logging

def event(type, nid):
    print(type+" "+str(nid))

gw = mysensors.SerialGateway('COM4', event)
logging.warning("Created SerialGateway")
print (gw)
gw.start()

Trying to wrangle through the pymysensors code...Could it be the windows threading implementation (gw.start()) causing some type of hung access to the serial port? Unfortunately, no messages (print or logging) that I put in the pymysensors actually get printed out (maybe due to being in a background thread?

martinhjelmare

For logging to work you have to set the level lower, and you should activate debug mode in pymysensors.

import mysensors.mysensors as mysensors
import logging

logging.basicConfig(level=logging.DEBUG)

def event(type, nid):
    print(type+" "+str(nid))

gw = mysensors.SerialGateway('COM4', event)
logging.warning("Created SerialGateway")
print (gw)

gw.debug = True

gw.start()

This is my console log when testing my fifo_queue branch.

In [1]: %paste
import mysensors.mysensors as mysensors
import logging

logging.basicConfig(level=logging.DEBUG)

def event(type, nid):
    print(type+" "+str(nid))

## -- End pasted text --

In [2]: gw = mysensors.SerialGateway('/dev/ttyUSB0', event)

In [3]: gw.debug = True

In [4]: gw.start()
INFO:mysensors.mysensors:Trying to connect to /dev/ttyUSB0

In [5]: INFO:mysensors.mysensors:/dev/ttyUSB0 is open...
INFO:mysensors.mysensors:Connected to /dev/ttyUSB0
INFO:mysensors.mysensors:n:0 c:0 t:3 s:9 p:gateway started, id=0, parent=0, distance=0

KeyboardInterrupt

In [5]: gw.stop()
INFO:mysensors.mysensors:Disconnecting from /dev/ttyUSB0
INFO:mysensors.mysensors:Stopping thread

Sometimes I get the following problem when I start the thread:

WARNING:mysensors.mysensors:Error decoding message from gateway, probably received bad byte.

A new gateway instance and restarting it always solves that. Sometimes messages seem to go out of sync, then it can be good to reconnect power to the gateway, also. But that has been very rare:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
    self.run()
  File "/home/martin/Dev/pymysensors/mysensors/mysensors.py", line 312, in run
    response = self.handle_queue()
  File "/home/martin/Dev/pymysensors/mysensors/mysensors.py", line 222, in handle_queue
    reply = func(*args, **kwargs)
  File "/home/martin/Dev/pymysensors/mysensors/mysensors.py", line 105, in logic
    msg = Message(data)
  File "/home/martin/Dev/pymysensors/mysensors/mysensors.py", line 397, in __init__
    self.decode(data)
  File "/home/martin/Dev/pymysensors/mysensors/mysensors.py", line 417, in decode
    self.sub_type) = [int(f) for f in data]
  File "/home/martin/Dev/pymysensors/mysensors/mysensors.py", line 417, in <listcomp>
    self.sub_type) = [int(f) for f in data]
ValueError: invalid literal for int() with base 10: 'read: 1-1-0 s=1,c=1,t=0'

humblehacker

@martinhjelmare said:

@humblehacker

Yes, it should be enabled by default, that's why I didn't mention it in my first advice. But it's good to double check anyway.

Are you sure you have two different instances of the Arduino IDE running when you test? Only one serial monitor can be active at a time, per IDE. It's not enough to open a second sketch window. You have to start the IDE twice from the start menu, or however you start it. Also, check the ports. You should see different headings in the serial monitor window, specifying the port.

What hardware are you using?

I had replied to this a couple days ago but it must not have posted so I apologize for the tardy response...
I think I might've narrowed down to what my problem is but first I'll reply to your question. My first test used two different instances of the IDE running on two different systems.
The gateway controller sketch was compiled on Windows 10 w/Arduino 1.6.6 and ran successfully on an Arduino UNO. The sensor/sender sketch was compiled on my Ubuntu MATE 15.10 laptop with Arduino 1.6.6 on an Arduino Leonardo.

Interestingly, I just tried swapping out the hardware so the Uno connected to Linux and the Leo connected to Windows. Sure enough, the Uno works as expected running both the sensor and the gateway sketch on Linux and Windows while the Leo won't work with any MySensors sketch on either Linux or Windows... So the problem may be related to the serial port on the ATMega32u4. As I mentioned earlier, I was able to get the Leo to work with the standard RF24 library, so it must be something about the Mysensors library that doesn't get along with the ATMega32u4 processors...
I'll try running my Arduino Nano now to see if it works with signals sent from my Uno.

Home Assistant - How to recognize MySensors nodes

8

11.7k

11.2k

113.0k