Home Assistant - How to recognize MySensors nodes

humblehacker

Hello,
I've been trying to get a MSG set up with HA for a couple days now with no luck. I've tried the recommendations made above and tried turning persistence on and off and while Ive been able to confirm HASS has registered the component, I still can't get anything to show up in the log... Any other suggestions about how I can debug? Another user made a comment about there being a MySensors MQTT client gateway available for HA but that was the only time Ive seen it mentioned. Is this another possibility?

martinhjelmare

@humblehacker

Hi!

There is general MQTT support in home-assistant, not specific to mysensors, but I have too little experience using the MQTT features in mysensors, to say if it will work or not. I guess it depends on how the mysensors protocol is translated by the MQTT gateway and posted to the broker. If you want to try it, I suggest using the dev branch of mysensors and following this guide:

http://forum.mysensors.org/topic/2352/guide-setting-up-and-testing-mqtt-client-gateway

Here is the home-assistant docs:
https://home-assistant.io/components/mqtt/

Looking at the topic structure and looking at the docs at home-assistant, I'm optimistic. Maybe I'll even try it myself.

Here are some advice if you want to continue trying the serial gateway approach:
Are you using the latest release of home-assistant (0.8)? That release supports debug level logging as default, which should make the debug mode of the mysensors component work. That was not the case before.

First, make sure your gateway and test node is talking without the controller. Hook up serial monitors and check the output on both sides. I'd be happy to look at that if you want to post it here.

Then, hook up your serial gateway to the home-assistant computer. Start home-assistant from the command line: (hass). Check the log output. Post it here if you want.

Then, powercycle your node. Make sure you have the serial monitor up on the node, or just reconnect it, and the node should reset. Look at the serial output on the node side, and check the log from hass, if there is anything. Post it here if you want. You can't have the serial monitor on the gateway active when starting hass, as the serial connection is exclusive.

Edit:
Also post your home-assistant config if you want, which is a crucial piece of evidence.

humblehacker

Are you using the latest release of home-assistant (0.8)? That release supports debug level logging as default, which should make the debug mode of the mysensors component work. That was not the case before.

Yes I can confirm I am running version 0.8 and having just reread your above post, I realized changing debug to info in mysensors.py only applies to >v0.8 so I had to change it back and reboot again. I"ve been using the terminal output from hass -ui as my main debugger and through this was able to confirm that the mysensors component is recognized. Also troubleshooting on the Home Assistant gitter page over the past few days and tried enabling and disabling persistence from the config.yaml.

First, make sure your gateway and test node is talking without the controller. Hook up serial monitors and check the output on both sides. I'd be happy to look at that if you want to post it here.
Then, hook up your serial gateway to the home-assistant computer. Start home-assistant from the command line: (hass). Check the log output. Post it here if you want.
Then, powercycle your node. Make sure you have the serial monitor up on the node, or just reconnect it, and the node should reset. Look at the serial output on the node side, and check the log from hass, if there is anything. Post it here if you want. You can't have the serial monitor on the gateway active when starting hass, as the serial connection is exclusive.

OK yes. This is the part I've really needed help with as I can't seem to find any documentation from the MySensors end about debugging via serial monitor. I tried opening a serial monitor as I normally would with the Arduino IDE and nothing happened but I dont know if the MySensors library uses a custom protocol or baud rate. Is there a tutorial or guide on the subject that you can point me to? I can connect an OLED and/or LCD screen directly too if that is possible.

martinhjelmare

For debugging with serial monitor, just plug in both the gateway and the node to your computer, either through USB if your arduino supports that or through FTDI adapter. Then open two instances of the Arduino IDE and open one serial monitor per IDE. Make sure you set board and port correct first, in the two IDEs, one per arduino. Mysensors use baudrate 115200, so make sure that's selected in the serial monitors.

For an explanation of the mysensors' serial api, see:
http://www.mysensors.org/download/serial_api_15

In the end of that page there are some examples of serial output.

humblehacker

Yes, it was just the baud rate that had me thrown off I guess, since the default is always 9600 on Arduino. Anyway, the serial gateway seems to be functioning as expected printing 0;0;3;0;9;gateway started, id=0, parent=0, distance=0 0;0;3;0;14;Gateway startup complete.

However when I load up the motion sensor sketch and open the monitor, it still shows nothing. I then tried a couple different sensors incl. a PIR sensor with the distance example and another with the ds18b20 temp sensor with the same result. Finally I loaded up the MockMySensors sketch and again the serial monitor was blank.
I then loaded up the default DigitalSerialRead sketch from the Arduino IDE and the serial output printed as expected. Then lastly, I even tested the PhysicalPixel sketch with the standard NRF24 library and it transmitted as expected! So the issue is definitely with the MySensors library. But then I still couldnt even get it to work using Codebender and that should be using a completely different library file. SO I'm all out of ideas... In any case, there still could be an issue on the Controller end, but I have no way of knowing until I can fix the sender function.

martinhjelmare

Have you activated (uncommented) DEBUG in MyConfig.h?

humblehacker

As I keep my sketchbook synchronized through Github, I can confirm that debugging is enabled in my master here
However, I will need to confirm that the local sketchbook on the laptop I was using yesterday is up to date with the remote master. I'm pretty sure I enabled debugging when I first installed it.

EDIT- Isn't debugging enabled by default now?- https://github.com/mysensors/Arduino/blob/master/libraries/MySensors/MyConfig.h

acmbc68

Hi,

I've been following this thread for a few days, as I am in the same situation and kielnino - I can see the MySensors component get loaded and then....silence....I know the MySensors GW and nodes are working from testing with serial monitors and another controller app (MySController).

Will probably be spending some time tracing through the pymysensors code tonight to see what gives. I do have debug enabled on homeassistant, but still no joy.

One thought - shouldn't MySensors on homeassistant be implemented as a platform (and not a sensor) much like MQTT.

I have got readings to work through MQTT (in my case, patched a node-red flow together to read the serial line and publish to a local mosquitto), but this is like driving around the city to get to the store next door...

Any further thoughts/hints as to where to start looking?

Thanks!

martinhjelmare

@humblehacker

Yes, it should be enabled by default, that's why I didn't mention it in my first advice. But it's good to double check anyway.

Are you sure you have two different instances of the Arduino IDE running when you test? Only one serial monitor can be active at a time, per IDE. It's not enough to open a second sketch window. You have to start the IDE twice from the start menu, or however you start it. Also, check the ports. You should see different headings in the serial monitor window, specifying the port.

What hardware are you using?

martinhjelmare

@acmbc68

Hi!

Regarding the current mysensors implementation in home assistant, it's a sensor platform. But it's not a separate component.

I have added a mysensors component together with a switch platform, in my github fork of home assistant. I haven't made a pull request yet, cause I'm waiting to have my other pull request to pymysensors approved, where I implement support for setting child values from the controller, cause it's a dependency.

https://github.com/MartinHjelmare/home-assistant/tree/mysensors-component-switch

https://github.com/MartinHjelmare/pymysensors/tree/fifo_queue

I'm not completely finished with the home assistant switch branch. I want to add support for multiple gateways as well, and I haven't had time to test my solution properly yet.

I've added more debug logging in my fifo_queue branch of pymysensors, and made the serial connection more robust, by closing it when the thread quits. I've also downgraded the required pyserial version to 2.5, due to a bug in pyserial 2.6 - 3. This has been fixed in the master branch of pyserial, but no new release exists, I think.

If you know how to change installed components in home assistant, you can give my branches a try. They should be working in their current state on github, but more testing is always good.

When I first started testing connecting my node with the gateway and home assistant, I had problems in the beginning, When I set the node id to 1 and made sure at least one value was sent from the node, I got it working. It might have been a coincidence, that it started working then, cause I'm not sure that home assistant needs sequential entity ids, but when you're debugging it's good to narrow down possible variables/problems. You also have to make sure that the gateway receives all the presentation data, including sketch name, from the node. Otherwise the sensor won't show up in the GUI. That's currently how it's written in the code.

Edit:
One more thing. Even though debug level logging should be enabled by default in home assistant 0.8, you still have to enable the logger component in your config.
https://home-assistant.io/components/logger/

acmbc68

Thanks @martinhjelmare

As per https://home-assistant.io/blog/posts/2/, all the components should be under the homeassistant install, so it shouldn't be a brain-wrecker to change those out.

I'll start giving it a whirl tonight.

acmbc68

Ok.
First of all - still no joy. Can't get any communications from MySensors to homeassistant.

Trying to tackle this bottom up. Thought it might be an issue with PySerial, but that reads data OK from the serial port:

import serial as serial
print ("starting")
ser = serial.Serial (
    port='COM4',
    baudrate=115200,
    parity=serial.PARITY_NONE,
    stopbits=serial.STOPBITS_ONE,
    bytesize=serial.EIGHTBITS
)

ser.flushInput()
ser.flushOutput()

print("connected to: " + ser.portstr)
count=1

while True:
        try:
                data_raw = ser.readline()
                print(data_raw)
        except serial.SerialException:
                print ("SERIAL ERROR")
                exit(1)

This results in what is expected:

starting
connected to: COM4
b'0;0;3;0;14;Gateway startup complete.\n'
b'0;0;3;0;9;read: 1-1-0 s=2,c=1,t=23,pt=2,l=2:0\n'
b'1;2;1;0;23;0\n'
b'0;0;3;0;9;read: 1-1-0 s=1,c=1,t=0,pt=7,l=5:25.0\n'
b'1;1;1;0;0;25.0\n'
b'0;0;3;0;9;read: 1-1-0 s=0,c=1,t=1,pt=7,l=5:84.0\n'
b'1;0;1;0;1;84.0\n'
b'0;0;3;0;9;read: 1-1-0 s=3,c=1,t=16,pt=0,l=1:0\n'
b'1;3;1;0;16;0\n'

(this is a four sensor node - Temp, Humidity, light level and presence detection)

I am trying to test the pymysensors (mysensors.mysensors) and I believe that is where the situation may originate.

Basically, pymysensors just hangs there on calling gw.start() in the code below:


import mysensors.mysensors as mysensors
import logging

def event(type, nid):
    print(type+" "+str(nid))

gw = mysensors.SerialGateway('COM4', event)
logging.warning("Created SerialGateway")
print (gw)
gw.start()

Trying to wrangle through the pymysensors code...Could it be the windows threading implementation (gw.start()) causing some type of hung access to the serial port? Unfortunately, no messages (print or logging) that I put in the pymysensors actually get printed out (maybe due to being in a background thread?

martinhjelmare

For logging to work you have to set the level lower, and you should activate debug mode in pymysensors.

import mysensors.mysensors as mysensors
import logging

logging.basicConfig(level=logging.DEBUG)

def event(type, nid):
    print(type+" "+str(nid))

gw = mysensors.SerialGateway('COM4', event)
logging.warning("Created SerialGateway")
print (gw)

gw.debug = True

gw.start()

This is my console log when testing my fifo_queue branch.

In [1]: %paste
import mysensors.mysensors as mysensors
import logging

logging.basicConfig(level=logging.DEBUG)

def event(type, nid):
    print(type+" "+str(nid))

## -- End pasted text --

In [2]: gw = mysensors.SerialGateway('/dev/ttyUSB0', event)

In [3]: gw.debug = True

In [4]: gw.start()
INFO:mysensors.mysensors:Trying to connect to /dev/ttyUSB0

In [5]: INFO:mysensors.mysensors:/dev/ttyUSB0 is open...
INFO:mysensors.mysensors:Connected to /dev/ttyUSB0
INFO:mysensors.mysensors:n:0 c:0 t:3 s:9 p:gateway started, id=0, parent=0, distance=0

KeyboardInterrupt

In [5]: gw.stop()
INFO:mysensors.mysensors:Disconnecting from /dev/ttyUSB0
INFO:mysensors.mysensors:Stopping thread

Sometimes I get the following problem when I start the thread:

WARNING:mysensors.mysensors:Error decoding message from gateway, probably received bad byte.

A new gateway instance and restarting it always solves that. Sometimes messages seem to go out of sync, then it can be good to reconnect power to the gateway, also. But that has been very rare:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
    self.run()
  File "/home/martin/Dev/pymysensors/mysensors/mysensors.py", line 312, in run
    response = self.handle_queue()
  File "/home/martin/Dev/pymysensors/mysensors/mysensors.py", line 222, in handle_queue
    reply = func(*args, **kwargs)
  File "/home/martin/Dev/pymysensors/mysensors/mysensors.py", line 105, in logic
    msg = Message(data)
  File "/home/martin/Dev/pymysensors/mysensors/mysensors.py", line 397, in __init__
    self.decode(data)
  File "/home/martin/Dev/pymysensors/mysensors/mysensors.py", line 417, in decode
    self.sub_type) = [int(f) for f in data]
  File "/home/martin/Dev/pymysensors/mysensors/mysensors.py", line 417, in <listcomp>
    self.sub_type) = [int(f) for f in data]
ValueError: invalid literal for int() with base 10: 'read: 1-1-0 s=1,c=1,t=0'

humblehacker

@martinhjelmare said:

@humblehacker

Yes, it should be enabled by default, that's why I didn't mention it in my first advice. But it's good to double check anyway.

Are you sure you have two different instances of the Arduino IDE running when you test? Only one serial monitor can be active at a time, per IDE. It's not enough to open a second sketch window. You have to start the IDE twice from the start menu, or however you start it. Also, check the ports. You should see different headings in the serial monitor window, specifying the port.

What hardware are you using?

I had replied to this a couple days ago but it must not have posted so I apologize for the tardy response...
I think I might've narrowed down to what my problem is but first I'll reply to your question. My first test used two different instances of the IDE running on two different systems.
The gateway controller sketch was compiled on Windows 10 w/Arduino 1.6.6 and ran successfully on an Arduino UNO. The sensor/sender sketch was compiled on my Ubuntu MATE 15.10 laptop with Arduino 1.6.6 on an Arduino Leonardo.

Interestingly, I just tried swapping out the hardware so the Uno connected to Linux and the Leo connected to Windows. Sure enough, the Uno works as expected running both the sensor and the gateway sketch on Linux and Windows while the Leo won't work with any MySensors sketch on either Linux or Windows... So the problem may be related to the serial port on the ATMega32u4. As I mentioned earlier, I was able to get the Leo to work with the standard RF24 library, so it must be something about the Mysensors library that doesn't get along with the ATMega32u4 processors...
I'll try running my Arduino Nano now to see if it works with signals sent from my Uno.

humblehacker

UPDATE:
YES!!! The Uno connected to the motion sensor is transmitting data to my Arduino Nano gateway and I can finally see the data on the serial monitor! Now I just need to reconnect the gateway to Home Assistant to see what it picks up.
Lesson learned as of the present is to only use ATMega328 processors like the Uno, Nano, Pro Mini etc... Most of my controllers at this point are based on the 32u4 so it would be good if we could figure out why the Leo boards don't work with MYS... Has anybody else experienced this problem?

humblehacker

Sure enough-

WARNING:mysensors.mysensors:Error decoding message from gateway, probably received bad byte.

@martinhjelmare- How do I create a new gateway instance? Simply restart?

martinhjelmare

@humblehacker

If using home assistant, it's enough to restart home assistant, to get a new gateway instance and reconnect.

humblehacker

success!.png
:smiley:
Other than the fact that my log seems to think its yesterday... I just need to find out why I can't use MySensors with Any ATMega32u4 devices... Attiny85/Trinket support would be nice too but I dont need to push my luck...

martinhjelmare

@humblehacker

:thumbsup:

kielnino

I'm still stuck with the import of sensors to home-assitant. Can someone please post his configuration.yaml? I Wonder if I made a stupid configuration error there.
Do I need the "inclusion mode" of the Gateway or does home-assistant handle the detection of new sensors? In the serial monitor all the communication between the nodes and the Gateway is looking good, but in home-assistant they won't show up.

Home Assistant - How to recognize MySensors nodes

21

11.7k

11.2k

113.1k