Raspberry Pi gateway + hass.io + MQTT + MySensors

[Sebex]

Ok so homeassistant is running and not crashing :) Now building the gateway, @monte to eventually communicate with hass.io do I need to select the gateway as --my-gateway=ethernet or mqtt?

[monte]

@sebex you can chose either. I prefer ethernet, so you don't rely on another piece of software - MQTT broker, no additional setup is required.
Read this docs especially the "Presentation" paragraph. At first I had trouble registering my nodes in HA, because it require particular way of presentation. "Send at least one initial value per V_TYPE. In version 2.x of MySensors, this has to be done in the loop function."

[Sebex]

@monte ok thanks, will take it into account.
Out of curiousity, since MQTT takes additional setup what''s the benefit of using MQTT? Do you know?

[monte]

@sebex I guess if you already have other mqtt services or devices in your system, or want to use a controller that is designed specifically around mqtt, nodered for example, or the one that doesn't have support for Mysensors specifically.
Also you could use Mysensors with remote mqtt server, your own, or third party. I tried this once, when I needed one sensor exposed to internet without dedicated server hardware at place, so I've built an esp8266 gateway with mqtt connected to free online mqtt broker. The nodered controller was also hosted on a free cloud :)
Not that it was particularly useful, or I would recommend it to someone, but it's a flexibility Mysensors has, and I guess it's the main reason so many people are using it for their projects.

[Sebex]

@monte ok it seems to work. Temp/humidity sensor is recognised by HA and data is getting through so looks good! Only connection isnt stable which is most likely due to that I still need to get capacitors for the NRF24L01+ on both ends, and need to solder the cables to the sensor. So still some tweaks to do but happy that it works, thanks for the support @monte :grinning:

Two general questions, any advice for which capacitor to select? Apart from 4.7 or 47uF, there's different types. Tanto, elco, bipolaire etc.
Furthermore I wondered why the 64bit hassio software didn't work on my Arduino, while it is a 64bit RPi 3+. Does Raspbian (or dockers?) as default run in 32?

[monte]

@sebex 4.7uF should be enough, I've never seen any problem with this value on my nodes. As for the type of the capacitor you probably will want to use ceramic or electrolytic (polarized) capacitor, but it doesn't do much difference for this application, if any. Those are just the most common types. Ceramic SMD capacitor of a size 0805 is the best choice IMO because it could be soldered directly to the VCC and GND pins on top of NRF24 module.
You can google about why raspbian is running like RPi is 32 bits, but as I understood it's better for downward compatibility and the fact that RPi3 won't get much boost (if any) from switching to 64 bit architecture.

[Sebex]

@monte ah smart makes sense and probably looks most neat as well. Thanks!

[Sebex]

Ok so I am experiencing node loss. After a while my sensor node is not recognized anymore by the gateway/mothernode. It seems to be related to the persistence file that is not accessible although the user account should have the correct rights, not sure how to verify. See HA dev logs below.

Developer Tools
Permission denied when writing to /config/home/pi/.homeassistant/mysensors.json
10:13 PM /usr/local/lib/python3.7/site-packages/mysensors/persistence.py (ERROR) - message first occurred at 10:06 PM and shows up 42 times
Node 1 is unknown
10:11 PM __main__.py (WARNING) - message first occurred at 10:06 PM and shows up 116 times
Failed to load sensors from file: home/pi/.homeassistant/mysensors.json
10:06 PM /usr/local/lib/python3.7/site-packages/mysensors/persistence.py (WARNING)
File does not exist or is not readable: home/pi/.homeassistant/mysensors.json.bak
10:06 PM /usr/local/lib/python3.7/site-packages/mysensors/persistence.py (WARNING)
Trying backup file: home/pi/.homeassistant/mysensors.json.bak
10:06 PM /usr/local/lib/python3.7/site-packages/mysensors/persistence.py (WARNING)
File does not exist or is not readable: home/pi/.homeassistant/mysensors.json
10:06 PM /usr/local/lib/python3.7/site-packages/mysensors/persistence.py (WARNING)```

# MySensors gateway
mysensors:
 gateways: 
  - device: '192.168.178.27'
    persistence_file: 'home/pi/.homeassistant/mysensors.json'
    tcp_port: 5003
 persistence: true  

logger:
  default: info
  logs:
    homeassistant.components.mysensors: debug
    mysensors: debug

@drock1985 saw your post in another thread related to this, do you have a suggestion?

[monte]

@Sebex you are using Hassio, so there is no /home/pi folder. Try persistence_file: 'mysensors.json'

[Sebex]

@monte seems to do the trick :grin: thanks again!!

[Sebex]

@monte, I'm seeing that a HA update is available. Since I'm using HA via docker the following applies to update.
$ sudo docker pull homeassistant/home-assistant:latest
To be on the safe side I'd like to backup my HA and/or RPi first to prevent me from losing all my settings. In your experience what's the most convenient for doing this. I did make a clone from the sd card a few weeks ago but it's not the most labour efficient method, do you make regular backups?

[monte]

@Sebex if you installed Hassio with provided script then you can find your Home Assistant config folder in /usr/share/hassio. You can also change this path to your liking in /etc/hassio.json config file.
To backup all of your HA settings and setups you need to just copy this folder. That's why I like idea of using Docker. You can lose all your RPi setup, but if you fave backed up your config folder it only matter of half an hour to recover everything from scratch.

[Sebex]

@monte can you copy this over ssh? tried using scp -r, which seemed to work but can't find the downloaded files somehow.
And in regard of updating HA(within UI I see that and update is available). I can just stop the container and run a new updated version right, without it affecting config files?

Furthermore I wondered about HA speed, if I want to look at the history of a sensors it takes quite a while to poll the data and create a graph. Is this subsequent to HA/rpi or can the performance be improved somehow?

[monte]

@Sebex yes, you can copy config over ssh, like every other directory you have access to. It should be located from where you were running scp command. Do you use linux on your desktop/laptop?
You can also create samba share on your raspberry with location of your config folder, and then access it from any pc on the network, it makes easy to config your HA.

Yes you can pull newer version of a container and all your config files should be in place.

Can't help you with UI speed, sorry. You can try to find anything on HA forum, maybe there is someone who made some performance tests.

[Sebex]

@monte thanks it worked using Samba. I use ssh from my mac. I also found the copied folder that I tried to copy with scp :laughing:

I tried pulling a new version but that didn't seem to work so easily. It seems to get stuck on initialising after which it shuts down and tries again. The logs show the error below, it's missing a file/directory. Not sure how to solve this.

2020/01/23 17:13:08 [error] 7#7: *1 open() "/usr/share/www/api/websocket" failed (2: No such file or directory), client: ::ffff:192.168.178.16, server: , request: "GET /api/websocket HTTP/1.1", host: "192.168.178.27:8123"
2020/01/23 17:13:15 [error] 7#7: *3 open() "/usr/share/www/api/websocket" failed (2: No such file or directory), client: ::ffff:192.168.178.16, server: , request: "GET /api/websocket HTTP/1.1", host: "192.168.178.27:8123"
2020/01/23 17:13:21 [error] 7#7: *4 open() "/usr/share/www/api/websocket" failed (2: No such file or directory), client: ::ffff:192.168.178.16, server: , request: "GET /api/websocket HTTP/1.1", host: "192.168.178.27:8123"

What I did to update the container is stop all running containers, and remove images, and re-execute the cmd for homeassistant again. Was this the way to go?

Also is there a way to copy the RPi settings? I quite often get a broken pipe and not sure what's causing it. I am thinking about reflashing the SD and reinstating the settings.

[monte]

@Sebex the best way to backup your Raspbian setup is to copy whole SD card to image file, apparently you can use mac's Disk Utility to do this.

I'm not sure how home assistant supervisor pulls and runs docker container, but it seems that it have some trouble. You can run setup script once again, maybe it will help.

For future, you should update via web ui. I've read that hassio images are updated later than standalone home assistant once. So you won't get update right away, when new version of HA is released. But when new version of hassio is released it will suggest you to update in Hass.io dashboard.

[Sebex]

@monte I had tried that one before, but know that it's useful to restore the file doesn't seem to work and I get an error. Perhaps bad luck, will have to try again.

In the meantime while reinstating HA I'm facing quite some difficulties in getting HA to install within docker via link. I'm facing a broken pipe quite often and have no idea how to fix this, tried different ssh settings but when installing HA it seems to get stuck at

pi@raspberrypi:~ $ sudo su
root@raspberrypi:/home/pi# curl -sL https://raw.githubusercontent.com/home-assistant/hassio-installer/master/hassio_install.sh | bash -s -- -m raspberrypi3
[Warning] No NetworkManager support on host.
[Info] Install supervisor Docker container
packet_write_wait: Connection to fe80::233b:33dc:8d3e:eb75%en0 port 22: Broken pipe

Yesterday it did pass but then hassio had troubles successfully launching. It got the below error in the logs, not sure what caused it.
error /usr/share/www/manifest.json

Now trying to install hassio again, any suggestions are welcome. I'm going to try a different SD card and wifi connection instead of ethernet.

[monte]

@Sebex you could try connecting a monitor and keyboard directly to RPi and running install script. Broken pipe means something is braking connection to ssh host. I can't really say what it is for sure, maybe just excessive CPU usage, or running out of ram? But that would be strange.
If you have backed up you HA setup, why not just flash clean Raspbian image and run install script on it. If it worked before, I don't see why it wouldn't work again.

And that piece of output you wrote, suggests you are using IPv6 for ssh connection. Is there any reason for this?

[Sebex]

@monte good idea I'll try that.

Regarding IPv6, there's no particular reason and I haven't noticed actually. I have been using ssh pi@raspberrypi.local to login, I can also connect directly to the IPv4 address. You think that can resolve it?

[monte]

@Sebex okay, I've googled for you, there are plenty reportings of similar issues. It may be caused by insufficient power supply. On intense CPU usage it may lack current. Try getting better power supply rated at 2.5A, 5V.