Dual Radio with Home Assistant Question

[ejlane]

Hello everyone,

This isn't exactly troubleshooting, but it seemed to fit in this category best. I've built a dual-radio gateway on a Raspberry Pi Zero as on here: https://www.openhardware.io/view/470/MySensors-Gateway-for-Raspberry-PI

I compile the software twice, once for each radio, and that all works just fine. However, I can't seem to get Home Assistant to use two different mqtt gateways. Is that just impossible? The Mqtt broker is on the same Raspberry Pi as the Home Assistant, which is on a RPi 3. If I have two entries in the configuration.yaml file, then it only seems to ever acknowledge one of them.

I'm open to switching away from mqtt, but it looked like it might be the best fit. My main desires are to keep the communication between the gateway and Home Assistant only on wires, so they are both on my ethernet network. I'm actually also open to modifying things so that everything is done only on the RPi 3. (That might actually be preferable, but since most things don't expect that, I didn't want to introduce extra unnecessary effort.) I'm using hassio right now, I think, so modifying or getting sudo/CLI access might be an issue.

Now, my main reason for wanting dual radios is that I want to use the RFM69 for it's lower frequency/longer range, as well as hardware encryption for all of my outdoor sensors. (Mainly garden stuff, at least for now.) My indoor things are going to be based on the nrf51. It seems plenty powerful, and is available in cheap modules with all the pins broken out. What I really don't want is for there to be any way for an outdoor sensor to masquerade as an indoor one. Like if there's something on my garage door, absolutely nothing that the outdoor sensor every sends should be possible to trigger the door to open. I thought having there be two sensor nets, and having them be separate by external/internal would be good safety, and keep physical access to the radios an impossibility as far as hacking the protocol for the indoor ones.

This way I don't have to worry if an outdoor one ever disappears, because worst case is it could turn on a sprinkler or lie about some plants being dry, etc. I mean, it's not like I have much valuable here that anyone is going to go to great efforts to hack into things, but I just don't like the idea of leaving things lying around that could wreak havoc if someone knew what they were doing. I'll sleep better this way.

Okay, so with that in mind, am I going about this the right way? I am open to writing a script that checks every incoming message from the outdoor radio gateway to ensure that only the whitelisted messages get passed along to Home Assistant. I just don't know enough about how any of the gateways are integrated into Home Assistant to have any idea whether I'm going about this in a good way or not. I don't really care what transport is used between the gateway and Home Assistant, other than it should be on a wire. Maybe something like what I want, or at least pieces of it, have already been thought of by someone else and they're already part of the system, but I'm just doing things wrong?

Thank you!

[electrik]

@ejlane
My configuration is this, and that seems to work correctly:

mysensors:
  gateways:
    - device: mqtt
      persistence_file: '/config/mysensors-mqtt.json'
      topic_in_prefix: 'mygateway1-out'
      topic_out_prefix: 'mygateway1-in'
    - device: mqtt
      persistence_file: '/config/mysensors-rfm-mqtt.json'
      topic_in_prefix: 'ms-rfm-from'
      topic_out_prefix: 'ms-rfm-to'
  optimistic: false
  persistence: true
  retain: false
  version: '2.3'

What exactly doesn't work in your case, and can you post your configuration?

[ejlane]

@electrik Hmmm, that looks very similar to how I had my first configuration. Unfortunately, I've tried a few different things since then, so I'm not sure. I will have to sit down and try getting that all set up again and see what happens.

Once I've done that, I can share the configuration and details of error messages, if any. Maybe I just did something stupid.

But in trying all that and the reading I did afterwards, it looked like Home Assistant would only accept a single mqtt connection/gateway, so I thought that my config was overwriting the settings by having two entries, so I changed it to both go to a single one. Thank you for letting me know otherwise!

[ejlane]

Well, it's still failing.

I'm using the current version 5.1 of the Mosquitto broker add-on to Home Assistant. In the configuration page I have this:

logins:
  - username: myOutdoorGateway
    password: outdoorPassword
  - username: myIndoorGateway
    password: indoorPassword
anonymous: false
customize:
  active: false
  folder: mosquitto
certfile: fullchain.pem
keyfile: privkey.pem
require_certificate: false

In the configuration.yaml file I have this:

# Configure a default setup of Home Assistant (frontend, api, etc)
default_config:

# Uncomment this if you are using SSL/TLS, running in Docker container, etc.
# http:
#   base_url: example.duckdns.org:8123

# Text to speech
tts:
  - platform: google_translate

group: !include groups.yaml
automation: !include automations.yaml
script: !include scripts.yaml
scene: !include scenes.yaml

mqtt:
  broker: 172.17.0.1

mysensors:
  gateways:
    - device: mqtt
      persistence_file: 'mysensors/myoutdoorsensors.json'
      topic_in_prefix: 'myOutdoorGateway-out'
      topic_out_prefix: 'myOutdoorGateway-in'
    - device: mqtt
      persistence_file: 'mysensors/myindoorsensors.json'
      topic_in_prefix: 'myIndoorGateway-out'
      topic_out_prefix: 'myIndoorGateway-in'
  optimistic: false
  persistence: true
  retain: false
  version: '2.3'

homeassistant:
  customize: !include customize.yaml

Both of the json files are there in the mysensors folder, but they are just empty.

I compiled the two gateways with:

./configure --my-transport=rf24 --my-rf24-irq-pin=33 --my-rf24-ce-pin=37 --my-rf24-cs-pin=36 --spi-spidev-device=/dev/spidev1.0 --spi-driver=SPIDEV  --my-gateway=mqtt --my-controller-ip-address=192.168.0.49 --my-mqtt-publish-topic-prefix=myIndoorGateway-out --my-mqtt-subscribe-topic-prefix=myIndoorGateway-in --my-mqtt-client-id=myIndoorGateway --my-leds-err-pin=29 --my-leds-rx-pin=31 --my-leds-tx-pin=32 --my-rf24-encryption-enabled --my-signing=password --my-security-password=indoors --my-mqtt-password=indoorPassword

./configure --my-transport=rfm69 --my-rfm69-frequency=915 --my-is-rfm69hw --my-gateway=mqtt --my-controller-ip-address=192.168.0.49 --my-mqtt-publish-topic-prefix=myOutdoorGateway-out --my-mqtt-subscribe-topic-prefix=myOutdoorGateway-in --my-mqtt-client-id=myOutdoorGateway --my-leds-err-pin=12 --my-leds-rx-pin=16 --my-leds-tx-pin=18 --my-signing=password --my-security-password=outdoors --my-rfm69-encryption-enabled --my-mqtt-password=outdoorPassword

However, for both gateways, when I run the test everything looks good until it tries to connect to the mqtt server.

pi@mysensors-gateway:~/MySensorsSecond $ ./bin/mysgwIndoor
Sep 14 11:53:56 INFO  Starting gateway...
Sep 14 11:53:56 INFO  Protocol version - 2.4.0-alpha
Sep 14 11:53:56 DEBUG MCO:BGN:INIT GW,CP=RNNGLSQX,FQ=NA,REL=0,VER=2.4.0-alpha
Sep 14 11:53:56 DEBUG TSF:LRT:OK
Sep 14 11:53:56 DEBUG TSM:INIT
Sep 14 11:53:56 DEBUG TSF:WUR:MS=0
Sep 14 11:53:56 DEBUG TSM:INIT:TSP OK
Sep 14 11:53:56 DEBUG TSM:INIT:GW MODE
Sep 14 11:53:56 DEBUG TSM:READY:ID=0,PAR=0,DIS=0
Sep 14 11:53:56 DEBUG MCO:REG:NOT NEEDED
Sep 14 11:53:56 DEBUG MCO:BGN:STP
Sep 14 11:53:56 DEBUG MCO:BGN:INIT OK,TSP=1
Sep 14 11:53:56 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:53:56 DEBUG connected to 192.168.0.49
Sep 14 11:53:57 DEBUG !GWT:RMQ:FAIL
Sep 14 11:53:57 DEBUG TSM:READY:NWD REQ
Sep 14 11:53:57 DEBUG ?TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Sep 14 11:53:57 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:53:57 DEBUG connected to 192.168.0.49
Sep 14 11:53:58 DEBUG !GWT:RMQ:FAIL
Sep 14 11:53:58 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:53:58 DEBUG connected to 192.168.0.49
Sep 14 11:53:59 DEBUG !GWT:RMQ:FAIL
Sep 14 11:53:59 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:53:59 DEBUG connected to 192.168.0.49
Sep 14 11:54:00 DEBUG !GWT:RMQ:FAIL
Sep 14 11:54:00 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:54:00 DEBUG connected to 192.168.0.49
Sep 14 11:54:01 DEBUG !GWT:RMQ:FAIL
Sep 14 11:54:01 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:54:01 DEBUG connected to 192.168.0.49

and

sudo ./bin/mysgw
Sep 14 11:43:26 INFO  Starting gateway...
Sep 14 11:43:26 INFO  Protocol version - 2.4.0-alpha
Sep 14 11:43:26 DEBUG MCO:BGN:INIT GW,CP=RPNGLS-X,FQ=NA,REL=0,VER=2.4.0-alpha
Sep 14 11:43:26 DEBUG TSF:LRT:OK
Sep 14 11:43:26 DEBUG TSM:INIT
Sep 14 11:43:26 DEBUG TSF:WUR:MS=0
Sep 14 11:43:26 DEBUG TSM:INIT:TSP OK
Sep 14 11:43:26 DEBUG TSM:INIT:GW MODE
Sep 14 11:43:26 DEBUG TSM:READY:ID=0,PAR=0,DIS=0
Sep 14 11:43:26 DEBUG MCO:REG:NOT NEEDED
Sep 14 11:43:26 DEBUG MCO:BGN:STP
Sep 14 11:43:26 DEBUG MCO:BGN:INIT OK,TSP=1
Sep 14 11:43:26 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:43:26 DEBUG connected to 192.168.0.49
Sep 14 11:43:27 DEBUG !GWT:RMQ:FAIL
Sep 14 11:43:27 DEBUG TSM:READY:NWD REQ
Sep 14 11:43:27 DEBUG ?TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Sep 14 11:43:27 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:43:27 DEBUG connected to 192.168.0.49
Sep 14 11:43:28 DEBUG !GWT:RMQ:FAIL
Sep 14 11:43:28 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:43:28 DEBUG connected to 192.168.0.49
Sep 14 11:43:29 DEBUG !GWT:RMQ:FAIL
Sep 14 11:43:29 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:43:29 DEBUG connected to 192.168.0.49
Sep 14 11:43:30 DEBUG !GWT:RMQ:FAIL
Sep 14 11:43:31 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:43:31 DEBUG connected to 192.168.0.49

So they both fail with the same error message of !GWT:RMQ:FAIL but I can't get the log parser to tell me what that means. From this page: https://www.mysensors.org/apidocs/group__MyGatewayTransportgrp.html it looks like it's just a generic failure, so I don't know what I have wrong.

In the log file for the MQTT broker, all I see is a bunch of messages about a socket error. They look like this, if it matters:

1600109639: New connection from 192.168.0.48 on port 1883.
1600109639: Socket error on client <unknown>, disconnecting.

Does this mean something to you? Any pointers for where next to look?

Thanks so much!

[mfalkvidd]

@ejlane this might be a shot in the dark, but could you try using different config files for the two gateways? By default, they use /etc/mysensors.conf which means (among other things) they will use the same file for emulating eeprom (specifid in the config file), and sharing the same eeprom file could cause all sorts of strange random stuff.

[ejlane]

@mfalkvidd
I went ahead and tried it. But right now I only have one running at a time, so they shouldn't be able to interfere, should they? (I haven't done the 'make install' on either one, I'm just running them manually while I test things out.)

Anyway, no difference. I tried setting anonymous to true in the Mosquitto broker settings, and that allowed me to then connect. Now I have a new error:

Sep 14 14:19:01 DEBUG GWT:RMQ:CONNECTING...
Sep 14 14:19:01 DEBUG connected to 192.168.0.49
Sep 14 14:19:01 DEBUG GWT:RMQ:OK
Sep 14 14:19:01 DEBUG GWT:TPS:TOPIC=myIndoorGateway-out/0/255/0/0/18,MSG SENT
Sep 14 14:19:01 DEBUG !MCO:PRO:RC=1
Sep 14 14:19:01 DEBUG !MCO:PRO:RC=1

According to the log parser, https://www.mysensors.org/build/parser?log=!MCO%3APRO%3ARC%3D1 this is a recursive call. So I still have something wrong.

I'll keep working on that. But it now looks like my issues are primarily not a mysensors thing, and some kind of problem where Mosquitto is not accepting the username/password combo that it should be.

Can anyone share their settings for that, from the Mosquitto settings page of Home Assistant? Maybe I have a formatting or spelling error in there.

[ejlane]

I'm having no luck getting it to work with a username and password to connect to the mqtt broker. Well, I can get the Home Assistant user to connect, but I can't get the mysensors gateway to connect at all.

It just keeps telling me that it failed, and in the mqtt logs, it says that it was an unauthorized user.

I've tried all kinds of varieties, and I just can't get it to connect.

I captured the outgoing packets from the gateway and put them into wireshark, and I never see a password sent. There are also "connect" flags in the header that aren't set for user name and password. See this page for what I was comparing to: https://dzone.com/articles/dissecting-mqtt-using-wireshark On that page the flags are set, and the password is also visible.

If it's really not sending the password then there's no way that mosquitto will authorize it. Now I just need to find out what I'm doing wrong with the build flags that is stopping it from sending passwords out.

[ejlane]

I might have it figured out. At least so far the one is connecting and not showing any errors. Now that I have it this far, I need to program a sensor and see if I can get it to connect and show up in the Home Assistant dashboard. So it might still be a while before I know for sure.

But I was browsing the mqtt transport source code at https://github.com/mysensors/MySensors/blob/development/core/MyGatewayTransportMQTTClient.cpp when I saw something that surprised me. I saw a function that had both the client-id and username as parameters. This tipped me off that I should look at that closer, because I had seen something about both of them, but in the back of my mind just assumed they were the same thing.

I went and looked in my build flags, and I didn't have the --my-mqtt-user anywhere. After adding it, I can now run the gateway code and it connects and just waits for activity, without throwing any errors. So this feels like good progress, and that this part is solved.

However, the indoor gateway is still showing the errors from before about a recursive problem. https://www.mysensors.org/build/parser?log=!MCO%3APRO%3ARC%3D1

How do I track down more info about this? Is browsing the source the only option? Is there a way for it to provide more details about just what is going on without editing the source code?

Thanks!

[ejlane]

I've found it in the source code, and it looks like it's just reporting that a recursion happened. Since the recursion level never gets above 1 in all the messages that I see, I'm going to ignore it for now and just keep it in mind in case I run into weird things in the future.

[ejlane]

@electrik

Well, I've made a bunch of progress, and found a few small errors. I simplified everything down to just using the 'Dimmable Light' from the Example folder in MySensors-provided source code. I did change a bit to match my radios and settings, as well as providing a value to the gateway the first time it runs so that it would show up in Home Assistant.

Now I have checked the mysensors.json file on the server, and it has gotten the data from the sketch! So I know that the radios, gateway, and MQTT server are all doing their jobs.

However, I can't find any new devices or entities in Home Assistant that match the light. Since this is my first time adding something through MySensors, I'm not sure what it's supposed to look like. Do they show up automatically, or do I need to go tweak something? One thing I read talked about having an entry in the configuration.yaml file for each device, but another said it shouldn't be needed anymore.

I'm going to mess with it some more next time I get a chance, but it's late here now so I need to be done for the day.

If it helps, here's what my mysensors.json file looks like for that sample sketch:

    "1": {
        "sensor_id": 1,
        "children": {
            "1": {
                "id": 1,
                "type": 31,
                "description": "",
                "values": {
                    "3": "0"
                }
            },
            "2": {
                "id": 2,
                "type": 31,
                "description": "",
                "values": {}
            },
            "3": {
                "id": 3,
                "type": 31,
                "description": "",
                "values": {}
            },
            "4": {
                "id": 4,
                "type": 31,
                "description": "",
                "values": {}
            }
        },
        "type": 17,
        "sketch_name": "Dimmable Light",
        "sketch_version": "1.0",
        "battery_level": 0,
        "protocol_version": "2.3.2",
        "heartbeat": 0
    }

Thanks!

[electrik]

@ejlane said in Dual Radio with Home Assistant Question:

. Do they show up automatically, or do I need to go tweak something?

They will, if your sketch complies to the presentation requirements. See https://www.home-assistant.io/integrations/sensor.mysensors/

[ejlane]

@electrik

Thank you - I hadn't seen that page yet. I've tried to very carefully follow this page, though: https://www.home-assistant.io/integrations/mysensors

When I use the 2.x example on the page you linked, I do get the sensor to show up in the Configuration/Entities page. However, when I do, it has the edit symbol crossed out and when I click on it, it says: "This entity does not have a unique ID, therefore its settings cannot be managed from the UI."

I've tried to look wherever I could, but I can't find a way to get MySensors to report an ID. Is it possible? If not, do I then edit the .json file, or where can I go to fix that?

Thank you so much! I feel like I'm super close to getting this!

[electrik]

Is it really needed to edit this entity?
If you add it to your dashboard, you should be able to use it.
On the top right click on the three dots, then edit dashboard. Create a card and add the sensor there and you should be good to go!

[ejlane]

@electrik

Looks like you're completely correct.

I don't know if I've ever used that function to manually add something to the dashboard, or if it was only long enough ago that I don't remember doing it that way. When I saw that warning about it not being editable, I thought that was an issue that I needed to address. (To me, it sounded like I wouldn't be able to turn a light on or off, for example.)

However, if that is not needed, then I guess I'm set! Now all I need to do is compare my code to the example code and make modifications where necessary to make mine show up.

Thank you so much!!!