Dual Radio with Home Assistant Question

[ejlane]

Well, it's still failing.

I'm using the current version 5.1 of the Mosquitto broker add-on to Home Assistant. In the configuration page I have this:

logins:
  - username: myOutdoorGateway
    password: outdoorPassword
  - username: myIndoorGateway
    password: indoorPassword
anonymous: false
customize:
  active: false
  folder: mosquitto
certfile: fullchain.pem
keyfile: privkey.pem
require_certificate: false

In the configuration.yaml file I have this:

# Configure a default setup of Home Assistant (frontend, api, etc)
default_config:

# Uncomment this if you are using SSL/TLS, running in Docker container, etc.
# http:
#   base_url: example.duckdns.org:8123

# Text to speech
tts:
  - platform: google_translate

group: !include groups.yaml
automation: !include automations.yaml
script: !include scripts.yaml
scene: !include scenes.yaml

mqtt:
  broker: 172.17.0.1

mysensors:
  gateways:
    - device: mqtt
      persistence_file: 'mysensors/myoutdoorsensors.json'
      topic_in_prefix: 'myOutdoorGateway-out'
      topic_out_prefix: 'myOutdoorGateway-in'
    - device: mqtt
      persistence_file: 'mysensors/myindoorsensors.json'
      topic_in_prefix: 'myIndoorGateway-out'
      topic_out_prefix: 'myIndoorGateway-in'
  optimistic: false
  persistence: true
  retain: false
  version: '2.3'

homeassistant:
  customize: !include customize.yaml

Both of the json files are there in the mysensors folder, but they are just empty.

I compiled the two gateways with:

./configure --my-transport=rf24 --my-rf24-irq-pin=33 --my-rf24-ce-pin=37 --my-rf24-cs-pin=36 --spi-spidev-device=/dev/spidev1.0 --spi-driver=SPIDEV  --my-gateway=mqtt --my-controller-ip-address=192.168.0.49 --my-mqtt-publish-topic-prefix=myIndoorGateway-out --my-mqtt-subscribe-topic-prefix=myIndoorGateway-in --my-mqtt-client-id=myIndoorGateway --my-leds-err-pin=29 --my-leds-rx-pin=31 --my-leds-tx-pin=32 --my-rf24-encryption-enabled --my-signing=password --my-security-password=indoors --my-mqtt-password=indoorPassword

./configure --my-transport=rfm69 --my-rfm69-frequency=915 --my-is-rfm69hw --my-gateway=mqtt --my-controller-ip-address=192.168.0.49 --my-mqtt-publish-topic-prefix=myOutdoorGateway-out --my-mqtt-subscribe-topic-prefix=myOutdoorGateway-in --my-mqtt-client-id=myOutdoorGateway --my-leds-err-pin=12 --my-leds-rx-pin=16 --my-leds-tx-pin=18 --my-signing=password --my-security-password=outdoors --my-rfm69-encryption-enabled --my-mqtt-password=outdoorPassword

However, for both gateways, when I run the test everything looks good until it tries to connect to the mqtt server.

pi@mysensors-gateway:~/MySensorsSecond $ ./bin/mysgwIndoor
Sep 14 11:53:56 INFO  Starting gateway...
Sep 14 11:53:56 INFO  Protocol version - 2.4.0-alpha
Sep 14 11:53:56 DEBUG MCO:BGN:INIT GW,CP=RNNGLSQX,FQ=NA,REL=0,VER=2.4.0-alpha
Sep 14 11:53:56 DEBUG TSF:LRT:OK
Sep 14 11:53:56 DEBUG TSM:INIT
Sep 14 11:53:56 DEBUG TSF:WUR:MS=0
Sep 14 11:53:56 DEBUG TSM:INIT:TSP OK
Sep 14 11:53:56 DEBUG TSM:INIT:GW MODE
Sep 14 11:53:56 DEBUG TSM:READY:ID=0,PAR=0,DIS=0
Sep 14 11:53:56 DEBUG MCO:REG:NOT NEEDED
Sep 14 11:53:56 DEBUG MCO:BGN:STP
Sep 14 11:53:56 DEBUG MCO:BGN:INIT OK,TSP=1
Sep 14 11:53:56 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:53:56 DEBUG connected to 192.168.0.49
Sep 14 11:53:57 DEBUG !GWT:RMQ:FAIL
Sep 14 11:53:57 DEBUG TSM:READY:NWD REQ
Sep 14 11:53:57 DEBUG ?TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Sep 14 11:53:57 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:53:57 DEBUG connected to 192.168.0.49
Sep 14 11:53:58 DEBUG !GWT:RMQ:FAIL
Sep 14 11:53:58 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:53:58 DEBUG connected to 192.168.0.49
Sep 14 11:53:59 DEBUG !GWT:RMQ:FAIL
Sep 14 11:53:59 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:53:59 DEBUG connected to 192.168.0.49
Sep 14 11:54:00 DEBUG !GWT:RMQ:FAIL
Sep 14 11:54:00 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:54:00 DEBUG connected to 192.168.0.49
Sep 14 11:54:01 DEBUG !GWT:RMQ:FAIL
Sep 14 11:54:01 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:54:01 DEBUG connected to 192.168.0.49

and

sudo ./bin/mysgw
Sep 14 11:43:26 INFO  Starting gateway...
Sep 14 11:43:26 INFO  Protocol version - 2.4.0-alpha
Sep 14 11:43:26 DEBUG MCO:BGN:INIT GW,CP=RPNGLS-X,FQ=NA,REL=0,VER=2.4.0-alpha
Sep 14 11:43:26 DEBUG TSF:LRT:OK
Sep 14 11:43:26 DEBUG TSM:INIT
Sep 14 11:43:26 DEBUG TSF:WUR:MS=0
Sep 14 11:43:26 DEBUG TSM:INIT:TSP OK
Sep 14 11:43:26 DEBUG TSM:INIT:GW MODE
Sep 14 11:43:26 DEBUG TSM:READY:ID=0,PAR=0,DIS=0
Sep 14 11:43:26 DEBUG MCO:REG:NOT NEEDED
Sep 14 11:43:26 DEBUG MCO:BGN:STP
Sep 14 11:43:26 DEBUG MCO:BGN:INIT OK,TSP=1
Sep 14 11:43:26 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:43:26 DEBUG connected to 192.168.0.49
Sep 14 11:43:27 DEBUG !GWT:RMQ:FAIL
Sep 14 11:43:27 DEBUG TSM:READY:NWD REQ
Sep 14 11:43:27 DEBUG ?TSF:MSG:SEND,0-0-255-255,s=255,c=3,t=20,pt=0,l=0,sg=0,ft=0,st=OK:
Sep 14 11:43:27 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:43:27 DEBUG connected to 192.168.0.49
Sep 14 11:43:28 DEBUG !GWT:RMQ:FAIL
Sep 14 11:43:28 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:43:28 DEBUG connected to 192.168.0.49
Sep 14 11:43:29 DEBUG !GWT:RMQ:FAIL
Sep 14 11:43:29 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:43:29 DEBUG connected to 192.168.0.49
Sep 14 11:43:30 DEBUG !GWT:RMQ:FAIL
Sep 14 11:43:31 DEBUG GWT:RMQ:CONNECTING...
Sep 14 11:43:31 DEBUG connected to 192.168.0.49

So they both fail with the same error message of !GWT:RMQ:FAIL but I can't get the log parser to tell me what that means. From this page: https://www.mysensors.org/apidocs/group__MyGatewayTransportgrp.html it looks like it's just a generic failure, so I don't know what I have wrong.

In the log file for the MQTT broker, all I see is a bunch of messages about a socket error. They look like this, if it matters:

1600109639: New connection from 192.168.0.48 on port 1883.
1600109639: Socket error on client <unknown>, disconnecting.

Does this mean something to you? Any pointers for where next to look?

Thanks so much!

[mfalkvidd]

@ejlane this might be a shot in the dark, but could you try using different config files for the two gateways? By default, they use /etc/mysensors.conf which means (among other things) they will use the same file for emulating eeprom (specifid in the config file), and sharing the same eeprom file could cause all sorts of strange random stuff.

[ejlane]

@mfalkvidd
I went ahead and tried it. But right now I only have one running at a time, so they shouldn't be able to interfere, should they? (I haven't done the 'make install' on either one, I'm just running them manually while I test things out.)

Anyway, no difference. I tried setting anonymous to true in the Mosquitto broker settings, and that allowed me to then connect. Now I have a new error:

Sep 14 14:19:01 DEBUG GWT:RMQ:CONNECTING...
Sep 14 14:19:01 DEBUG connected to 192.168.0.49
Sep 14 14:19:01 DEBUG GWT:RMQ:OK
Sep 14 14:19:01 DEBUG GWT:TPS:TOPIC=myIndoorGateway-out/0/255/0/0/18,MSG SENT
Sep 14 14:19:01 DEBUG !MCO:PRO:RC=1
Sep 14 14:19:01 DEBUG !MCO:PRO:RC=1

According to the log parser, https://www.mysensors.org/build/parser?log=!MCO%3APRO%3ARC%3D1 this is a recursive call. So I still have something wrong.

I'll keep working on that. But it now looks like my issues are primarily not a mysensors thing, and some kind of problem where Mosquitto is not accepting the username/password combo that it should be.

Can anyone share their settings for that, from the Mosquitto settings page of Home Assistant? Maybe I have a formatting or spelling error in there.

[ejlane]

I'm having no luck getting it to work with a username and password to connect to the mqtt broker. Well, I can get the Home Assistant user to connect, but I can't get the mysensors gateway to connect at all.

It just keeps telling me that it failed, and in the mqtt logs, it says that it was an unauthorized user.

I've tried all kinds of varieties, and I just can't get it to connect.

I captured the outgoing packets from the gateway and put them into wireshark, and I never see a password sent. There are also "connect" flags in the header that aren't set for user name and password. See this page for what I was comparing to: https://dzone.com/articles/dissecting-mqtt-using-wireshark On that page the flags are set, and the password is also visible.

If it's really not sending the password then there's no way that mosquitto will authorize it. Now I just need to find out what I'm doing wrong with the build flags that is stopping it from sending passwords out.

[ejlane]

I might have it figured out. At least so far the one is connecting and not showing any errors. Now that I have it this far, I need to program a sensor and see if I can get it to connect and show up in the Home Assistant dashboard. So it might still be a while before I know for sure.

But I was browsing the mqtt transport source code at https://github.com/mysensors/MySensors/blob/development/core/MyGatewayTransportMQTTClient.cpp when I saw something that surprised me. I saw a function that had both the client-id and username as parameters. This tipped me off that I should look at that closer, because I had seen something about both of them, but in the back of my mind just assumed they were the same thing.

I went and looked in my build flags, and I didn't have the --my-mqtt-user anywhere. After adding it, I can now run the gateway code and it connects and just waits for activity, without throwing any errors. So this feels like good progress, and that this part is solved.

However, the indoor gateway is still showing the errors from before about a recursive problem. https://www.mysensors.org/build/parser?log=!MCO%3APRO%3ARC%3D1

How do I track down more info about this? Is browsing the source the only option? Is there a way for it to provide more details about just what is going on without editing the source code?

Thanks!

[ejlane]

I've found it in the source code, and it looks like it's just reporting that a recursion happened. Since the recursion level never gets above 1 in all the messages that I see, I'm going to ignore it for now and just keep it in mind in case I run into weird things in the future.

[ejlane]

@electrik

Well, I've made a bunch of progress, and found a few small errors. I simplified everything down to just using the 'Dimmable Light' from the Example folder in MySensors-provided source code. I did change a bit to match my radios and settings, as well as providing a value to the gateway the first time it runs so that it would show up in Home Assistant.

Now I have checked the mysensors.json file on the server, and it has gotten the data from the sketch! So I know that the radios, gateway, and MQTT server are all doing their jobs.

However, I can't find any new devices or entities in Home Assistant that match the light. Since this is my first time adding something through MySensors, I'm not sure what it's supposed to look like. Do they show up automatically, or do I need to go tweak something? One thing I read talked about having an entry in the configuration.yaml file for each device, but another said it shouldn't be needed anymore.

I'm going to mess with it some more next time I get a chance, but it's late here now so I need to be done for the day.

If it helps, here's what my mysensors.json file looks like for that sample sketch:

    "1": {
        "sensor_id": 1,
        "children": {
            "1": {
                "id": 1,
                "type": 31,
                "description": "",
                "values": {
                    "3": "0"
                }
            },
            "2": {
                "id": 2,
                "type": 31,
                "description": "",
                "values": {}
            },
            "3": {
                "id": 3,
                "type": 31,
                "description": "",
                "values": {}
            },
            "4": {
                "id": 4,
                "type": 31,
                "description": "",
                "values": {}
            }
        },
        "type": 17,
        "sketch_name": "Dimmable Light",
        "sketch_version": "1.0",
        "battery_level": 0,
        "protocol_version": "2.3.2",
        "heartbeat": 0
    }

Thanks!

[electrik]

@ejlane said in Dual Radio with Home Assistant Question:

. Do they show up automatically, or do I need to go tweak something?

They will, if your sketch complies to the presentation requirements. See https://www.home-assistant.io/integrations/sensor.mysensors/

[ejlane]

@electrik

Thank you - I hadn't seen that page yet. I've tried to very carefully follow this page, though: https://www.home-assistant.io/integrations/mysensors

When I use the 2.x example on the page you linked, I do get the sensor to show up in the Configuration/Entities page. However, when I do, it has the edit symbol crossed out and when I click on it, it says: "This entity does not have a unique ID, therefore its settings cannot be managed from the UI."

I've tried to look wherever I could, but I can't find a way to get MySensors to report an ID. Is it possible? If not, do I then edit the .json file, or where can I go to fix that?

Thank you so much! I feel like I'm super close to getting this!

[electrik]

Is it really needed to edit this entity?
If you add it to your dashboard, you should be able to use it.
On the top right click on the three dots, then edit dashboard. Create a card and add the sensor there and you should be good to go!

[ejlane]

@electrik

Looks like you're completely correct.

I don't know if I've ever used that function to manually add something to the dashboard, or if it was only long enough ago that I don't remember doing it that way. When I saw that warning about it not being editable, I thought that was an issue that I needed to address. (To me, it sounded like I wouldn't be able to turn a light on or off, for example.)

However, if that is not needed, then I guess I'm set! Now all I need to do is compare my code to the example code and make modifications where necessary to make mine show up.

Thank you so much!!!