Any plan to implement Ethernet GW?

[Andrew Sanjanwala]

Alright, false alarm. Root issue in my case was a faulty environment and/or operator that resulted in binaries being compiled for the wrong platform.

Once corrected, it compiled and launched your branch and connected to the gateway without incident. Next up is experimenting with the nodes and actuators. I'll see if the persistence file can be written to.

[martinhjelmare]

@Andrew-Sanjanwala

:thumbsup:

[Dave Dan]

Hey @martinhjelmare,

I've been able to do some tests this time.

From a Sensoring standpoint (retreive information from sensors to show in dashboard) works perfetly fine.

I found the issue that I was referring before.

Let's say you have already included a number of nodes and everything is working but for any reason you want to reset and start again. My logic was to remove the MSPersistance.json file, create a new one and the system will reconfigure again.

What I'm seeing is that if you do that the system enters in a limbo situation where no nodes are added but those that were beforer are not added either. you can't see them in the dashboard nor in the Developers Tools section.

Next step is to start testing relays, dimmers, etc.

Let me know if you need more detils on the scenario.

thanks!

[martinhjelmare]

@Dave-Dan said:

My logic was to remove the MSPersistance.json file, create a new one and the system will reconfigure again.

Can you describe more in detail how you did this? Do you have an example persistence file before and after your modifications?

[Dave Dan]

Sure!

the persistence once I've added some nodes looks like this:

{"104": {"sketch_version": "1.0", "children": {"0": {"id": 0, "values": {"1": "38.0"}, "type": 7}, "1": {"id": 1, "values": {"0": "22.0"}, "type": 6}, "2": {"id": 2, "values": {"23": "34"}, "type": 16}, "3": {"id": 3, "values": {"16": "0"}, "type": 1}}, "battery_level": 0, "sensor_id": 104, "type": 17, "sketch_name": "dyaSensor (HTLM)"}, "106": {"sketch_version": "1.0", "children": {"0": {"id": 0, "values": {"1": "37.0"}, "type": 7}, "1": {"id": 1, "values": {"0": "22.0"}, "type": 6}, "2": {"id": 2, "values": {"23": "5"}, "type": 16}, "3": {"id": 3, "values": {"16": "0"}, "type": 1}}, "battery_level": 9, "sensor_id": 106, "type": 17, "sketch_name": "dyaSensor (HTLM)"}, "156": {"sketch_version": null, "children": {}, "battery_level": 0, "sensor_id": 156, "type": 17, "sketch_name": null}, "101": {"sketch_version": "1.2", "children": {"0": {"id": 0, "values": {"1": "34.0"}, "type": 7}, "1": {"id": 1, "values": {"0": "21.0"}, "type": 6}, "2": {"id": 2, "values": {}, "type": 16}, "3": {"id": 3, "values": {"16": "0"}, "type": 1}}, "battery_level": 0, "sensor_id": 101, "type": 17, "sketch_name": "dyaSensor (HTLM) v1.2"}, "102": {"sketch_version": "1.0", "children": {"0": {"id": 0, "values": {"1": "36.0"}, "type": 7}, "1": {"id": 1, "values": {"0": "18.0"}, "type": 6}, "2": {"id": 2, "values": {"23": "69"}, "type": 16}, "3": {"id": 3, "values": {"16": "0"}, "type": 1}}, "battery_level": 2, "sensor_id": 102, "type": 17, "sketch_name": "dyaSensor (HTLM)"}}

after my modification (meaning, clear everything) looks this way:

{}

[martinhjelmare]

@Dave-Dan

Did you do it like this?

1. Stop hass.
2. Remove old persistence file.
3. Create new persistence file with the same name and path, containing an empty JSON object, "{}".
4. Start hass.
5. Start node. Result: Messages are coming in to gateway, but node sensors are not added to entities nor shown in GUI.

[Dave Dan]

Exactly that.

maybe something to do with the fact that those items are already included in the DB (for history, etc) and the system can't add them again? (just guessing)

One more thing.

If I move the file and create a new one instead of deleting it. And after trying to add the items again I stop everything and restore the backup everything works again.

[martinhjelmare]

@Dave-Dan

I don't know yet, but I'll try to reproduce this and figure it out. Thanks for this report, and taking your time to test this.

I haven't had time to push my latest commits, that I had planned. I want to just test them quickly on my side before I push. But since you experience this problem, I think it's best to troubleshoot this first, before I push any new commits.

I'll post back here, once I have more info.

[Dave Dan]

hey @martinhjelmare,

would it be possible to have an updated compilation with the 0.16 versión? I think the one you have above is 0.15 still (or older, right?)

I'd like to do more testing and integrate other items (for example the new Vera integration) with MySensors actuations, etc.

thanks in advance.

[martinhjelmare]

@Dave-Dan

I'd like to investigate the issue you reported before I push any new commits or rebase off dev. Sorry to keep you waiting.

[Dave Dan]

ahh, ok, makes sense.

no problem at all! :)

[drock1985]

@Dave-Dan

out of curiosity, did you try a different name for the persistence file after deleting your first one, or did you use the same name?

[Dave Dan]

hey @drock1985,

No I didn't test that before, but I just did with the same results :(

I can see in the log how messages are arriving to HA (see below). It's just I can't see them either in the group I created for them or in the Developer Tools.

16-03-31 09:42:37 DEBUG (Thread-20) [mysensors.mysensors] Received 101;1;1;0;0;20.0

16-03-31 09:42:37 DEBUG (Thread-20) [mysensors.mysensors] Received 101;0;1;0;1;36.0

16-03-31 09:42:37 DEBUG (Thread-20) [mysensors.mysensors] Received 101;3;1;0;16;0

If I go back and restore the previous file I can see them on both the group and the Developer Tools section.

@martinhjelmare, hope this helps as well ... is is there any specific test you want me to perform please let me know.

[martinhjelmare]

@Dave-Dan

I finally had some time to test this again. Sorry for the long delay.

I followed the steps here:

@martinhjelmare said:

@Dave-Dan

Did you do it like this?

1. Stop hass.
2. Remove old persistence file.
3. Create new persistence file with the same name and path, containing an empty JSON object, "{}".
4. Start hass.
5. Start node. Result: Messages are coming in to gateway, but node sensors are not added to entities nor shown in GUI.

but I'm not able to reproduce your experienced problem. On my end, the node and sensor are added ok. Do you have any more clues what might be different on your end?

HA config:

mysensors:
  gateways:
    - device: '192.168.1.18'
      persistence_file: '/home/martin/.homeassistant/mysensors3.json'
      tcp_port: 5003
  debug: true
  persistence: true
  version: '1.5'
  optimistic: false

console log:

configuration.yaml  home-assistant.db  home-assistant.log  known_devices.yaml  lib  mysensors3.json
martin@martin-nuc:~/.homeassistant$ cat mysensors3.json
{"1": {"children": {"1": {"values": {"40": "ffffff", "2": "0", "3": "0"}, "id": 1, "type": 26}}, "sketch_version": "1.1", "sketch_name": "DimmableLED", "battery_level": 0, "sensor_id": 1, "type": 17}}
martin@martin-nuc:~/.homeassistant$ rm mysensors3.json
martin@martin-nuc:~/.homeassistant$ ls
configuration.yaml  home-assistant.db  home-assistant.log  known_devices.yaml  lib
martin@martin-nuc:~/.homeassistant$ atom .
martin@martin-nuc:~/.homeassistant$ ls
configuration.yaml  home-assistant.db  home-assistant.log  known_devices.yaml  lib  mysensors3.json
martin@martin-nuc:~/.homeassistant$ cat mysensors3.json
{}
martin@martin-nuc:~/.homeassistant$ cat mysensors3.json
{"1": {"battery_level": 0, "sensor_id": 1, "sketch_name": "DimmableLED", "children": {"1": {"values": {"40": "ffffff", "2": "0", "3": "0"}, "id": 1, "type": 26}}, "sketch_version": "1.1", "type": 17}}

sensor serial log:

send: 1-1-0-0 s=255,c=3,t=15,pt=2,l=2,sg=0,st=ok:0
send: 1-1-0-0 s=255,c=0,t=17,pt=0,l=5,sg=0,st=ok:1.5.1
send: 1-1-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=ok:0
read: 0-0-1 s=255,c=3,t=15,pt=2,l=2,sg=0:0
read: 0-0-1 s=255,c=3,t=6,pt=0,l=1,sg=0:M
sensor started, id=1, parent=0, distance=1
send: 1-1-0-0 s=1,c=0,t=26,pt=0,l=0,sg=0,st=ok:
send: 1-1-0-0 s=255,c=3,t=11,pt=0,l=11,sg=0,st=ok:DimmableLED
send: 1-1-0-0 s=255,c=3,t=12,pt=0,l=3,sg=0,st=ok:1.1
send: 1-1-0-0 s=1,c=2,t=3,pt=0,l=0,sg=0,st=ok:
send: 1-1-0-0 s=1,c=1,t=2,pt=2,l=2,sg=0,st=ok:0
send: 1-1-0-0 s=1,c=1,t=3,pt=2,l=2,sg=0,st=ok:0
send: 1-1-0-0 s=1,c=1,t=40,pt=0,l=6,sg=0,st=ok:ffffff
read: 0-0-1 s=1,c=1,t=2,pt=0,l=1,sg=0:1
Changing level to 100, from 0
send: 1-1-0-0 s=1,c=1,t=2,pt=2,l=2,sg=0,st=ok:1
send: 1-1-0-0 s=1,c=1,t=3,pt=2,l=2,sg=0,st=ok:100
read: 0-0-1 s=1,c=1,t=2,pt=0,l=1,sg=0:0
Changing level to 0, from 100
send: 1-1-0-0 s=1,c=1,t=2,pt=2,l=2,sg=0,st=ok:0
send: 1-1-0-0 s=1,c=1,t=3,pt=2,l=2,sg=0,st=ok:0

hass log:

16-04-10 14:23:01 INFO (Thread-8) [mysensors.mysensors] Trying to connect to ('192.168.1.18', 5003)
16-04-10 14:23:01 INFO (Thread-21) [homeassistant.core] Timer:starting
16-04-10 14:23:01 INFO (Thread-8) [mysensors.mysensors] Connected to ('192.168.1.18', 5003)
16-04-10 14:23:03 INFO (Thread-23) [homeassistant.components.http] "GET /api/stream?api_password=no_password_set&restrict=state_changed,component_loaded,service_registered HTTP/1.1" 200 -
16-04-10 14:23:03 INFO (Thread-24) [homeassistant.components.http] "GET /api/bootstrap HTTP/1.1" 200 -
16-04-10 14:23:30 INFO (Thread-3) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sun.sun, old_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.69, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>, new_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.66, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>>
16-04-10 14:24:30 INFO (Thread-2) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sun.sun, old_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.66, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>, new_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.6, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>>
16-04-10 14:25:30 INFO (Thread-16) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sun.sun, old_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.6, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>, new_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.54, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>>
16-04-10 14:25:41 INFO (Thread-25) [homeassistant.components.http] "GET /static/frontend-833d09737fec24f9219efae87c5bfd2a.html HTTP/1.1" 200 -
16-04-10 14:25:42 INFO (Thread-26) [homeassistant.components.http] "GET /static/favicon.ico HTTP/1.1" 200 -
16-04-10 14:25:42 INFO (Thread-27) [homeassistant.components.http] "GET /api/bootstrap HTTP/1.1" 200 -
16-04-10 14:25:42 INFO (Thread-28) [homeassistant.components.http] "GET /api/stream?api_password=no_password_set&restrict=state_changed,component_loaded,service_registered HTTP/1.1" 200 -
16-04-10 14:25:53 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:53 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:53 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:53 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:55 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:55 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:55 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:55 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:59 INFO (Thread-8) [homeassistant.components.mysensors] Adding new devices: <Entity DimmableLED 1.1: off>
16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=None, new_state=<state light.dimmableled_11=unavailable; friendly_name=DimmableLED 1.1 @ 14:25:59 10-04-2016>>
16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=None, new_state=<state group.all_lights=unknown; hidden=True, entity_id=[], auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>>
16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=<state group.all_lights=unknown; hidden=True, entity_id=[], auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>, new_state=None>
16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=None, new_state=<state group.all_lights=unknown; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>>
16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=unavailable; friendly_name=DimmableLED 1.1 @ 14:25:59 10-04-2016>, new_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:25:59 10-04-2016>>
16-04-10 14:25:59 INFO (Thread-17) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=<state group.all_lights=unknown; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>, new_state=<state group.all_lights=off; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>>
16-04-10 14:25:59 INFO (Thread-23) [homeassistant.components.api] Found broken event stream to 127.0.0.1, cleaning up
16-04-10 14:26:19 INFO (Thread-29) [homeassistant.core] Bus:Handling <Event call_service[L]: service=turn_on, domain=homeassistant, service_data=entity_id=light.dimmableled_11, service_call_id=140012061724232-1>
16-04-10 14:26:19 INFO (Thread-15) [homeassistant.core] Bus:Handling <Event call_service[L]: service=turn_on, domain=light, service_data=entity_id=['light.dimmableled_11'], service_call_id=140012061724232-2>
16-04-10 14:26:19 INFO (Thread-9) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=140012061724232-2>
16-04-10 14:26:19 INFO (Thread-15) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=140012061724232-1>
16-04-10 14:26:19 INFO (Thread-29) [homeassistant.components.http] "POST /api/services/homeassistant/turn_on HTTP/1.1" 200 -
16-04-10 14:26:21 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:25:59 10-04-2016>, new_state=<state light.dimmableled_11=off; V_STATUS=on, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:25:59 10-04-2016>>
16-04-10 14:26:22 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=off; V_STATUS=on, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:25:59 10-04-2016>, new_state=<state light.dimmableled_11=on; V_RGB=ffffff, V_PERCENTAGE=100, battery_level=0, rgb_color=[255, 255, 255], child_id=1, V_STATUS=on, brightness=255, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, node_id=1 @ 14:26:22 10-04-2016>>
16-04-10 14:26:22 INFO (Thread-7) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=<state group.all_lights=off; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>, new_state=<state group.all_lights=on; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:26:22 10-04-2016>>
16-04-10 14:26:26 INFO (Thread-30) [homeassistant.core] Bus:Handling <Event call_service[L]: service=turn_off, domain=homeassistant, service_data=entity_id=light.dimmableled_11, service_call_id=140012061724232-3>
16-04-10 14:26:26 INFO (Thread-6) [homeassistant.core] Bus:Handling <Event call_service[L]: service=turn_off, domain=light, service_data=entity_id=['light.dimmableled_11'], service_call_id=140012061724232-4>
16-04-10 14:26:26 INFO (Thread-13) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=140012061724232-4>
16-04-10 14:26:26 INFO (Thread-6) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=140012061724232-3>
16-04-10 14:26:26 INFO (Thread-30) [homeassistant.components.http] "POST /api/services/homeassistant/turn_off HTTP/1.1" 200 -
16-04-10 14:26:27 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=on; V_RGB=ffffff, V_PERCENTAGE=100, battery_level=0, rgb_color=[255, 255, 255], child_id=1, V_STATUS=on, brightness=255, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, node_id=1 @ 14:26:22 10-04-2016>, new_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=100, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:26:27 10-04-2016>>
16-04-10 14:26:27 INFO (Thread-18) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=<state group.all_lights=on; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:26:22 10-04-2016>, new_state=<state group.all_lights=off; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:26:27 10-04-2016>>
16-04-10 14:26:28 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=100, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:26:27 10-04-2016>, new_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:26:27 10-04-2016>>

[Dave Dan]

hey Martin.

Sorry, I've been traveling for a while.

So I just rebuilt my environment and I can't repro the issue anymore.

If you can't either and no one else is experiencing this I think we can close this down. My environment might be the only thing that is creating the issue.

If the issue happen again I can update with the new information.

[martinhjelmare]

@Dave-Dan

Great! Then I'll move forward and push my latest changes to my github branch. I'll post here when that is done. You're welcome to test these by the update instructions above.

If nothing comes up, my plan is to have this merged in the next release of home assistant.

[subutux]

Hi,

I've also tested your pymysensors version with tcp gateway. I've had an excessive cpu usage on my pi Zero (actually, a constant 100%). After some debugging I've found that I've had to add a sleep of 0.2 to the while loop of the top socket readout. My cpu usage returned normal.

What I've also encountered is that when the tcp gateway looses connection, pymysensors still thinks it's connected. I've buildin an active check by requesting the gateway version every 10 seconds.

Would you like me to make a pull request ?

[martinhjelmare]

@subutux

Thanks for the feedback!

I've got some commits that are not pushed yet with some changes, but a pull request should be good anyhow. Great!

[martinhjelmare]

I've now pushed some updates to my ethernet gateway branch of pymysensors and rebased my home-assistant mysensors-tcp-gateway branch off latest upstream dev branch and pushed that to my github.

To upgrade follow the instructions below.

@martinhjelmare said:

I've updated the branch at my fork of pymysensors. If you have installed the ethernet branch of my fork of home-assistant, you should uninstall/reinstall that, or just make a new virtual environment and install again. You should remove the lib directory in your home assistant config directory as well.

Eg:

rm -r ~/.homeassistant/lib
mkvirtualenv -p /usr/bin/python3 hasstest2
cd ~/dev/home-assistant
git branch -D mysensors-tcp-gateway
git fetch origin mysensors-tcp-gateway
git checkout mysensors-tcp-gateway
pip3 install -e .
pip3 install colorlog
pip3 install python-Levenshtein

[martinhjelmare]

@subutux

Please rebase your pymysensors branch off my latest updates, before you make the PR. Thanks.