Any plan to implement Ethernet GW?

[Dave Dan]

Sure!

the persistence once I've added some nodes looks like this:

{"104": {"sketch_version": "1.0", "children": {"0": {"id": 0, "values": {"1": "38.0"}, "type": 7}, "1": {"id": 1, "values": {"0": "22.0"}, "type": 6}, "2": {"id": 2, "values": {"23": "34"}, "type": 16}, "3": {"id": 3, "values": {"16": "0"}, "type": 1}}, "battery_level": 0, "sensor_id": 104, "type": 17, "sketch_name": "dyaSensor (HTLM)"}, "106": {"sketch_version": "1.0", "children": {"0": {"id": 0, "values": {"1": "37.0"}, "type": 7}, "1": {"id": 1, "values": {"0": "22.0"}, "type": 6}, "2": {"id": 2, "values": {"23": "5"}, "type": 16}, "3": {"id": 3, "values": {"16": "0"}, "type": 1}}, "battery_level": 9, "sensor_id": 106, "type": 17, "sketch_name": "dyaSensor (HTLM)"}, "156": {"sketch_version": null, "children": {}, "battery_level": 0, "sensor_id": 156, "type": 17, "sketch_name": null}, "101": {"sketch_version": "1.2", "children": {"0": {"id": 0, "values": {"1": "34.0"}, "type": 7}, "1": {"id": 1, "values": {"0": "21.0"}, "type": 6}, "2": {"id": 2, "values": {}, "type": 16}, "3": {"id": 3, "values": {"16": "0"}, "type": 1}}, "battery_level": 0, "sensor_id": 101, "type": 17, "sketch_name": "dyaSensor (HTLM) v1.2"}, "102": {"sketch_version": "1.0", "children": {"0": {"id": 0, "values": {"1": "36.0"}, "type": 7}, "1": {"id": 1, "values": {"0": "18.0"}, "type": 6}, "2": {"id": 2, "values": {"23": "69"}, "type": 16}, "3": {"id": 3, "values": {"16": "0"}, "type": 1}}, "battery_level": 2, "sensor_id": 102, "type": 17, "sketch_name": "dyaSensor (HTLM)"}}

after my modification (meaning, clear everything) looks this way:

{}

[martinhjelmare]

@Dave-Dan

Did you do it like this?

1. Stop hass.
2. Remove old persistence file.
3. Create new persistence file with the same name and path, containing an empty JSON object, "{}".
4. Start hass.
5. Start node. Result: Messages are coming in to gateway, but node sensors are not added to entities nor shown in GUI.

[Dave Dan]

Exactly that.

maybe something to do with the fact that those items are already included in the DB (for history, etc) and the system can't add them again? (just guessing)

One more thing.

If I move the file and create a new one instead of deleting it. And after trying to add the items again I stop everything and restore the backup everything works again.

[martinhjelmare]

@Dave-Dan

I don't know yet, but I'll try to reproduce this and figure it out. Thanks for this report, and taking your time to test this.

I haven't had time to push my latest commits, that I had planned. I want to just test them quickly on my side before I push. But since you experience this problem, I think it's best to troubleshoot this first, before I push any new commits.

I'll post back here, once I have more info.

[Dave Dan]

hey @martinhjelmare,

would it be possible to have an updated compilation with the 0.16 versión? I think the one you have above is 0.15 still (or older, right?)

I'd like to do more testing and integrate other items (for example the new Vera integration) with MySensors actuations, etc.

thanks in advance.

[martinhjelmare]

@Dave-Dan

I'd like to investigate the issue you reported before I push any new commits or rebase off dev. Sorry to keep you waiting.

[Dave Dan]

ahh, ok, makes sense.

no problem at all! :)

[drock1985]

@Dave-Dan

out of curiosity, did you try a different name for the persistence file after deleting your first one, or did you use the same name?

[Dave Dan]

hey @drock1985,

No I didn't test that before, but I just did with the same results :(

I can see in the log how messages are arriving to HA (see below). It's just I can't see them either in the group I created for them or in the Developer Tools.

16-03-31 09:42:37 DEBUG (Thread-20) [mysensors.mysensors] Received 101;1;1;0;0;20.0

16-03-31 09:42:37 DEBUG (Thread-20) [mysensors.mysensors] Received 101;0;1;0;1;36.0

16-03-31 09:42:37 DEBUG (Thread-20) [mysensors.mysensors] Received 101;3;1;0;16;0

If I go back and restore the previous file I can see them on both the group and the Developer Tools section.

@martinhjelmare, hope this helps as well ... is is there any specific test you want me to perform please let me know.

[martinhjelmare]

@Dave-Dan

I finally had some time to test this again. Sorry for the long delay.

I followed the steps here:

@martinhjelmare said:

@Dave-Dan

Did you do it like this?

1. Stop hass.
2. Remove old persistence file.
3. Create new persistence file with the same name and path, containing an empty JSON object, "{}".
4. Start hass.
5. Start node. Result: Messages are coming in to gateway, but node sensors are not added to entities nor shown in GUI.

but I'm not able to reproduce your experienced problem. On my end, the node and sensor are added ok. Do you have any more clues what might be different on your end?

HA config:

mysensors:
  gateways:
    - device: '192.168.1.18'
      persistence_file: '/home/martin/.homeassistant/mysensors3.json'
      tcp_port: 5003
  debug: true
  persistence: true
  version: '1.5'
  optimistic: false

console log:

configuration.yaml  home-assistant.db  home-assistant.log  known_devices.yaml  lib  mysensors3.json
martin@martin-nuc:~/.homeassistant$ cat mysensors3.json
{"1": {"children": {"1": {"values": {"40": "ffffff", "2": "0", "3": "0"}, "id": 1, "type": 26}}, "sketch_version": "1.1", "sketch_name": "DimmableLED", "battery_level": 0, "sensor_id": 1, "type": 17}}
martin@martin-nuc:~/.homeassistant$ rm mysensors3.json
martin@martin-nuc:~/.homeassistant$ ls
configuration.yaml  home-assistant.db  home-assistant.log  known_devices.yaml  lib
martin@martin-nuc:~/.homeassistant$ atom .
martin@martin-nuc:~/.homeassistant$ ls
configuration.yaml  home-assistant.db  home-assistant.log  known_devices.yaml  lib  mysensors3.json
martin@martin-nuc:~/.homeassistant$ cat mysensors3.json
{}
martin@martin-nuc:~/.homeassistant$ cat mysensors3.json
{"1": {"battery_level": 0, "sensor_id": 1, "sketch_name": "DimmableLED", "children": {"1": {"values": {"40": "ffffff", "2": "0", "3": "0"}, "id": 1, "type": 26}}, "sketch_version": "1.1", "type": 17}}

sensor serial log:

send: 1-1-0-0 s=255,c=3,t=15,pt=2,l=2,sg=0,st=ok:0
send: 1-1-0-0 s=255,c=0,t=17,pt=0,l=5,sg=0,st=ok:1.5.1
send: 1-1-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=ok:0
read: 0-0-1 s=255,c=3,t=15,pt=2,l=2,sg=0:0
read: 0-0-1 s=255,c=3,t=6,pt=0,l=1,sg=0:M
sensor started, id=1, parent=0, distance=1
send: 1-1-0-0 s=1,c=0,t=26,pt=0,l=0,sg=0,st=ok:
send: 1-1-0-0 s=255,c=3,t=11,pt=0,l=11,sg=0,st=ok:DimmableLED
send: 1-1-0-0 s=255,c=3,t=12,pt=0,l=3,sg=0,st=ok:1.1
send: 1-1-0-0 s=1,c=2,t=3,pt=0,l=0,sg=0,st=ok:
send: 1-1-0-0 s=1,c=1,t=2,pt=2,l=2,sg=0,st=ok:0
send: 1-1-0-0 s=1,c=1,t=3,pt=2,l=2,sg=0,st=ok:0
send: 1-1-0-0 s=1,c=1,t=40,pt=0,l=6,sg=0,st=ok:ffffff
read: 0-0-1 s=1,c=1,t=2,pt=0,l=1,sg=0:1
Changing level to 100, from 0
send: 1-1-0-0 s=1,c=1,t=2,pt=2,l=2,sg=0,st=ok:1
send: 1-1-0-0 s=1,c=1,t=3,pt=2,l=2,sg=0,st=ok:100
read: 0-0-1 s=1,c=1,t=2,pt=0,l=1,sg=0:0
Changing level to 0, from 100
send: 1-1-0-0 s=1,c=1,t=2,pt=2,l=2,sg=0,st=ok:0
send: 1-1-0-0 s=1,c=1,t=3,pt=2,l=2,sg=0,st=ok:0

hass log:

16-04-10 14:23:01 INFO (Thread-8) [mysensors.mysensors] Trying to connect to ('192.168.1.18', 5003)
16-04-10 14:23:01 INFO (Thread-21) [homeassistant.core] Timer:starting
16-04-10 14:23:01 INFO (Thread-8) [mysensors.mysensors] Connected to ('192.168.1.18', 5003)
16-04-10 14:23:03 INFO (Thread-23) [homeassistant.components.http] "GET /api/stream?api_password=no_password_set&restrict=state_changed,component_loaded,service_registered HTTP/1.1" 200 -
16-04-10 14:23:03 INFO (Thread-24) [homeassistant.components.http] "GET /api/bootstrap HTTP/1.1" 200 -
16-04-10 14:23:30 INFO (Thread-3) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sun.sun, old_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.69, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>, new_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.66, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>>
16-04-10 14:24:30 INFO (Thread-2) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sun.sun, old_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.66, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>, new_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.6, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>>
16-04-10 14:25:30 INFO (Thread-16) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sun.sun, old_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.6, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>, new_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.54, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>>
16-04-10 14:25:41 INFO (Thread-25) [homeassistant.components.http] "GET /static/frontend-833d09737fec24f9219efae87c5bfd2a.html HTTP/1.1" 200 -
16-04-10 14:25:42 INFO (Thread-26) [homeassistant.components.http] "GET /static/favicon.ico HTTP/1.1" 200 -
16-04-10 14:25:42 INFO (Thread-27) [homeassistant.components.http] "GET /api/bootstrap HTTP/1.1" 200 -
16-04-10 14:25:42 INFO (Thread-28) [homeassistant.components.http] "GET /api/stream?api_password=no_password_set&restrict=state_changed,component_loaded,service_registered HTTP/1.1" 200 -
16-04-10 14:25:53 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:53 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:53 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:53 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:55 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:55 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:55 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:55 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
16-04-10 14:25:59 INFO (Thread-8) [homeassistant.components.mysensors] Adding new devices: <Entity DimmableLED 1.1: off>
16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=None, new_state=<state light.dimmableled_11=unavailable; friendly_name=DimmableLED 1.1 @ 14:25:59 10-04-2016>>
16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=None, new_state=<state group.all_lights=unknown; hidden=True, entity_id=[], auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>>
16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=<state group.all_lights=unknown; hidden=True, entity_id=[], auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>, new_state=None>
16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=None, new_state=<state group.all_lights=unknown; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>>
16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=unavailable; friendly_name=DimmableLED 1.1 @ 14:25:59 10-04-2016>, new_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:25:59 10-04-2016>>
16-04-10 14:25:59 INFO (Thread-17) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=<state group.all_lights=unknown; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>, new_state=<state group.all_lights=off; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>>
16-04-10 14:25:59 INFO (Thread-23) [homeassistant.components.api] Found broken event stream to 127.0.0.1, cleaning up
16-04-10 14:26:19 INFO (Thread-29) [homeassistant.core] Bus:Handling <Event call_service[L]: service=turn_on, domain=homeassistant, service_data=entity_id=light.dimmableled_11, service_call_id=140012061724232-1>
16-04-10 14:26:19 INFO (Thread-15) [homeassistant.core] Bus:Handling <Event call_service[L]: service=turn_on, domain=light, service_data=entity_id=['light.dimmableled_11'], service_call_id=140012061724232-2>
16-04-10 14:26:19 INFO (Thread-9) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=140012061724232-2>
16-04-10 14:26:19 INFO (Thread-15) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=140012061724232-1>
16-04-10 14:26:19 INFO (Thread-29) [homeassistant.components.http] "POST /api/services/homeassistant/turn_on HTTP/1.1" 200 -
16-04-10 14:26:21 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:25:59 10-04-2016>, new_state=<state light.dimmableled_11=off; V_STATUS=on, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:25:59 10-04-2016>>
16-04-10 14:26:22 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=off; V_STATUS=on, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:25:59 10-04-2016>, new_state=<state light.dimmableled_11=on; V_RGB=ffffff, V_PERCENTAGE=100, battery_level=0, rgb_color=[255, 255, 255], child_id=1, V_STATUS=on, brightness=255, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, node_id=1 @ 14:26:22 10-04-2016>>
16-04-10 14:26:22 INFO (Thread-7) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=<state group.all_lights=off; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>, new_state=<state group.all_lights=on; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:26:22 10-04-2016>>
16-04-10 14:26:26 INFO (Thread-30) [homeassistant.core] Bus:Handling <Event call_service[L]: service=turn_off, domain=homeassistant, service_data=entity_id=light.dimmableled_11, service_call_id=140012061724232-3>
16-04-10 14:26:26 INFO (Thread-6) [homeassistant.core] Bus:Handling <Event call_service[L]: service=turn_off, domain=light, service_data=entity_id=['light.dimmableled_11'], service_call_id=140012061724232-4>
16-04-10 14:26:26 INFO (Thread-13) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=140012061724232-4>
16-04-10 14:26:26 INFO (Thread-6) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=140012061724232-3>
16-04-10 14:26:26 INFO (Thread-30) [homeassistant.components.http] "POST /api/services/homeassistant/turn_off HTTP/1.1" 200 -
16-04-10 14:26:27 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=on; V_RGB=ffffff, V_PERCENTAGE=100, battery_level=0, rgb_color=[255, 255, 255], child_id=1, V_STATUS=on, brightness=255, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, node_id=1 @ 14:26:22 10-04-2016>, new_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=100, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:26:27 10-04-2016>>
16-04-10 14:26:27 INFO (Thread-18) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=<state group.all_lights=on; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:26:22 10-04-2016>, new_state=<state group.all_lights=off; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:26:27 10-04-2016>>
16-04-10 14:26:28 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=100, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:26:27 10-04-2016>, new_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:26:27 10-04-2016>>

[Dave Dan]

hey Martin.

Sorry, I've been traveling for a while.

So I just rebuilt my environment and I can't repro the issue anymore.

If you can't either and no one else is experiencing this I think we can close this down. My environment might be the only thing that is creating the issue.

If the issue happen again I can update with the new information.

[martinhjelmare]

@Dave-Dan

Great! Then I'll move forward and push my latest changes to my github branch. I'll post here when that is done. You're welcome to test these by the update instructions above.

If nothing comes up, my plan is to have this merged in the next release of home assistant.

[subutux]

Hi,

I've also tested your pymysensors version with tcp gateway. I've had an excessive cpu usage on my pi Zero (actually, a constant 100%). After some debugging I've found that I've had to add a sleep of 0.2 to the while loop of the top socket readout. My cpu usage returned normal.

What I've also encountered is that when the tcp gateway looses connection, pymysensors still thinks it's connected. I've buildin an active check by requesting the gateway version every 10 seconds.

Would you like me to make a pull request ?

[martinhjelmare]

@subutux

Thanks for the feedback!

I've got some commits that are not pushed yet with some changes, but a pull request should be good anyhow. Great!

[martinhjelmare]

I've now pushed some updates to my ethernet gateway branch of pymysensors and rebased my home-assistant mysensors-tcp-gateway branch off latest upstream dev branch and pushed that to my github.

To upgrade follow the instructions below.

@martinhjelmare said:

I've updated the branch at my fork of pymysensors. If you have installed the ethernet branch of my fork of home-assistant, you should uninstall/reinstall that, or just make a new virtual environment and install again. You should remove the lib directory in your home assistant config directory as well.

Eg:

rm -r ~/.homeassistant/lib
mkvirtualenv -p /usr/bin/python3 hasstest2
cd ~/dev/home-assistant
git branch -D mysensors-tcp-gateway
git fetch origin mysensors-tcp-gateway
git checkout mysensors-tcp-gateway
pip3 install -e .
pip3 install colorlog
pip3 install python-Levenshtein

[martinhjelmare]

@subutux

Please rebase your pymysensors branch off my latest updates, before you make the PR. Thanks.

[martinhjelmare]

@subutux

I went ahead and added your fix for the 100% cpu usage. Although I settled for a very short 0.02 seconds. It was enough for my intel based processor. If you still experience problems and think we should increase it, let me know.

Please make a PR for the check of the connection here https://github.com/theolind/pymysensors if you have time.

My plan is to try and have this make the release of home assistant that is planned during this week. I think it should be good enough for a first version now. We can add polish for the next release.

[drock1985]

Hi @martinhjelmare

I see HA got released early, and that the ethernetgateway was added. I'll check it out later on tonight but, thanks again for all the work you do on the MySensors project.

[martinhjelmare]

@drock1985

Yeah, finally! I've been a bit disappointed with myself, that it took a long time and for not finishing this earlier, but now it feels good to have it merged. We still have to polish some things, like checking that the connection is still ok, so all feedback is very welcome and valuable.

I need to update the docs at home-assistant.io, and then I might make a new post about the release.

BIG THANKS to everyone that tested this:
@drock1985
@Dave-Dan
@Andrew-Sanjanwala
@subutux

[subutux]

@martinhjelmare As soon as i have the time (and clean up my code because it was a hack) i'll create a pull request. i've mine also at 0.02 and my (pi zero) CPU is now at around 8-10%.