Any plan to implement Ethernet GW?



  • @martinhjelmare

    Hi,

    This is the error I am getting when starting up HASS.

    (hasstest) pi@pitest:~/dev/home-assistant $ hass
    Config directory: /home/pi/.homeassistant
    16-03-05 20:07:56 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: service=turn_off, domain=homeassistant>
    16-03-05 20:07:57 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: service=turn_on, domain=homeassistant>
    16-03-05 20:07:57 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: service=toggle, domain=homeassistant>
    16-03-05 20:07:57 INFO (MainThread) [homeassistant.bootstrap] Home Assistant core initialized
    16-03-05 20:07:57 INFO (MainThread) [homeassistant.loader] Loaded history from homeassistant.components.history
    16-03-05 20:07:57 INFO (MainThread) [homeassistant.loader] Loaded recorder from homeassistant.components.recorder
    16-03-05 20:07:57 INFO (MainThread) [homeassistant.loader] Loaded http from homeassistant.components.http
    16-03-05 20:07:57 INFO (MainThread) [homeassistant.loader] Loaded introduction from homeassistant.components.introduction
    16-03-05 20:07:57 INFO (MainThread) [homeassistant.loader] Loaded updater from homeassistant.components.updater
    16-03-05 20:07:57 INFO (MainThread) [homeassistant.loader] Loaded conversation from homeassistant.components.conversation
    16-03-05 20:07:57 INFO (MainThread) [homeassistant.loader] Loaded mysensors from homeassistant.components.mysensors
    16-03-05 20:07:59 INFO (MainThread) [homeassistant.loader] Loaded logbook from homeassistant.components.logbook
    16-03-05 20:07:59 INFO (MainThread) [homeassistant.loader] Loaded sun from homeassistant.components.sun
    16-03-05 20:07:59 INFO (MainThread) [homeassistant.loader] Loaded discovery from homeassistant.components.discovery
    16-03-05 20:07:59 INFO (MainThread) [homeassistant.loader] Loaded frontend from homeassistant.components.frontend
    16-03-05 20:07:59 INFO (MainThread) [homeassistant.loader] Loaded api from homeassistant.components.api
    16-03-05 20:07:59 INFO (MainThread) [homeassistant.components.introduction]
    
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
            Hello, and welcome to Home Assistant!
    
            We'll hope that we can make all your dreams come true.
    
            Here are some resources to get started:
    
             - Configuring Home Assistant:
               https://home-assistant.io/getting-started/configuration/
    
             - Available components:
               https://home-assistant.io/components/
    
             - Troubleshooting your configuration:
               https://home-assistant.io/getting-started/troubleshooting-configuration/
    
             - Getting help:
               https://home-assistant.io/help/
    
            This message is generated by the introduction component. You can
            disable it in configuration.yaml.
    
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
    16-03-05 20:07:59 INFO (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=introduction>
    16-03-05 20:07:59 INFO (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=recorder>
    16-03-05 20:07:59 INFO (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=http>
    16-03-05 20:07:59 INFO (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=history>
    16-03-05 20:07:59 INFO (MainThread) [requests.packages.urllib3.connectionpool] Starting new HTTPS connection (1): pypi.python.org
    16-03-05 20:08:00 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: old_state=None, new_state=<state updater.updater=0.14.2; friendly_name=Update Available @ 20:08:00 05-03-2016>, entity_id=updater.updater>
    16-03-05 20:08:00 INFO (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=updater>
    16-03-05 20:08:00 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: service=process, domain=conversation>
    16-03-05 20:08:00 INFO (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=conversation>
    16-03-05 20:08:00 ERROR (MainThread) [homeassistant.components.mysensors] Missing required configuration items in mysensors: port
    16-03-05 20:08:00 ERROR (MainThread) [homeassistant.bootstrap] component mysensors failed to initialize
    16-03-05 20:08:00 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: service=log, domain=logbook>
    16-03-05 20:08:00 INFO (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=logbook>
    16-03-05 20:08:00 INFO (MainThread) [requests.packages.urllib3.connectionpool] Starting new HTTP connection (1): maps.googleapis.com
    16-03-05 20:08:01 INFO (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: old_state=None, new_state=<state sun.sun=below_horizon; next_setting=22:12:29 06-03-2016, elevation=-21.55, next_rising=10:44:01 06-03-2016, friendly_name=Sun @ 20:08:01 05-03-2016>, entity_id=sun.sun>
    16-03-05 20:08:01 INFO (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=sun>
    16-03-05 20:08:01 INFO (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=discovery>
    16-03-05 20:08:01 INFO (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=api>
    16-03-05 20:08:01 INFO (MainThread) [homeassistant.core] Bus:Handling <Event component_loaded[L]: component=frontend>
    16-03-05 20:08:01 INFO (MainThread) [homeassistant.core] Starting Home Assistant (13 threads)
    16-03-05 20:08:01 INFO (MainThread) [homeassistant.core] Bus:Handling <Event homeassistant_start[L]>
    16-03-05 20:08:01 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: service=stop, domain=homeassistant>
    16-03-05 20:08:01 INFO (MainThread) [homeassistant.core] Bus:Handling <Event service_registered[L]: service=restart, domain=homeassistant>
    16-03-05 20:08:01 INFO (Thread-17) [homeassistant.components.http] Starting web interface at http://0.0.0.0:8123
    16-03-05 20:08:01 INFO (Thread-15) [homeassistant.core] Timer:starting
    16-03-05 20:08:01 INFO (Thread-18) [homeassistant.components.http] "GET /api/stream?api_password=no_password_set&restrict=state_changed,component_loaded,service_registered HTTP/1.1" 200 -
    16-03-05 20:08:01 INFO (Thread-19) [homeassistant.components.http] "GET /api/bootstrap HTTP/1.1" 200 -^C16-03-05 20:08:01 INFO (MainThread) [homeassistant.core] Stopping
    16-03-05 20:08:01 INFO (MainThread) [homeassistant.core] Bus:Handling <Event homeassistant_stop[L]>
    16-03-05 20:08:02 INFO (Thread-16) [netdisco.service] Scanning
    ^CProcess Process-1:
    Traceback (most recent call last):
    Traceback (most recent call last):
      File "/home/pi/.virtualenvs/hasstest/bin/hass", line 9, in <module>
        load_entry_point('homeassistant', 'console_scripts', 'hass')()
      File "/home/pi/dev/home-assistant/homeassistant/__main__.py", line 330, in main
        keep_running, exit_code = run_hass_process(hass_proc)
      File "/usr/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
        self.run()
      File "/usr/lib/python3.4/multiprocessing/process.py", line 93, in run
        self._target(*self._args, **self._kwargs)
      File "/home/pi/dev/home-assistant/homeassistant/__main__.py", line 248, in setup_and_run_hass
        exit_code = int(hass.block_till_stopped())
    TypeError: 'bool' object is not iterable
      File "/home/pi/dev/home-assistant/homeassistant/core.py", line 100, in block_till_stopped
        self.stop()
      File "/home/pi/dev/home-assistant/homeassistant/core.py", line 107, in stop
        self.pool.stop()
      File "/home/pi/dev/home-assistant/homeassistant/util/__init__.py", line 364, in stop
        self.block_till_done()
      File "/home/pi/dev/home-assistant/homeassistant/util/__init__.py", line 358, in block_till_done
        self._work_queue.join()
      File "/usr/lib/python3.4/queue.py", line 86, in join
        self.all_tasks_done.wait()
      File "/usr/lib/python3.4/threading.py", line 290, in wait
        waiter.acquire()
    KeyboardInterrupt
    
     mysensors:
       gateways:
         - device: '192.168.1.10'
           persistence_file: '~/.homeassistant/mysensors3.json'
           tcp_port: 5003
       debug: true
       persistence: true
       version: '1.5'
    

  • Plugin Developer

    @drock1985

    Did you checkout correct branch?

    git fetch origin mysensors-tcp-gateway
    git checkout mysensors-tcp-gateway
    

    You should remove the lib directory in ~/.homeassistant/ before you start hass again after the git checkout also.

    rm -r ~/.homeassistant/lib
    


  • Ugh, forgot to run the checkout command. palm smacks head

    Booting now, will report back in a minute if it is working or not.


  • Plugin Developer

    @drock1985

    Don't forget to use the correct ip address of your gateway in your network.



  • @martinhjelmare

    It works! Found my motion sensor with ease. Will start adding lights now shortly

    Thanks again, loving this.

    EDIT: Quick question: To autostart this, would I just use the normal Home-Assistant instructions in the virtualenv, or is there another method to do it?


  • Plugin Developer

    @drock1985

    👍

    When you have time, can you run git log and tell me the top commit message, so I know which version you are using.

    Regarding autostart, no I don't think you can use the normal instructions together with the virtualenv. You could write a script, that changes to the virtualenv and starts hass, and run that script at login or similar.

    If you don't have another production install of home assistant on the computer, you can skip the virtualenv alltogether and just do:

    pip3 install --user "https://github.com/MartinHjelmare/home-assistant/archive/mysensors-tcp-gateway.zip"
    

    or if you want to install as root:

    sudo pip3 install "https://github.com/MartinHjelmare/home-assistant/archive/mysensors-tcp-gateway.zip"
    

    This way you should be able to autostart the usual way, per instructions on the web.

    But then if you want to test an update from me, you have to make sure you uninstall everything correctly and manually remove ~/.homeassistant/lib before you install again.



  • @martinhjelmare

    Here is the info you requested:

    commit d07f8097a6186611371b8bc918fcb0d436bb39ba
    Author: MartinHjelmare <marhje52@kth.se>
    Date:   Sat Feb 27 04:55:38 2016 +0100
    
        Add mysensors tcp ethernet gateway
    
        * Replace CONF_PORT with CONF_DEVICE and ATTR_PORT with ATTR_DEVICE.
        * Add tcp_port in config.
        * Use fork pymysensors repo, temporarily.
        * Try to guess if tcp or serial gateway is configured, by validating
            device name as an ip address. If successful setup tcp gateway, if it
            fails, setup serial gateway.
        * Update device_state_attributes to show correct device, ethernet or
            serial.
    
    commit 278fdc0983a8528945e13cf912da82f3bc5fe14c
    Merge: 6dc49df 8ffa368
    Author: Daniel Høyer Iversen <daniel.h.iversen@ntnu.no>
    Date:   Fri Feb 26 10:49:10 2016 +0100
    
        Merge pull request #1385 from balloob/rfxtrx
    
        improve rfxtrx sensor
    :
    

  • Plugin Developer

    @drock1985

    Sorry, I forgot I've only updated the pymysensors project, so you checking the home-assistant git log is no use. Well, I think your're running the lastest version anyway, since you got it working just now. Everything should be ok.

    Let me know if you experience any problems.

    If everything seems ok for you after some time of testing, I don't see why we can't release this very soon. My limited testing after the lastest update, show no problems.



  • Hey @martinhjelmare,

    I finally got this working.

    I'm doing tests with a couple of multi-sensors (Temp, Hum, Light & motion). The discovery worked very well, no issue at all after following all the above steps.

    What I'm not seeing wotking are the updates. I always see the same values. I'm checking messages with MYSController and I see the GW is receiving the events and updates but I can't see new values showing in the portal.

    Still testing ...


  • Plugin Developer

    @Dave-Dan

    Do you mean that the sensors are not showing the latest values in the gui?

    If the last value from a sensor is the same as the previous value, a state change won't happen. So the value will be reported as being as old as the previous value. Could this explain what you see?



  • actually, I just got it working ...

    yes, I was referrring to that. The way I have built the sensor is that everytime a movement is detected is sending all sensors statuses to the GW. This sensors are designed to be plugged (no battery powered) so no concern on power consumption.

    I restarted the server and now updates are working.

    What I'm seeing in the log is constantly this message:
    16-03-14 10:39:53 INFO (Thread-9) [mysensors.mysensors] Permission denied when writing to /home/pi/dev/home-assistant/path/mysensors3.json

    If I'm not wrong this is the persistence.

    Apart of this I now see updates coming in ...


  • Plugin Developer

    Good!

    The path to the persistence file must exist, not the file itself, but the directory tree above, and hass should have write permission to it.

    I usually put the file in the same directory as configuration.yml.



  • Ok, couple of things that I'm seeing now.

    1. Everytime I restart the server the MySensor nodes are no longer in the system. I have to restart them to get them in again. All of them are in the persistence file.

    2. Now i can only have 1 node added.
      While in the persistence file I see multiple nodes, once I add one of them I can't add anymore. No error if fired (at least not that I've seen).


  • Plugin Developer

    @Dave-Dan

    Can you attach the persistence file here and post your config, and I'll take a look?



  • Sure!

    Consistence file:

    {"156": {"sketch_version": null, "children": {}, "battery_level": 0, "sketch_name": null, "type": 17, "sensor_id": 156}, "102": {"sketch_version": null, "children": {}, "battery_level": 0, "sketch_name": null, "type": 17, "sensor_id": 102}}
    

    the Configuration is very light. Actually is the by default config and i just added this to test MySensors:

    mysensors:
      gateways:
        - device: 192.168.1.X
          persistence_file: '/home/pi/.homeassistant/MSPersistence.json'
          tcp_port: 5003
      debug: true
      persistence: true
      version: '1.5'
    

    let me know if you need anything else.


  • Plugin Developer

    In the persistence file I can see two issues.

    Sketch name is null for both nodes.
    Both nodes don't have any children.

    Are you using a non default example sketch, if so can you post it?



  • hey Martin,

    that's correct, those are the 2 items I saw that was little curious about.

    The sketch I'm using, while custom, is a mix of different sketches that can be found in the MySensors page.

    As you'll see below, each node (which are identical) have all the information (Sketch name, version, etc). And all of them have 4 childs:
    - Ambient Humidity
    - Ambient Temperature
    - Ambient Light
    - Movement (Motion Sensor Trip).

    But I might have something wrong so ... happy to learn! 🙂

    #include <SPI.h>
    #include <MySensor.h>  
    #include <DHT.h>  
    
    #define NODE_ID 102
    
    #define SKETCH_NAME "dyaSensor (HTLM) v1.2"
    #define SKETCH_VERSION "1.2"
    
    #define CHILD_ID_HUM 0
    #define CHILD_ID_TEMP 1
    #define CHILD_ID_LIGHT 2
    #define CHILD_ID_MOV 3
    
    #define HUMIDITY_SENSOR_DIGITAL_PIN 4
    #define DIGITAL_INPUT_SENSOR 3   // The digital input you attached your motion sensor.  (Only 2 and 3 generates interrupt!)
    #define INTERRUPT DIGITAL_INPUT_SENSOR-2 // Usually the interrupt = pin -2 (on uno/nano anyway)
    #define LIGHT_SENSOR_ANALOG_PIN 0
    
    #defin TEMP_CORRECTION 2
    
    unsigned long SLEEP_TIME = 1000; // Sleep time between reads (in milliseconds)
    
    MySensor gw;
    DHT dht;
    float lastTemp;
    float lastHum;
    int lastLightLevel;
    boolean metric = true; 
    
    //Building Messages
    MyMessage msgHum(CHILD_ID_HUM, V_HUM);            // Initialize humidity message
    MyMessage msgTemp(CHILD_ID_TEMP, V_TEMP);         // Initialize temperature message
    MyMessage msgLum(CHILD_ID_LIGHT, V_LIGHT_LEVEL);  // Initialize Light message
    MyMessage msgMov(CHILD_ID_MOV, V_TRIPPED);        // Initialize motion message
    
    void setup()  
    { 
    
      gw.begin(NULL, NODE_ID, false);
      dht.setup(HUMIDITY_SENSOR_DIGITAL_PIN); 
    
      // Send the Sketch Version Information to the Gateway
      gw.sendSketchInfo(SKETCH_NAME, SKETCH_VERSION);
    
      pinMode(DIGITAL_INPUT_SENSOR, INPUT);      // sets the motion sensor digital pin as input
    
      // Register all sensors to gw (they will be created as child devices)
      Serial.println("Presenting Humidity Sensor");
      gw.present(CHILD_ID_HUM, S_HUM, "Humidity Sensor");
      
      Serial.println("Presenting Temperature Sensor");
      gw.present(CHILD_ID_TEMP, S_TEMP, "Temperature Sensor");
      
      Serial.println("Light Level Sensor");
      gw.present(CHILD_ID_LIGHT, S_LIGHT_LEVEL, "Light Level Sensor");
      
      Serial.println("Presenting Motion Sensor");
      gw.present(CHILD_ID_MOV, S_MOTION, "Motion Sensor");
      
      metric = gw.getConfig().isMetric;
    }
    
    void loop()      
    {  
    
      // By calling process() you route messages in the background
      gw.process();
      
      delay(dht.getMinimumSamplingPeriod());
    
      float temperature = dht.getTemperature()-TEMP_CORRECTION;
      if (isnan(temperature)) {
          Serial.println("Failed reading temperature from DHT");
      } else if (temperature != lastTemp) {
        lastTemp = temperature;
        if (!metric) {
          temperature = dht.toFahrenheit(temperature);
        }
        gw.send(msgTemp.set(temperature, 1));
        Serial.print("- T: ");
        Serial.println(temperature);
      }
      
      float humidity = dht.getHumidity();
      if (isnan(humidity)) {
          Serial.println("Failed reading humidity from DHT");
      } else if (humidity != lastHum) {
          lastHum = humidity;
          gw.send(msgHum.set(humidity, 1));
          Serial.print("- H: ");
          Serial.println(humidity);
      }
    
      int lightLevel = (1023-analogRead(LIGHT_SENSOR_ANALOG_PIN))/10.23; 
      if (lightLevel != lastLightLevel) {
        lastLightLevel = lightLevel;
        gw.send(msgLum.set(lightLevel));
        Serial.print("- L: ");
        Serial.println(lightLevel);    
      }
    
      // Read digital motion value
      boolean tripped = digitalRead(DIGITAL_INPUT_SENSOR) == HIGH; 
      gw.send(msgMov.set(tripped?"1":"0"));  // Send tripped value to gw 
      Serial.println(tripped?"- Movement Detected":"- No Movement Detected");
    
      gw.sleep(INTERRUPT,CHANGE, SLEEP_TIME);
    }
    
    


  • Hey @martinhjelmare

    Does the above reply to @drock1985 means that I can do a basic installation as per https://home-assistant.io/

    pip3 install homeassistant
    hass --open-ui
    

    followed by

    sudo pip3 install "https://github.com/MartinHjelmare/home-assistant/archive/mysensors-tcp-gateway.zip"
    

    and will have a valid testing env without going into all the above steps of creating a virtualenv??

    If so I'd rather more this avenue because I can clean up everything with a new installation, install the zip and have a new testing env in minutes 🙂


  • Plugin Developer

    @Dave-Dan

    I can't see any big issues with your sketch, just one small:

    I guess this is a typo:

    #defin TEMP_CORRECTION 2
    

    Since your sketch looks ok, I think something must have gone wrong with the communication between node and gateway. Have you been monitoring the comm, with a serial monitor, on the node side for example, to make sure all messages are sent ok?


  • Plugin Developer

    @Dave-Dan

    The zip file you install from my github account, is the mysensors tcp gateway branch of home-assistant. So you shouldn't install home-assistant first, if you're going to install my branch.

    If you don't have any other version of home-assistant that you wish to keep, on the same computer, you can install my branch directly. You just have to make sure you uninstall it and remove ~/.homeassistant/lib before installing an update from me, if I commit some updates.

    Yeah, setting up a dev machine can be a hassle the first time, and not so useful if you're not developing with it.



  • @martinhjelmare

    thanks for the hint ... definetly a typo :$

    Yes, I'm monitoring the GW with MYSController as my GW is ethernet and I see all messages showing with no issue. But, I'll be happy to review again. 🙂


  • Plugin Developer

    @Dave-Dan

    Yes, please start from the beginning:

    Remove the persistence file
    Start hass
    Wait until you see that hass has (😄) connected to the gateway
    Start a node
    Start the next node

    If you can post the serial log from that here, and also the log from hass, that would be great.



  • @martinhjelmare

    Haven't had any issues with the sensors I have on my Dev unit; so far so good.

    Are you thinking of releasing the ethernet GW code in this weeks update?



  • @martinhjelmare

    Though I'm eager to try, I can't actually launch the Ethernet Gateway branch in a Windows environment. The best error I can provide is the generic failed to create process on calling hass. Platform is Windows Home Server 2011 running Python 3.5.1. All previous and current release branches of home assistant launch without error.

    Any thoughts or other steps I can take to debug it further?


  • Plugin Developer

    @drock1985

    Good to hear. Have you tried connecting more than one node to the gateway at the same time?

    Unfortunately I don't think it will make this release, cause I won't have time until Sunday, and the release might be finished before that. I will add some updates this weekend that you are welcome to try. After that we should be ready for release.


  • Plugin Developer

    @Andrew-Sanjanwala

    My experience with python on Windows is very limited. How do you normally start home assistant?



  • @martinhjelmare

    I typically launch either using hass or hass --open-ui from an administrative powershell session. No-go in either of these two cases.


  • Plugin Developer

    @Andrew-Sanjanwala

    Did you install directly from my github branch using pip?



  • Alright, false alarm. Root issue in my case was a faulty environment and/or operator that resulted in binaries being compiled for the wrong platform.

    Once corrected, it compiled and launched your branch and connected to the gateway without incident. Next up is experimenting with the nodes and actuators. I'll see if the persistence file can be written to.


  • Plugin Developer



  • Hey @martinhjelmare,

    I've been able to do some tests this time.

    From a Sensoring standpoint (retreive information from sensors to show in dashboard) works perfetly fine.

    I found the issue that I was referring before.

    Let's say you have already included a number of nodes and everything is working but for any reason you want to reset and start again. My logic was to remove the MSPersistance.json file, create a new one and the system will reconfigure again.

    What I'm seeing is that if you do that the system enters in a limbo situation where no nodes are added but those that were beforer are not added either. you can't see them in the dashboard nor in the Developers Tools section.

    Next step is to start testing relays, dimmers, etc.

    Let me know if you need more detils on the scenario.

    thanks!


  • Plugin Developer

    @Dave-Dan said:

    My logic was to remove the MSPersistance.json file, create a new one and the system will reconfigure again.

    Can you describe more in detail how you did this? Do you have an example persistence file before and after your modifications?



  • Sure!

    the persistence once I've added some nodes looks like this:

    {"104": {"sketch_version": "1.0", "children": {"0": {"id": 0, "values": {"1": "38.0"}, "type": 7}, "1": {"id": 1, "values": {"0": "22.0"}, "type": 6}, "2": {"id": 2, "values": {"23": "34"}, "type": 16}, "3": {"id": 3, "values": {"16": "0"}, "type": 1}}, "battery_level": 0, "sensor_id": 104, "type": 17, "sketch_name": "dyaSensor (HTLM)"}, "106": {"sketch_version": "1.0", "children": {"0": {"id": 0, "values": {"1": "37.0"}, "type": 7}, "1": {"id": 1, "values": {"0": "22.0"}, "type": 6}, "2": {"id": 2, "values": {"23": "5"}, "type": 16}, "3": {"id": 3, "values": {"16": "0"}, "type": 1}}, "battery_level": 9, "sensor_id": 106, "type": 17, "sketch_name": "dyaSensor (HTLM)"}, "156": {"sketch_version": null, "children": {}, "battery_level": 0, "sensor_id": 156, "type": 17, "sketch_name": null}, "101": {"sketch_version": "1.2", "children": {"0": {"id": 0, "values": {"1": "34.0"}, "type": 7}, "1": {"id": 1, "values": {"0": "21.0"}, "type": 6}, "2": {"id": 2, "values": {}, "type": 16}, "3": {"id": 3, "values": {"16": "0"}, "type": 1}}, "battery_level": 0, "sensor_id": 101, "type": 17, "sketch_name": "dyaSensor (HTLM) v1.2"}, "102": {"sketch_version": "1.0", "children": {"0": {"id": 0, "values": {"1": "36.0"}, "type": 7}, "1": {"id": 1, "values": {"0": "18.0"}, "type": 6}, "2": {"id": 2, "values": {"23": "69"}, "type": 16}, "3": {"id": 3, "values": {"16": "0"}, "type": 1}}, "battery_level": 2, "sensor_id": 102, "type": 17, "sketch_name": "dyaSensor (HTLM)"}}
    

    after my modification (meaning, clear everything) looks this way:

    {}
    

  • Plugin Developer

    @Dave-Dan

    Did you do it like this?

    1. Stop hass.
    2. Remove old persistence file.
    3. Create new persistence file with the same name and path, containing an empty JSON object, "{}".
    4. Start hass.
    5. Start node. Result: Messages are coming in to gateway, but node sensors are not added to entities nor shown in GUI.


  • Exactly that.

    maybe something to do with the fact that those items are already included in the DB (for history, etc) and the system can't add them again? (just guessing)

    One more thing.

    If I move the file and create a new one instead of deleting it. And after trying to add the items again I stop everything and restore the backup everything works again.


  • Plugin Developer

    @Dave-Dan

    I don't know yet, but I'll try to reproduce this and figure it out. Thanks for this report, and taking your time to test this.

    I haven't had time to push my latest commits, that I had planned. I want to just test them quickly on my side before I push. But since you experience this problem, I think it's best to troubleshoot this first, before I push any new commits.

    I'll post back here, once I have more info.



  • hey @martinhjelmare,

    would it be possible to have an updated compilation with the 0.16 versión? I think the one you have above is 0.15 still (or older, right?)

    I'd like to do more testing and integrate other items (for example the new Vera integration) with MySensors actuations, etc.

    thanks in advance.


  • Plugin Developer

    @Dave-Dan

    I'd like to investigate the issue you reported before I push any new commits or rebase off dev. Sorry to keep you waiting.



  • ahh, ok, makes sense.

    no problem at all! 🙂



  • @Dave-Dan

    out of curiosity, did you try a different name for the persistence file after deleting your first one, or did you use the same name?



  • hey @drock1985,

    No I didn't test that before, but I just did with the same results 😞

    I can see in the log how messages are arriving to HA (see below). It's just I can't see them either in the group I created for them or in the Developer Tools.

    16-03-31 09:42:37 DEBUG (Thread-20) [mysensors.mysensors] Received 101;1;1;0;0;20.0
    
    16-03-31 09:42:37 DEBUG (Thread-20) [mysensors.mysensors] Received 101;0;1;0;1;36.0
    
    16-03-31 09:42:37 DEBUG (Thread-20) [mysensors.mysensors] Received 101;3;1;0;16;0
    

    If I go back and restore the previous file I can see them on both the group and the Developer Tools section.

    @martinhjelmare, hope this helps as well ... is is there any specific test you want me to perform please let me know.


  • Plugin Developer

    @Dave-Dan

    I finally had some time to test this again. Sorry for the long delay.

    I followed the steps here:

    @martinhjelmare said:

    @Dave-Dan

    Did you do it like this?

    1. Stop hass.
    2. Remove old persistence file.
    3. Create new persistence file with the same name and path, containing an empty JSON object, "{}".
    4. Start hass.
    5. Start node. Result: Messages are coming in to gateway, but node sensors are not added to entities nor shown in GUI.

    but I'm not able to reproduce your experienced problem. On my end, the node and sensor are added ok. Do you have any more clues what might be different on your end?

    HA config:

    mysensors:
      gateways:
        - device: '192.168.1.18'
          persistence_file: '/home/martin/.homeassistant/mysensors3.json'
          tcp_port: 5003
      debug: true
      persistence: true
      version: '1.5'
      optimistic: false
    

    console log:

    configuration.yaml  home-assistant.db  home-assistant.log  known_devices.yaml  lib  mysensors3.json
    martin@martin-nuc:~/.homeassistant$ cat mysensors3.json
    {"1": {"children": {"1": {"values": {"40": "ffffff", "2": "0", "3": "0"}, "id": 1, "type": 26}}, "sketch_version": "1.1", "sketch_name": "DimmableLED", "battery_level": 0, "sensor_id": 1, "type": 17}}
    martin@martin-nuc:~/.homeassistant$ rm mysensors3.json
    martin@martin-nuc:~/.homeassistant$ ls
    configuration.yaml  home-assistant.db  home-assistant.log  known_devices.yaml  lib
    martin@martin-nuc:~/.homeassistant$ atom .
    martin@martin-nuc:~/.homeassistant$ ls
    configuration.yaml  home-assistant.db  home-assistant.log  known_devices.yaml  lib  mysensors3.json
    martin@martin-nuc:~/.homeassistant$ cat mysensors3.json
    {}
    martin@martin-nuc:~/.homeassistant$ cat mysensors3.json
    {"1": {"battery_level": 0, "sensor_id": 1, "sketch_name": "DimmableLED", "children": {"1": {"values": {"40": "ffffff", "2": "0", "3": "0"}, "id": 1, "type": 26}}, "sketch_version": "1.1", "type": 17}}
    

    sensor serial log:

    send: 1-1-0-0 s=255,c=3,t=15,pt=2,l=2,sg=0,st=ok:0
    send: 1-1-0-0 s=255,c=0,t=17,pt=0,l=5,sg=0,st=ok:1.5.1
    send: 1-1-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0,st=ok:0
    read: 0-0-1 s=255,c=3,t=15,pt=2,l=2,sg=0:0
    read: 0-0-1 s=255,c=3,t=6,pt=0,l=1,sg=0:M
    sensor started, id=1, parent=0, distance=1
    send: 1-1-0-0 s=1,c=0,t=26,pt=0,l=0,sg=0,st=ok:
    send: 1-1-0-0 s=255,c=3,t=11,pt=0,l=11,sg=0,st=ok:DimmableLED
    send: 1-1-0-0 s=255,c=3,t=12,pt=0,l=3,sg=0,st=ok:1.1
    send: 1-1-0-0 s=1,c=2,t=3,pt=0,l=0,sg=0,st=ok:
    send: 1-1-0-0 s=1,c=1,t=2,pt=2,l=2,sg=0,st=ok:0
    send: 1-1-0-0 s=1,c=1,t=3,pt=2,l=2,sg=0,st=ok:0
    send: 1-1-0-0 s=1,c=1,t=40,pt=0,l=6,sg=0,st=ok:ffffff
    read: 0-0-1 s=1,c=1,t=2,pt=0,l=1,sg=0:1
    Changing level to 100, from 0
    send: 1-1-0-0 s=1,c=1,t=2,pt=2,l=2,sg=0,st=ok:1
    send: 1-1-0-0 s=1,c=1,t=3,pt=2,l=2,sg=0,st=ok:100
    read: 0-0-1 s=1,c=1,t=2,pt=0,l=1,sg=0:0
    Changing level to 0, from 100
    send: 1-1-0-0 s=1,c=1,t=2,pt=2,l=2,sg=0,st=ok:0
    send: 1-1-0-0 s=1,c=1,t=3,pt=2,l=2,sg=0,st=ok:0
    

    hass log:

    16-04-10 14:23:01 INFO (Thread-8) [mysensors.mysensors] Trying to connect to ('192.168.1.18', 5003)
    16-04-10 14:23:01 INFO (Thread-21) [homeassistant.core] Timer:starting
    16-04-10 14:23:01 INFO (Thread-8) [mysensors.mysensors] Connected to ('192.168.1.18', 5003)
    16-04-10 14:23:03 INFO (Thread-23) [homeassistant.components.http] "GET /api/stream?api_password=no_password_set&restrict=state_changed,component_loaded,service_registered HTTP/1.1" 200 -
    16-04-10 14:23:03 INFO (Thread-24) [homeassistant.components.http] "GET /api/bootstrap HTTP/1.1" 200 -
    16-04-10 14:23:30 INFO (Thread-3) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sun.sun, old_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.69, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>, new_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.66, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>>
    16-04-10 14:24:30 INFO (Thread-2) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sun.sun, old_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.66, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>, new_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.6, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>>
    16-04-10 14:25:30 INFO (Thread-16) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=sun.sun, old_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.6, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>, new_state=<state sun.sun=above_horizon; next_rising=03:40:47 11-04-2016, icon=mdi:air-conditioner, elevation=35.54, next_setting=17:52:06 10-04-2016, friendly_name=Customized Sun @ 14:23:01 10-04-2016>>
    16-04-10 14:25:41 INFO (Thread-25) [homeassistant.components.http] "GET /static/frontend-833d09737fec24f9219efae87c5bfd2a.html HTTP/1.1" 200 -
    16-04-10 14:25:42 INFO (Thread-26) [homeassistant.components.http] "GET /static/favicon.ico HTTP/1.1" 200 -
    16-04-10 14:25:42 INFO (Thread-27) [homeassistant.components.http] "GET /api/bootstrap HTTP/1.1" 200 -
    16-04-10 14:25:42 INFO (Thread-28) [homeassistant.components.http] "GET /api/stream?api_password=no_password_set&restrict=state_changed,component_loaded,service_registered HTTP/1.1" 200 -
    16-04-10 14:25:53 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
    16-04-10 14:25:53 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
    16-04-10 14:25:53 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
    16-04-10 14:25:53 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
    16-04-10 14:25:55 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
    16-04-10 14:25:55 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
    16-04-10 14:25:55 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
    16-04-10 14:25:55 INFO (Thread-8) [homeassistant.components.mysensors] No sketch_name: node 1
    16-04-10 14:25:59 INFO (Thread-8) [homeassistant.components.mysensors] Adding new devices: <Entity DimmableLED 1.1: off>
    16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=None, new_state=<state light.dimmableled_11=unavailable; friendly_name=DimmableLED 1.1 @ 14:25:59 10-04-2016>>
    16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=None, new_state=<state group.all_lights=unknown; hidden=True, entity_id=[], auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>>
    16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=<state group.all_lights=unknown; hidden=True, entity_id=[], auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>, new_state=None>
    16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=None, new_state=<state group.all_lights=unknown; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>>
    16-04-10 14:25:59 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=unavailable; friendly_name=DimmableLED 1.1 @ 14:25:59 10-04-2016>, new_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:25:59 10-04-2016>>
    16-04-10 14:25:59 INFO (Thread-17) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=<state group.all_lights=unknown; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>, new_state=<state group.all_lights=off; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>>
    16-04-10 14:25:59 INFO (Thread-23) [homeassistant.components.api] Found broken event stream to 127.0.0.1, cleaning up
    16-04-10 14:26:19 INFO (Thread-29) [homeassistant.core] Bus:Handling <Event call_service[L]: service=turn_on, domain=homeassistant, service_data=entity_id=light.dimmableled_11, service_call_id=140012061724232-1>
    16-04-10 14:26:19 INFO (Thread-15) [homeassistant.core] Bus:Handling <Event call_service[L]: service=turn_on, domain=light, service_data=entity_id=['light.dimmableled_11'], service_call_id=140012061724232-2>
    16-04-10 14:26:19 INFO (Thread-9) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=140012061724232-2>
    16-04-10 14:26:19 INFO (Thread-15) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=140012061724232-1>
    16-04-10 14:26:19 INFO (Thread-29) [homeassistant.components.http] "POST /api/services/homeassistant/turn_on HTTP/1.1" 200 -
    16-04-10 14:26:21 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:25:59 10-04-2016>, new_state=<state light.dimmableled_11=off; V_STATUS=on, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:25:59 10-04-2016>>
    16-04-10 14:26:22 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=off; V_STATUS=on, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:25:59 10-04-2016>, new_state=<state light.dimmableled_11=on; V_RGB=ffffff, V_PERCENTAGE=100, battery_level=0, rgb_color=[255, 255, 255], child_id=1, V_STATUS=on, brightness=255, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, node_id=1 @ 14:26:22 10-04-2016>>
    16-04-10 14:26:22 INFO (Thread-7) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=<state group.all_lights=off; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:25:59 10-04-2016>, new_state=<state group.all_lights=on; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:26:22 10-04-2016>>
    16-04-10 14:26:26 INFO (Thread-30) [homeassistant.core] Bus:Handling <Event call_service[L]: service=turn_off, domain=homeassistant, service_data=entity_id=light.dimmableled_11, service_call_id=140012061724232-3>
    16-04-10 14:26:26 INFO (Thread-6) [homeassistant.core] Bus:Handling <Event call_service[L]: service=turn_off, domain=light, service_data=entity_id=['light.dimmableled_11'], service_call_id=140012061724232-4>
    16-04-10 14:26:26 INFO (Thread-13) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=140012061724232-4>
    16-04-10 14:26:26 INFO (Thread-6) [homeassistant.core] Bus:Handling <Event service_executed[L]: service_call_id=140012061724232-3>
    16-04-10 14:26:26 INFO (Thread-30) [homeassistant.components.http] "POST /api/services/homeassistant/turn_off HTTP/1.1" 200 -
    16-04-10 14:26:27 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=on; V_RGB=ffffff, V_PERCENTAGE=100, battery_level=0, rgb_color=[255, 255, 255], child_id=1, V_STATUS=on, brightness=255, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, node_id=1 @ 14:26:22 10-04-2016>, new_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=100, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:26:27 10-04-2016>>
    16-04-10 14:26:27 INFO (Thread-18) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=group.all_lights, old_state=<state group.all_lights=on; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:26:22 10-04-2016>, new_state=<state group.all_lights=off; hidden=True, entity_id=('light.dimmableled_11',), auto=True, order=1, friendly_name=all lights @ 14:26:27 10-04-2016>>
    16-04-10 14:26:28 INFO (Thread-8) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=light.dimmableled_11, old_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=100, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:26:27 10-04-2016>, new_state=<state light.dimmableled_11=off; V_STATUS=off, node_id=1, V_RGB=ffffff, V_PERCENTAGE=0, battery_level=0, device=192.168.1.18:5003, friendly_name=DimmableLED 1.1, child_id=1 @ 14:26:27 10-04-2016>>
    


  • hey Martin.

    Sorry, I've been traveling for a while.

    So I just rebuilt my environment and I can't repro the issue anymore.

    If you can't either and no one else is experiencing this I think we can close this down. My environment might be the only thing that is creating the issue.

    If the issue happen again I can update with the new information.


  • Plugin Developer

    @Dave-Dan

    Great! Then I'll move forward and push my latest changes to my github branch. I'll post here when that is done. You're welcome to test these by the update instructions above.

    If nothing comes up, my plan is to have this merged in the next release of home assistant.



  • Hi,

    I've also tested your pymysensors version with tcp gateway. I've had an excessive cpu usage on my pi Zero (actually, a constant 100%). After some debugging I've found that I've had to add a sleep of 0.2 to the while loop of the top socket readout. My cpu usage returned normal.

    What I've also encountered is that when the tcp gateway looses connection, pymysensors still thinks it's connected. I've buildin an active check by requesting the gateway version every 10 seconds.

    Would you like me to make a pull request ?


  • Plugin Developer

    @subutux

    Thanks for the feedback!

    I've got some commits that are not pushed yet with some changes, but a pull request should be good anyhow. Great!


  • Plugin Developer

    I've now pushed some updates to my ethernet gateway branch of pymysensors and rebased my home-assistant mysensors-tcp-gateway branch off latest upstream dev branch and pushed that to my github.

    To upgrade follow the instructions below.

    @martinhjelmare said:

    I've updated the branch at my fork of pymysensors. If you have installed the ethernet branch of my fork of home-assistant, you should uninstall/reinstall that, or just make a new virtual environment and install again. You should remove the lib directory in your home assistant config directory as well.

    Eg:

    rm -r ~/.homeassistant/lib
    mkvirtualenv -p /usr/bin/python3 hasstest2
    cd ~/dev/home-assistant
    git branch -D mysensors-tcp-gateway
    git fetch origin mysensors-tcp-gateway
    git checkout mysensors-tcp-gateway
    pip3 install -e .
    pip3 install colorlog
    pip3 install python-Levenshtein
    

  • Plugin Developer

    @subutux

    Please rebase your pymysensors branch off my latest updates, before you make the PR. Thanks.


  • Plugin Developer

    @subutux

    I went ahead and added your fix for the 100% cpu usage. Although I settled for a very short 0.02 seconds. It was enough for my intel based processor. If you still experience problems and think we should increase it, let me know.

    Please make a PR for the check of the connection here https://github.com/theolind/pymysensors if you have time.

    My plan is to try and have this make the release of home assistant that is planned during this week. I think it should be good enough for a first version now. We can add polish for the next release.



  • Hi @martinhjelmare

    I see HA got released early, and that the ethernetgateway was added. I'll check it out later on tonight but, thanks again for all the work you do on the MySensors project.


  • Plugin Developer

    @drock1985

    Yeah, finally! I've been a bit disappointed with myself, that it took a long time and for not finishing this earlier, but now it feels good to have it merged. We still have to polish some things, like checking that the connection is still ok, so all feedback is very welcome and valuable.

    I need to update the docs at home-assistant.io, and then I might make a new post about the release.

    BIG THANKS to everyone that tested this:
    @drock1985
    @Dave-Dan
    @Andrew-Sanjanwala
    @subutux



  • @martinhjelmare As soon as i have the time (and clean up my code because it was a hack) i'll create a pull request. i've mine also at 0.02 and my (pi zero) CPU is now at around 8-10%.


Log in to reply
 

Suggested Topics

  • 1
  • 17
  • 5
  • 3
  • 5
  • 8

64
Online

11.4k
Users

11.1k
Topics

112.7k
Posts