MySensors: trying to add Things in WebThings Gateway

Nca78

@alowhum said in MySensors: trying to add Things in WebThings Gateway:

The add-on was never tested with the ethernet gateway option. It should work theoretically.. so it might just be that. I will test it when I get a chance.

You could try running the serial gateway version. If that works, then we know what the issue is.

It's not the problem I'm using an ethernet gateway on an ESP8266 and it's running great now.

@rmalbers said in MySensors: trying to add Things in WebThings Gateway:

I have it installed and everything appears fine (sensor work in Domoticx)

Are you trying to run both Domoticz and Things Gateway in parallel connecting to the same MySensors gateway ?

alowhum

@nca78 Thanks, that saves me some test work :-)

@rmalbers I believe you cannot have two controllers ingest the data at the same time. One of them will 'win', or both will get some of the messages.

A possible future feature could be the re-broadcast the incoming messages on another port (e.g. 5004), so that Domoticz could ingest those. Then you could have two controllers both getting the same MySensors data.

rmalbers

Ok, I think I tried it with domoticz shutdown but I'll try that again. Thanks to the other reply that they have this working, that gives my hope! Na78: What version of mysensor gateway code are you running? Mine is at V2.2 and I'm wondering if that might be my problem. Thanks for the help on this BTW!

rmalbers

Ok, with Domoticz shutdown I get the same stuff in the log. I'm going to see if I can tell more what's going on with wireshark. I'm thinking it has to be something stupid that I have going on, I know my MySensors gateway is good because it's working with Domoticz, the Mozilla WebThings gateway is working because it works with Dark sky weather, so I would think it's something going on with the comm between the two gateways.

rmalbers

Ok, I built a new serial USB MySensor gateway. (The old GW is unplugged and Domoticz is shutdown.) The new MS gateway is working because under the IDE I can turn on the com serial port debug and see it going, communicating with the sensors.) I move the USB serial gw to the rasp pi WebThings gateway and set the addon to: /dev/ttyUSB0 and to USB serial gateway. I get the same messages as above. I'm also seeing this in a couple of log files on the WebThings gateway: cat: /tmp/gateway.addonManager: No such device or address
cat: /tmp/gateway.plugin.mysensors-adapter: No such device or address
So it sure sounds like in this case it's not seeing or able to open that USB device. That is the device name that appears when I plug the usb serial gateway USB cable into the WebThings gateway rasp pi.

alowhum

@rmalbers Great work, that narrows things down a lot.

It seems the add-on was crashing. Based on your reporting I dove into the issue and I think I may have found a bug in how it loaded settings.

I'm uploading version 0.0.8 now. It should take a day or two before the Mozilla teams accepts the new version.

rmalbers

Wow, that sounds great! At first I was thinking it might be a permissions issue with opening the USB port but I opened permissions up and found that didn't seem to be the issue.

alowhum

@rmalbers It's been accepted. You should already automatically have the new version.

rmalbers

Ok, when I went into the WT GW the first time the addon was at V0.0.7 but there was an new 'Update' tab that I hit and the new addon version installed. With the USB gateway I get this (I think I started the cut/paste at the right spot):
2019-06-19 15:49:54.172 INFO : Loading add-on: mysensors-adapter
2019-06-19 15:49:56.494 INFO : mysensors: initialising adapter from class
2019-06-19 15:49:56.496 INFO : mysensors: IpcClient: Connected to server, registering...
2019-06-19 15:49:56.500 INFO : mysensors: IpcClient: Received manager message: b'{"messageType":"registerPluginReply","data":{"pluginId":"mysensors-adapter","ipcBaseAddr":"gateway.plugin.mysensors-adapter"}}'
2019-06-19 15:49:56.503 INFO : mysensors: IpcClient: Registered with PluginServer
2019-06-19 15:49:56.505 INFO : mysensors: AddonManagerProxy: add_adapter: mysensors-adapter
2019-06-19 15:50:13.088 INFO : About to call startPairing on MySensorsAdapter
2019-06-19 15:50:13.096 INFO : mysensors: -Connection status was present in the config data.
2019-06-19 15:50:13.098 INFO : mysensors: -Gateway choice: USB Serial gateway
2019-06-19 15:50:13.099 INFO : mysensors: Error extracting settings from config object: local variable 'ip_address' referenced before assignment
2019-06-19 15:50:13.101 INFO : mysensors: AddonMangerProxy: recv: b'{"messageType":"startPairing","data":{"timeout":60,"adapterId":"mysensors-adapter","pluginId":"mysensors-adapter"}}'
2019-06-19 15:50:18.322 INFO : Opened a new things socket
2019-06-19 15:51:13.092 INFO : Pairing timeout
2019-06-19 15:51:13.095 ERROR : Thing was not added
2019-06-19 15:51:13.113 ERROR : addNewThing cancelled
2019-06-19 15:51:13.118 INFO : mysensors: PAIRING INITIATED
2019-06-19 15:51:13.120 INFO : mysensors: Error re-requesting node ID's from devices in the MySensors network:'MySensorsAdapter' object has no attribute 't'
2019-06-19 15:51:13.121 INFO : mysensors: AddonMangerProxy: recv: b'{"messageType":"cancelPairing","data":{"adapterId":"mysensors-adapter","pluginId":"mysensors-adapter"}}'

rmalbers

This log is from an attempt (with the new addon) using 'Ethernet' GW and the IP address of the GW that I've been using with Domoticz for the last year or so, (it's at mysensor v2.2) This looks like it was communicating with the gw and thus the sensors but still didn't work. This could be because I am at V2.2 of mysensors(?).:
2019-06-19 15:33:17.874 INFO : Loading add-on: mysensors-adapter
2019-06-19 15:33:20.227 INFO : mysensors: initialising adapter from class
2019-06-19 15:33:20.229 INFO : mysensors: IpcClient: Connected to server, registering...
2019-06-19 15:33:20.233 INFO : mysensors: IpcClient: Received manager message: b'{"messageType":"registerPluginReply","data":{"pluginId":"mysensors-adapter","ipcBaseAddr":"gateway.plugin.mysensors-adapter"}}'
2019-06-19 15:33:20.236 INFO : mysensors: IpcClient: Registered with PluginServer
2019-06-19 15:33:20.238 INFO : mysensors: AddonManagerProxy: add_adapter: mysensors-adapter
2019-06-19 15:33:20.314 ERROR : mysensors: DEBUG:mysensors.persistence:Loading sensors from persistence file /home/pi/.mozilla-iot/config/mysensors-adapter-persistence.json
2019-06-19 15:33:20.319 ERROR : mysensors: DEBUG:mysensors.persistence:Saving sensors to persistence file /home/pi/.mozilla-iot/config/mysensors-adapter-persistence.json
2019-06-19 15:33:20.330 ERROR : mysensors: INFO:mysensors.gateway_tcp:Trying to connect to ('192.168.1.98', 5003)
2019-06-19 15:33:20.393 ERROR : mysensors: INFO:mysensors:Connected to <_SelectorSocketTransport fd=14 read=idle write=<idle, bufsize=0>>
2019-06-19 15:33:20.425 ERROR : mysensors: DEBUG:mysensors:Receiving 0;255;3;0;14;Gateway startup complete.
2019-06-19 15:33:20.434 ERROR : mysensors: INFO:mysensors.handler:n:0 c:255 t:3 s:14 p:Gateway startup complete.
2019-06-19 15:33:20.436 ERROR : mysensors: DEBUG:mysensors:Sending 255;255;3;0;20;
2019-06-19 15:33:20.438 ERROR : mysensors: DEBUG:mysensors:Receiving 0;255;0;0;18;2.2.0
2019-06-19 15:33:20.442 ERROR : mysensors: DEBUG:mysensors:Sending 0;255;3;0;26;0
2019-06-19 15:33:27.927 ERROR : mysensors: DEBUG:mysensors:Receiving 1;0;1;0;0;22.6
2019-06-19 15:33:27.932 ERROR : mysensors: WARNING:mysensors:Node 1 is unknown
2019-06-19 15:33:27.934 ERROR : mysensors: INFO:mysensors:Requesting new presentation for node 1
2019-06-19 15:33:27.935 ERROR : mysensors: DEBUG:mysensors:Sending 1;255;3;0;19;
2019-06-19 15:33:29.635 INFO : About to call startPairing on MySensorsAdapter
2019-06-19 15:33:29.642 INFO : mysensors: -Connection status was present in the config data.
2019-06-19 15:33:29.644 INFO : mysensors: -Gateway choice: Ethernet gateway
2019-06-19 15:33:29.646 INFO : mysensors: Selected IP address and port: 192.168.1.98
2019-06-19 15:33:29.648 INFO : mysensors: >> incoming message > internal > id: 0; child: 255; subtype: 14; payload: Gateway startup complete.
2019-06-19 15:33:29.649 INFO : mysensors: >> incoming message > presentation > id: 0; child: 255; subtype: 18; payload: 2.2.0
2019-06-19 15:33:29.651 INFO : mysensors: RECREATING DEVICES FROM PERSISTENCE
2019-06-19 15:33:29.652 INFO : mysensors: Re-requesting presentation of all nodes on the network
2019-06-19 15:33:29.654 INFO : mysensors: AddonMangerProxy: recv: b'{"messageType":"startPairing","data":{"timeout":60,"adapterId":"mysensors-adapter","pluginId":"mysensors-adapter"}}'
2019-06-19 15:33:29.656 ERROR : mysensors: DEBUG:mysensors:Sending 0;255;3;0;26;0
2019-06-19 15:33:29.695 INFO : Opened a new things socket
2019-06-19 15:33:30.333 ERROR : mysensors: DEBUG:mysensors.persistence:Saving sensors to persistence file /home/pi/.mozilla-iot/config/mysensors-adapter-persistence.json
2019-06-19 15:33:30.495 ERROR : mysensors: DEBUG:mysensors:Sending 0;255;3;0;2;
2019-06-19 15:33:30.558 ERROR : mysensors: DEBUG:mysensors:Receiving 0;255;3;0;2;2.2.0
2019-06-19 15:33:34.611 ERROR : mysensors: DEBUG:mysensors:Receiving 2;0;1;0;0;23.2
2019-06-19 15:33:34.614 ERROR : mysensors: WARNING:mysensors:Node 2 is unknown
2019-06-19 15:33:34.617 ERROR : mysensors: INFO:mysensors:Requesting new presentation for node 2
2019-06-19 15:33:34.618 ERROR : mysensors: DEBUG:mysensors:Sending 2;255;3;0;19;
2019-06-19 15:33:40.597 ERROR : mysensors: DEBUG:mysensors:Sending 0;255;3;0;2;
2019-06-19 15:33:40.689 ERROR : mysensors: DEBUG:mysensors:Receiving 0;255;3;0;2;2.2.0
2019-06-19 15:33:50.699 ERROR : mysensors: DEBUG:mysensors:Sending 0;255;3;0;2;
2019-06-19 15:33:50.820 ERROR : mysensors: DEBUG:mysensors:Receiving 0;255;3;0;2;2.2.0
2019-06-19 15:34:00.595 ERROR : mysensors: DEBUG:mysensors:Receiving 1;0;1;0;0;22.6
2019-06-19 15:34:00.599 ERROR : mysensors: WARNING:mysensors:Node 1 is unknown
2019-06-19 15:34:00.601 ERROR : mysensors: INFO:mysensors:Requesting new presentation for node 1
2019-06-19 15:34:00.602 ERROR : mysensors: DEBUG:mysensors:Sending 1;255;3;0;19;
2019-06-19 15:34:00.801 ERROR : mysensors: DEBUG:mysensors:Sending 0;255;3;0;2;
2019-06-19 15:34:00.875 ERROR : mysensors: DEBUG:mysensors:Receiving 0;255;3;0;2;2.2.0
2019-06-19 15:34:09.359 ERROR : mysensors: DEBUG:mysensors:Receiving 2;0;1;0;0;23.2
2019-06-19 15:34:09.362 ERROR : mysensors: WARNING:mysensors:Node 2 is unknown
2019-06-19 15:34:09.365 ERROR : mysensors: INFO:mysensors:Requesting new presentation for node 2
2019-06-19 15:34:09.366 ERROR : mysensors: DEBUG:mysensors:Sending 2;255;3;0;19;
2019-06-19 15:34:10.903 ERROR : mysensors: DEBUG:mysensors:Sending 0;255;3;0;2;
2019-06-19 15:34:10.980 ERROR : mysensors: DEBUG:mysensors:Receiving 0;255;3;0;2;2.2.0
2019-06-19 15:34:21.005 ERROR : mysensors: DEBUG:mysensors:Sending 0;255;3;0;2;
2019-06-19 15:34:21.112 ERROR : mysensors: DEBUG:mysensors:Receiving 0;255;3;0;2;2.2.0
2019-06-19 15:34:29.646 INFO : Pairing timeout
2019-06-19 15:34:29.651 ERROR : Thing was not added
2019-06-19 15:34:29.653 ERROR : addNewThing cancelled

rmalbers

FWIW: I was just thinking I should mention, after the 'Update' of the addon the first time I tired it I got the same/similar messages as I had in my OP. I rebooted at that point and those messages went away so I'm thinking that after addon install or update a reboot is probably a good idea.

rmalbers

I noticed something with the Ethernet GW: On the addon configure page under the 'IP address' the text has an IP address example that includes the port. When I include the port I get this data in the log file: 2019-06-19 17:47:34.750 ERROR : mysensors: INFO:mysensors.gateway_tcp:Trying to connect to ('192.168.1.98:5003', 5003)
2019-06-19 17:47:34.758 ERROR : mysensors: ERROR:mysensors.gateway_tcp:Failed to connect to socket at ('192.168.1.98:5003', 5003)

and it doesn't seem to connect.
When I leave off the port number I get this in the log file:
2019-06-19 17:49:44.095 ERROR : mysensors: INFO:mysensors.gateway_tcp:Trying to connect to ('192.168.1.98', 5003)
2019-06-19 17:49:44.111 ERROR : mysensors: INFO:mysensors:Connected to <_SelectorSocketTransport fd=14 read=idle write=<idle, bufsize=0>>

and it seems to connect. If I'm interpreting this correctly you have to leave the port number off the IP string for it to work properly, not sure which way it really should be, just pointing this out in case it hadn't been noticed.

alowhum

Thank you for the logs.

Perhaps this is a point where it crashes and reboots the add-on:

2019-06-19 15:33:27.935 ERROR : mysensors: DEBUG:mysensors:Sending 1;255;3;0;19;
2019-06-19 15:33:29.635 INFO : About to call startPairing on MySensorsAdapter

I also noticed that it keeps re-requesting the presentation of your nodes:

2019-06-19 15:33:34.617 ERROR : mysensors: INFO:mysensors:Requesting new presentation for node 2

This could mean:

You have created a node that somehow has the same node-ID as an earlier but different node. Although in theory the add-on handles this case now.
Some data is missing that is required to actually create the device. This could be the name.

Can you share the Arduino Code? Could it be that your presentation function does not send names? Your presentation should look something like this:

sendSketchInfo(F("Sensor name"), F("1.1"));
present(CHILD_ID, S_BINARY, F("Button name"));

Here's another example sketch:
https://forum.mysensors.org/topic/10342/mozilla-webthings-mysensors-add-on-adapter/14

// By the way, I don't think using v2.2 should matter. Internally the PyMySensors library actually uses 2.2.

rmalbers

I need to keep better track of what software is running on a given cpu/sensor, sorry about that but I didn't think of it at the time! I 'think' this is what is running on the office sensor. I should probably build a new sensor using this code and make sure it works the same, (log files match what I've been giving you):
// Enable debug prints to serial monitor
//#define MY_DEBUG

// Enable and select radio type attached
#include <MyConfig.h>

#define MY_RADIO_NRF24

//#define MY_RADIO_RFM69

#include <SPI.h>
#include <MySensors.h>
#include <DallasTemperature.h>
#include <OneWire.h>

#define COMPARE_TEMP 1 // Send temperature only if changed? 1 = Yes 0 = No

#define ONE_WIRE_BUS 3 // Pin where dallase sensor is connected
#define MAX_ATTACHED_DS18B20 16
unsigned long SLEEP_TIME = 30000; // Sleep time between reads (in milliseconds)
OneWire oneWire(ONE_WIRE_BUS); // Setup a oneWire instance to communicate with any OneWire devices (not just Maxim/Dallas temperature ICs)
DallasTemperature sensors(&oneWire); // Pass the oneWire reference to Dallas Temperature.
float lastTemperature[MAX_ATTACHED_DS18B20];
int numSensors = 0;
bool receivedConfig = false;
bool metric = true;
// Initialize temperature message
MyMessage msg(0, V_TEMP);

void before()
{
// Startup up the OneWire library
sensors.begin();
}

void setup()
{
// requestTemperatures() will not block current thread
sensors.setWaitForConversion(false);
}

void presentation() {
// Send the sketch version information to the gateway and Controller
sendSketchInfo("Temperature Sensor", "1.1");

// Fetch the number of attached temperature sensors  
numSensors = sensors.getDeviceCount();

// Present all sensors to controller
for (int i = 0; i<numSensors && i<MAX_ATTACHED_DS18B20; i++) {
	present(i, S_TEMP);
}

}

void loop()
{
// Fetch temperatures from Dallas sensors
sensors.requestTemperatures();

// query conversion time and sleep until conversion completed
int16_t conversionTime = sensors.millisToWaitForConversion(sensors.getResolution());
// sleep() call can be replaced by wait() call if node need to process incoming messages (or if node is repeater)
sleep(conversionTime);

// Read temperatures and send them to controller 
for (int i = 0; i<numSensors && i<MAX_ATTACHED_DS18B20; i++) {

	// Fetch and round temperature to one decimal
	float temperature = static_cast<float>(static_cast<int>((getControllerConfig().isMetric ? sensors.getTempCByIndex(i) : sensors.getTempFByIndex(i)) * 10.)) / 10.;

	// Only send data if temperature has changed and no error

#if COMPARE_TEMP == 1
if (lastTemperature[i] != temperature && temperature != -127.00 && temperature != 85.00) {
#else
if (temperature != -127.00 && temperature != 85.00) {
#endif

		// Send in the new temperature
		send(msg.setSensor(i).set(temperature, 1));
		// Save new temperatures for next compare
		lastTemperature[i] = temperature;
	}
	}
sleep(SLEEP_TIME);
}

alowhum

When you create a new post, click on the 'code' button when you want to insert code.

Back on topic, I don't think that's the actual code, since the log showed that no sketch name was sent. Receiving the sketch name is what triggers the creation of the thing, so it's kind of important.

For example, here's a node presentation from my own logs:

2019-06-20 10:20:03.441 ERROR  : mysensors: DEBUG:mysensors:Receiving 33;255;3;0;11;Fireplace
2019-06-20 10:20:03.446 ERROR  : mysensors: DEBUG:mysensors:Receiving 33;255;3;0;12;1.2

...Yet I don't see "Temperature Sensor" anywhere in your logs.

Also in your errors I can see this strange situation where the add-on keeps requesting node 0 to present itself.

2019-06-19 15:34:00.801 ERROR : mysensors: DEBUG:mysensors:Sending 0;255;3;0;2;
2019-06-19 15:34:00.875 ERROR : mysensors: DEBUG:mysensors:Receiving 0;255;3;0;2;2.2.0

Node 0 should theoretically be the MySensors gateway. It only responds with it's version, and not a name. Perhaps that's also an issue (although theoretically it shouldn't be, since node 0 is never turned into a thing).

Come to think of it, I don't see any messages being handled by the add-on other than the initial two messages from the Gateway.

You should see all kinds of things with >> like:

2019-06-20 10:30:34.498 INFO   : mysensors: << Requesting presentation from 35
2019-06-20 10:30:34.500 INFO   : mysensors: >> incoming message > presentation > id: 35; child: 255; subtype: 17; payload: 2.3.1
2019-06-20 10:30:34.501 INFO   : mysensors: >> incoming message > internal > id: 35; child: 255; subtype: 11; payload: Anemone
2019-06-20 10:30:34.503 INFO   : mysensors: >> incoming message > internal > id: 35; child: 255; subtype: 12; payload: 1.0
2019-06-20 10:30:34.505 INFO   : mysensors: >> incoming message > set > id: 35; child: 1; subtype: 2; payload: 1

rmalbers

Ok, I'll look at this and probably just start rebuilding my sensor's and gateway making sure I get those names in there. When I built this over a year ago I was playing with different boards and different sensors and once I got things working in Domoticz I moved on. That's why I have so much program source code out there. This time I'll put a unique, my version number in there so I can keep better track. I think I'll even physically label the sensors this time! Thanks for all your time on this, I really like the looks of the WebThings GW and it's really a good fit for the stuff I'm doing/trying to do. Oh, thanks for that about the code button, didn't know about that.

alowhum

@rmalbers Wait one second. I've been playing with the code and I have a suspicion about what's going on. I'm uploading a new version of the add-on, try that first.

rmalbers

@alowhum Wow, ok, that's sounds great. I'll watch here for word.

alowhum

Version 0.1.0 has been accepted. Try it and let me know how it goes?

rmalbers

Ok ,updated addon and rebooted GW. This test was in Ethernet mode. No luck but the log sure looks different:
2019-06-21 14:32:15.079 INFO : Opened a new things socket
2019-06-21 14:32:15.185 INFO : About to call startPairing on MySensorsAdapter
2019-06-21 14:32:15.197 INFO : mysensors: -Connection status is present in the config data.
2019-06-21 14:32:15.199 INFO : mysensors: RECREATING DEVICES FROM PERSISTENCE
2019-06-21 14:32:15.200 INFO : mysensors:
2019-06-21 14:32:15.202 INFO : mysensors: #0
2019-06-21 14:32:15.203 INFO : mysensors: -Gateway choice: Ethernet gateway
2019-06-21 14:32:15.205 INFO : mysensors: Selected IP address and port: 192.168.1.98
2019-06-21 14:32:15.206 INFO : mysensors:
2019-06-21 14:32:15.208 INFO : mysensors: >> incoming message > internal > id: 0; child: 255; subtype: 14; payload: Gateway startup complete.
2019-06-21 14:32:15.209 INFO : mysensors: Re-requesting presentation of all nodes on the network
2019-06-21 14:32:15.211 INFO : mysensors: Re-request thread created
2019-06-21 14:32:15.218 INFO : mysensors:
2019-06-21 14:32:15.220 INFO : mysensors: >> incoming message > presentation > id: 0; child: 255; subtype: 18; payload: 2.2.0
2019-06-21 14:32:15.221 INFO : mysensors:
2019-06-21 14:32:15.222 INFO : mysensors: >> incoming message > internal > id: 0; child: 255; subtype: 14; payload: Gateway startup complete.
2019-06-21 14:32:15.224 INFO : mysensors:
2019-06-21 14:32:15.225 INFO : mysensors: >> incoming message > presentation > id: 0; child: 255; subtype: 18; payload: 2.2.0
2019-06-21 14:32:15.227 INFO : mysensors: AddonMangerProxy: recv: b'{"messageType":"startPairing","data":{"timeout":60,"adapterId":"mysensors-adapter","pluginId":"mysensors-adapter"}}'
2019-06-21 14:32:15.465 ERROR : mysensors: Node 2 is unknown
2019-06-21 14:32:36.851 ERROR : mysensors: Node 1 is unknown
2019-06-21 14:32:50.254 ERROR : mysensors: Node 2 is unknown
2019-06-21 14:33:09.496 ERROR : mysensors: Node 1 is unknown
2019-06-21 14:33:15.190 INFO : Pairing timeout
2019-06-21 14:33:15.196 ERROR : Thing was not added
2019-06-21 14:33:15.198 ERROR : addNewThing cancelled
2019-06-21 14:33:15.201 INFO : mysensors: PAIRING INITIATED
2019-06-21 14:33:15.204 INFO : mysensors: Thread already existed
2019-06-21 14:33:15.205 INFO : mysensors: Re-requesting presentation of all nodes on the network
2019-06-21 14:33:15.207 INFO : mysensors: Re-request of node presentation restarted
2019-06-21 14:33:15.208 INFO : mysensors: AddonMangerProxy: recv: b'{"messageType":"cancelPairing","data":{"adapterId":"mysensors-adapter","pluginId":"mysensors-adapter"}}'
2019-06-21 14:33:25.022 ERROR : mysensors: Node 2 is unknown

MySensors: trying to add Things in WebThings Gateway

12

11.7k

11.2k

113.1k