2.0 Discussion: Units, sensor types and protocol
-
@daulagari said:
My proposal is basically to merge the ChildSensorId and ChildSensorType into one, being the ChildSensorType, so:
OK. That touches on one of my critiques of the direction it's going. (Maybe "critique" would sound stronger than I intend, I'm not disparaging the current direction, just suggesting another one).
I think that ChildSensorID is an appropriate concept for identification and addressing (ie: included in packets), while "ChildSensorType" is static metadata which should be collected during configuration but does not need to be passed in every packet. So I would simplify in the other direction - get rid of ChildSensorType in the packet addressing heirarchy, rather than getting rid of ChildSensorID.
This idea is not currently popular, in large part I think because the metadata describing a given ChildSensor would have to be collected/stored by each Controller plugin, as the currently favored architecture is to have a very thin gateway node which just passes packets between controller plugins and the OTA packets. My alternative vision would be to have a thicker gateway which could, for example, store the metadata for each ChildSensor (eg: ChildSensorType, but expandable) gathered from nodes during config, and provide it to the controller plugins.
With this approach there is no barrier to having multiple children of the same type (but with different ChildIDs) because the concept of addressing & identity have not been conflated with the ideas of types and metadata. In my preferred alternatives, the normal OTA packet for routine reports and commands omits the semantics and includes only addressing/identity, while the initial configuration passes semantics, types and names that are relatively static and do not change per-packet. Moving ChildSensorType from per-packet addressing to static metadata would be one of the implications of this design principle. (If you apply this to values, then we also include a numeric ValueIndex in the packet and move the ValueType info to the metadata.)
(I went even further and suggested omitting the ChildID from the regular OTA packet by having only NodeID | ValueIndex for addressing, and treating ChildID as static metadata which can be associated with each ValueIndex. Thus for example we could say that ValueIndex 3 contains a temperature value associated with child 2, and ValueIndex 4 contains a humidty value also associated with child 2. The central system (thick gateway or controller plugin) would use saved metadata to know which values come from or go to a given child ID container, but the OTA packet doesn't have to include that every time. Not surprisingly, this was a bit too radical a revamping and has not gained traction. And that's OK too, I'm only one voice with only part of the picture here).
Just to be complete, EncodingFormat (eg: 1, 2, or 4 byte integer or unsigned interger, or 4 byte float, or float encoded a text string etc) could be either config time metadata if it's always the same, or could be included in the OTA packet if it changes dynamically. I can see a case for either design.
I'm mentioning this because you seem interested in looking into deeper changes to the architecture, but I'm not pushing these concepts here now - my ideas on such things have been heard and a few have influenced the direction hek is going, but the overall design concept has not been persuasive enough. That's fair.
The system here is pretty good already, so it's not critical to make such changes. So rather the push MySensor too much, I've been (intermittantly) working on my own protocol ideas (at the spec level) which incorporates these and other concepts, and which could co-exist with MySensors. However my tentative design has additional design goals including high bandwidth distribution of pixel control data and medium high bandwidth collection of rapidly changing "sensor" data (eg: joysticks used for low latency pattern. It would be able to handle the MySensors use case of low bandwidth sensor networking as a less challenging subset. However my design is an off-and-on project on paper so far, while MySensors is a working system, so I give a lot of weight to the latter.
@Zeph said:
I think that ChildSensorID is an appropriate concept for identification and addressing (ie: included in packets), while "ChildSensorType" is static metadata which should be collected during configuration but does not need to be passed in every packet. So I would simplify in the other direction - get rid of ChildSensorType in the packet addressing heirarchy, rather than getting rid of ChildSensorID.
100% agree. Sending ChildSensorType is just redundant static metadata. Just having implement a MySensor-conroller for fhem I noticed there's no way to request Presentation-messages from Sensors so a Controller after Sensor startup, so the controller has no way to verify it's persistent 'static' metadata without a user going to restart the sensors. This actually is a point that needs to improove.
But the argument a controller would have to be 'thin' (in terms of not being statefull in respect to sensor details) is a no-arg. You cannot do automation without knowing details about the controlled items.- Norbert
-
Yep ChildSensorType is redundant.
If thin things like the Arduino based MQTT gateway would manage without it we could probably remove it from the normal payload.Request for metadata is a good idea. Would require sensor to keep this available at all time in memory or eeprom when controller requests it.
-
Yep ChildSensorType is redundant.
If thin things like the Arduino based MQTT gateway would manage without it we could probably remove it from the normal payload.Request for metadata is a good idea. Would require sensor to keep this available at all time in memory or eeprom when controller requests it.
@hek said:
Request for metadata is a good idea. Would require sensor to keep this available at all time in memory.
This is allready in memory, just requires a move of send-presentation-messages code into own method that is called from begin and in response to 'request-metadata'.
-
Yep ChildSensorType is redundant.
If thin things like the Arduino based MQTT gateway would manage without it we could probably remove it from the normal payload.Request for metadata is a good idea. Would require sensor to keep this available at all time in memory or eeprom when controller requests it.
@hek said:
If thin things like the Arduino based MQTT gateway would manage without it we could probably remove it from the normal payload.
As MQTT-protocoll is not stateless (it does Quality-of-service with message-storage and redelivery - though the existing gateway code doesn't support this yet) the MQTT-gateway cannot be a thin thing anyway. I wouldn't mind if it doesn't run on Uno due to memory-constraints, there's the Mega2560 or even the DUE which seems to be affordable as you only need a single gateway per install.
-
what about a "RS232" sensor?
The TV in my living room has a serial port to control it (Can do everything I can with the remote) and also request status from the TV (eg. powerstate, channel being watched etc). Could be cool to include that into some kind of domotica solution
/ Thomas
-
Perhaps rs232 and ir can be fused together as one type? Both are by definition serial protocols. Each with a protocol and symbol rate specific to the appliance at hand.
-
Think "uart" :)
-
Think that, for UART it should be enough to send characters (or strings) to the sensor, and send back replies to the master (If anything comes in at the serial port)
I asume that if one needs to use serial to communicate with some remote device, one will setup baudrate etc. for that particular device in the arduino sketch. So no need to use initialisation commands for the serial stuff.
For IR it's a bit more complicated, because there is no single protocol definition like serial. for IR it could be RC5, RC6, something else, it can be different carrier frequences etc. for each brand. And it can be send to multiple devices, using the same IR "blaster"
/ Thomas
-
Think that, for UART it should be enough to send characters (or strings) to the sensor, and send back replies to the master (If anything comes in at the serial port)
I asume that if one needs to use serial to communicate with some remote device, one will setup baudrate etc. for that particular device in the arduino sketch. So no need to use initialisation commands for the serial stuff.
For IR it's a bit more complicated, because there is no single protocol definition like serial. for IR it could be RC5, RC6, something else, it can be different carrier frequences etc. for each brand. And it can be send to multiple devices, using the same IR "blaster"
/ Thomas
@Thomas-Bowman-Mørch well, yes. But I am more thinking a "protocol". No matter what serial interface the sensor has externally, any serial interface implement some form of protocol for managing handshaking, flow control, etc. And considering the limitation in the data packets in RF24, we cannot expect to be able to implement a "stream".
By having a protocol, the sensor node can ensure it collects enough data to be able to satisfy it's external device, i.e. caching data packets for making a larger continuous serial transmission. Or reading a larger continuous transmission and caching it from the device, before starting to transmitting it to the controller. If transmission starts too early, data sent from the device, could be lost (depending on HW/SW solution) as serial protocols do not have to implement flow control (and rarely does).
Both IR and UART share the same caveats so in that respect, I think it makes good sense to figure out a command type that can satisfy both types as the needs are similar. And then it potentially can also be applied to other serial protocols (CAN for instance). -
Hi,
Not very usefull post for you here, but I want to say I'm very pleased to see this discussion and the news I can read here or what appears in dev on the github.
About the presentation of sub-type accepted. After getting an almost complete controller for Jeedom, we have difficulties to "predict" things to automate and make it very simple for user to use the plugin. Like for exemple, create commands. For relays and dimmer it was easy, but after when we look for servo or some extra type it's impossible to know for now what we can create.
So reading, the sensor will tell it in presentation it's just perfect.Same for the presentation during node of lib used, that's just better than during the sensors, make sense.
I read also on the protocol page, that there is a way for a node to request data from gateway/controller. Can this point be clarified, maybe it needs also the sensor/node to say what it can request ? That's also for automation of autolink. For us (Jeedom plugin) and I think for other, if we know the sensor is able to request some data, we can automaticly create the information item and the user will have only to map it to the equipment it wants to point. I don't know if this is still included/planned as I'm aware many of the "smart" part must be on the controller side, but with beautiful things like Scene Controller and LCD display, I can imagine there is a need ?
I read also the OTA process will change a lot. So, is it better waiting 2.0 to include it on our side ? I mean, for the controller part, will this change a lot of things ?
And last point, there is actually a internal message type Reboot. This one is indicated only working for OTA bootloader. Is that true ?
If so, is it possible in 2.0 to include a way to request a complete presentation of an already started sensor. (that's why I ask for the Reboot message, simpler to request a reboot so the node will present itself at the same type)
Sometimes it's easier when migrating for exemple or reinstall, and we know the ID of sensor to ask it for presentating itself than going there to unplug it.That's my thoughs after getting a little more in deep of protocol for the controler/plugin :)
Cédric
-
Hi, I think we could integrate an uuid system in protocol, like this one https://code.google.com/p/tinkerit/wiki/TrueRandom .
-
Hi,
Not very usefull post for you here, but I want to say I'm very pleased to see this discussion and the news I can read here or what appears in dev on the github.
About the presentation of sub-type accepted. After getting an almost complete controller for Jeedom, we have difficulties to "predict" things to automate and make it very simple for user to use the plugin. Like for exemple, create commands. For relays and dimmer it was easy, but after when we look for servo or some extra type it's impossible to know for now what we can create.
So reading, the sensor will tell it in presentation it's just perfect.Same for the presentation during node of lib used, that's just better than during the sensors, make sense.
I read also on the protocol page, that there is a way for a node to request data from gateway/controller. Can this point be clarified, maybe it needs also the sensor/node to say what it can request ? That's also for automation of autolink. For us (Jeedom plugin) and I think for other, if we know the sensor is able to request some data, we can automaticly create the information item and the user will have only to map it to the equipment it wants to point. I don't know if this is still included/planned as I'm aware many of the "smart" part must be on the controller side, but with beautiful things like Scene Controller and LCD display, I can imagine there is a need ?
I read also the OTA process will change a lot. So, is it better waiting 2.0 to include it on our side ? I mean, for the controller part, will this change a lot of things ?
And last point, there is actually a internal message type Reboot. This one is indicated only working for OTA bootloader. Is that true ?
If so, is it possible in 2.0 to include a way to request a complete presentation of an already started sensor. (that's why I ask for the Reboot message, simpler to request a reboot so the node will present itself at the same type)
Sometimes it's easier when migrating for exemple or reinstall, and we know the ID of sensor to ask it for presentating itself than going there to unplug it.That's my thoughs after getting a little more in deep of protocol for the controler/plugin :)
Cédric
@lunarok said:
If so, is it possible in 2.0 to include a way to request a complete presentation of an already started sensor. (that's why I ask for the Reboot message, simpler to request a reboot so the node will present itself at the same type)
Sometimes it's easier when migrating for exemple or reinstall, and we know the ID of sensor to ask it for presentating itself than going there to unplug it.True, reboot would send presentation again. But another option is to save the presentations in the api. Still a bit unclear which option I will choose.
@aliasdoc
What do you mean the random number should be used for? -
Hi. I would like my nodes measuring various values (temperature, humidity, ...) to send those values at specific times. I don't want to include the RTC on the node, so would like the node to request a sleep period from the controller. I can see there is no such possibility with the version 1.4 of the protocol. What do you think about including such thing?
-
Right now you can use the Time.h library and you don't need an RTC.
Just sync the time with MySensors call...
gw.requestTime(receiveTime);with the function:
void receiveTime(unsigned long controllerTime) { Serial.print("Time value received: "); Serial.println(controllerTime); RTC.set(controllerTime); } -
You can use VAR1-5 for this in 1.4.
2.0 will have both VAR (used mostly for pushing custom data) and CONFIG (for custom node configuration).
@hek said:
You can use VAR1-5 for this in 1.4.
2.0 will have both VAR (used mostly for pushing custom data) and CONFIG (for custom node configuration).
Yeah, good point. I somehow missed these general purpose variables. It might be good to add a dedicated type for it if you'll find it useful. If not I can live with the VAR for sure ;). Thanks.
