Generalizing MySensors

Zeph

I think this is discussion is converging, as I've said. Good points made and taken all around.

There seems to be agreement that the sensactuator nodes should be controller agnostic.

The largest area where they are not is in choosing a "sensor type" which is Vera specific. That's what started this rethinking for me - in trying to annotate the MySensor libary's variable and sensor type enums (in the lost forum posts), I wasn't sure how to describe what a "sensor type" really represented. I tried to find some independent meaning to the Sensor type that I could document, but failed. If it says the sensor type is heater, what does that imply? Low/Medium/High or a thermostat or what? It is controlling a heater or just monitoring one, or both? The conclusion was that type means nothing more or less than "you may be able to fit this 'sensor' into what Vera calls a heater", meaning there may be a user interface object in Vera into whose displays and user controls you can reasonably well map the variables and behavior of this sensor, and the icons displayed for that object in Vera may more or less fit the functionality of this sensor. There was no Vera-independent semantics to the sensor-type.

However, hek suggests that this is just a hint which can be ignored, so one could say that it's at worst a small and meaningless overhead for some controllers, and at best makes Vera easier to interface with. Not a show stopper to keep it around for now, since it can be ignored. It might be more clear to call this data "Vera-hint" than "sensor type", tho. Other controllers could try to discern something from the Vera-hint, or ignore it.

That said, moving the metric/imperial conversion to a centalized location seems like a good move, and hek's suggestion of letting senseactuator nodes specify what variables a given child device ('sensor') implements, seems like an important move in the right direction. I think that any controller could benefit from getting a list of what variables a "sensor" (an addressable node+child-id grouping of variables) generates and accepts, even Vera. So this goes a long way towards addressing my concerns, whether or not the sensactuator nodes continue to present a vera-hint as well.

John feels strongly that all the adapter functionality should be implemented entirely within each home automation controller. I've never argued against that option, only suggested keeping open some other possibilities as well. However, if no need to implement some or all of the adaptation outside the controller ever comes up, that would of course be fine with me.

hek makes the point that the gateway (wireless hub) does not have room or performance to implement very much. Given that I've never suggested that the gateway be forced to implement any adapter functionality, we are not exactly disagreeing (our views are not incompatible). However that brings up a more general point, in that I would not design the architecture at any deep level to assume that the gateway will will continue to be limited to an ATMega328p @16MHz and 2K of RAM.

I DO see reason to design for the Arduino Pro Micro for the sensactuator nodes: There can be many of them, and the APM is very cheap, it's small, and it can be very low power to accommodate battery power (and it's Arduino compatible so it can inherit a rich set of device libraries). And it's adequate for most sensactuator tasks; in most cases more power would be wasted anyway. I don't see much on the near term horizon that fill the sensactuator node niche much better than the APM class device for MySensors. (Only possible exception: if the nRF51822 became more widely used in the form factors and price point the APM+nrf24L01+ occupy).

But the gateway needs to be on full time and wired to the controller, so it can be wall powered and even a bit physically larger if need be, and since there is only one of them it could be a bit more expensive. Arduino Mega clones are $15 - no faster but 4x the RAM and more peripherals. A teensy 3.1 has 64K of RAM and runs 32 bit ARM code using the Arduino IDE and library interfaces for $17. Arduino Due clones with an ARM and 96 KB can be had for $20. ARM based STM32F103 with 20K RAM, 72 MHz processor and lots of IO are available on eBay for under $7. Or you can get a ST32F411 Nucleo for about $10 with floating point ARM (they can use the Arduino-like mBed system). The RPi is $35, the BBB is $55 now, and there are several other GHz class ARM devices under $80 like the cubieboards (plus shipping; the other prices include shipping BTW). Given all that, I'm going to be very surprised if two years from now most of the MySensors wireless gateways/hubs are still running ATMega328p processors.

So while I'm not proposing now to force all gateways to be larger than an ATMega328p, I'm also not assuming in designing an architecture that the gateway hardware can never do more than that chip could do - at least in some configurations. If we had to spend an extra $5-15 dollars for a much more capable (single) gateway in order to interface with a given controller we wanted to run, that would not be a limiting factor for most of us.

AND - John may be right - maybe there will never come a case where it makes any sense to implement the adaptation functions anywhere except on the controller (eg: inside a Vera or a PC running home automation software, etc). Time will tell. If I differ some in the degree of flexibility I suggest, I am nevertheless respecting the other views which have been well stated here, and have no certainty that such flexibility will be needed. We seem to be agreeing on the fundamentals.

Rasmus Eneman

I have thought a bit about this, not from a MySensors perspective but as a Controller perspective as I'm developing one.
For me SensorType can be interesting (however it will need to be remodeled a bit). If I get a temperature from the sensor,
it's probably a Thermometer, but it may be the inner core temperature or something else. If I get a unit that supports turning
on or off it may be a Lamp, but it may also be a bunch of other things. To create the best experience for the user I would
need to know not just that it's a temperature reading, but from where.

So for this to work S_TEMP would be S_THERMOMETER. Then I know that a V_TEMP is a environment temperature
reading and not a inner core temperature. Obviously all of the SensorTypes would have to be looked over if they need
similar changes.

So, I would prefer if SensorType described what a human would classify a sensor node as.

I would also prefer if all units reported were in SI units. If a user want to display the as something else, that is a display
setting and nothing else.

hek

@Rasmus-Eneman said:

S_TEMP would be S_THERMOMETER

:thumbsup:
Sounds very reasonable.

Damme

I just wanted to make a pointer to http://forum.mysensors.org/topic/260/mysensors-mqtt-gateway , Might be a combined discussion about the future of the protocol, I am thinking about the address-layout. Using Vera-style or just ignore some parts of the address.

I have not been reading everything in this discussion, I haven't had time yet. But if you come to any conclusion it would be goot to write them down! :)

John

@Zeph

The largest area where they are not is in choosing a "sensor type" which is Vera specific.......There was no Vera-independent semantics to the sensor-type.......Other controllers could try to discern something from the Vera-hint, or ignore it.

I totally agree with you about the sensor type limitations. It would certainly be nice if the Vera would provide an interface where sensor types can be extended. But i'm not well known enough with the Vera to know if this is the case or not. All though, as you mention, that the sensor type is just a hint which could be ignored, it does give some extra information about the sensor produced variable type values. And where the total sensor types table is concerned it mixes up sensor type semantics. As i read it now in your post it then appears this mix up is because of the Vera and I better understand the protocol part of the discussion. Thanks for this clarification (I shouldn't be doing multiple things at the same time).

As example and thoughts spin off.
In your example you are referring to heater, where as i understand an heater could be a collection of possible sensors and actuators, like temperature and switches. In this example it would be best to have an N_HEATER (heater node) sensor node with a S_TEMP sensor and for example a S_SWITCH actuator. from which produces V_TEMP and V_ON/V_OFF. But even then taken this as an example you still do not know if it is only monitoring, etc... which is only something you can assume depending on the reported sensor types. And it would even be better if the user can define the node him/her self of course.

John feels strongly that all the adapter functionality should be implemented entirely within each home automation controller. I've never argued against that option, only suggested keeping open some other possibilities as well. However, if no need to implement some or all of the adaptation outside the controller ever comes up, that would of course be fine with me.

Yes i do, and certainly with the open sourced, or plugin capable automation systems. But, i am open for possibilities where there are protocol based implementations possible on the hardware instead of controller protocol type. And yes this would acquire more capable hub hardware. It would be nice if there where more protocol based solutions like offering an Vera hub, MQTT hub, JSON Hub.

Zeph

@Rasmus-Eneman said:

I have thought a bit about this, not from a MySensors perspective but as a Controller perspective as I'm developing one.
For me SensorType can be interesting (however it will need to be remodeled a bit). If I get a temperature from the sensor,
it's probably a Thermometer, but it may be the inner core temperature or something else. If I get a unit that supports turning
on or off it may be a Lamp, but it may also be a bunch of other things. To create the best experience for the user I would
need to know not just that it's a temperature reading, but from where.

No disagreement there, the question is where and how to specify that.

So for this to work S_TEMP would be S_THERMOMETER. Then I know that a V_TEMP is a environment temperature
reading and not a inner core temperature. Obviously all of the SensorTypes would have to be looked over if they need
similar changes.

Looking through the sensor types with a perspective like that is what started me on this thread (initially in the messages that got lost).

As you say, a measurement of "temperature" could mean many things. It could be a simple themometer, or it could be a dewpoint, or it could be a daily max from a weather station or it could be the intake or outlet temp from a heater or cooler device, or it could be a themostat setting - all in degrees C.

One approach would be to define a new "sensor type" (as you use it, I'm going to use a different terminology for this concept below) for every type of temperature variable; basically invert the current heirarchy and define many sensors types for every variable type. Put another way, the variable could be the addressable unit, and one of its attributes could be a sensor type from the list defined for that type of variable. So we find that there is a V_TEMPERATURE variable, and we ask it "what kind of V_TEMPERATURE are you?" and it responds that its a "S_TEMP_HIGHLIMIT" or a "S_TEMP_FREEAIR" or some other S_TEMP_* type of temperature.

HOWEVER, hek has made the point that there is utility to defining the a container for multiple related variables (what's called a sensor in the current system, and addressed with a node id and child id). The current concept of the "Sensor Type" is associated with that group or container. So you might have a S_WEATHERSTATION which (potentially) contains V_TEMPERATURE and V_RELHUMIDITY and V_WINDSPEED etc variables. (Multiple variable types for a given sensor type, in this way of looking at it).

The problem with the container is that real world sensors vary a lot - one weather station will include a UV sensor and another won't, or one will measure peak wind speeds and another will not, or one includes both inside and outside temps, etc. So it's hard to pre-specify what all the variables a S_WEATHERSTATION may have, and if you do, most actual ones will be pretty sparsely populated. And then we look at a home automation controller - which of those measurements will their nearest analog to the S_WEATHERSTATION implement? Probably not exactly the same as all other controllers.

I think the closest we've come to sorting through this inherent complexity is something like this:

There are Containers which are individually addressable and may report more than one measurement (or accept more than one setting). All of the measurements within a given container are physically or logically related to each other. (This is the thing addressable with a node address and child id).
Each measurement (or setting) associated with a container has a V_* variable type which says what is measured, and if possible specifies the standard units or enumerated states.
Perhaps each measurement can also have a "semantic" attribute, chosen from a list specific to the variable type (eg: V_TEMPERATURE can have VS_TEMP_FREEAIR or VS_TEMP_DEWPOIT etc). (VS=Variable Subtype or Variable Semantics). This is where I'd put what you were calling a sensor type - as a variable subtype attribute.
The central authority can ask each node about the containers (children) it has. It can ask each child what variables it reports or accepts. It can find the V_* variable type and the VS_* subtypes for each variable.
The node can also report a hint from the S_ list for each container (child). This hint does not limit what variables the container may report or accept, and it may or may not match well with the way any given controller lumps and splits things - but the hint can be ignored.
There has to be some configuration for each controller, which maps variables do defined by the nodes, into variable and objects which make sense for the given controller. (I was calling this the adapter function). This mapping is not required to make a one to one association between addressable variable containers ("children" or "sensors") and the objects in that controller's universe.

An example. Suppose a node reports outdoor temp & humidity and wind speed and direction and UV index, all associated with a single measurement location and reported as variables in the same child container. And there's a controller which doesn't have a "weather station" object to group all of those variables. But it does have an "environment monitor" object which combines temp, humidity and noise level. The adapter function could map the node's temp and humidity to the temp and humidity of the controller "environment monitor" object (leaving noise level unreported), and find some other controller object(s) for wind speed and UV index - or ignore them. (There could be another parallel "controller" like a cloud recording system which does make use of them even if the given home automation controller does not).

So, I would prefer if SensorType described what a human would classify a sensor node as.

If we associate SensorType with a multi-variable container, then there are many mix-and-match varieties of these, as perceived by a human, not all of which are going to be implemented in any given controller. If we mean "variable subtype" as described above, then no problem.

I would also prefer if all units reported were in SI units. If a user want to display the as something else, that is a display setting and nothing else.

I think we are coming to some agreement about that part (single set of standard units for reporting, rather than pushing the user display preferences out to each node). But even that's not quite as simple as saying "SI" and walking away. I've seen wind speed reported in m/s and kph for example. And pressure has multiple possible units. As do light or sound. Even temp has C and K. I've seen fluid flow in cubic meters/hour or liters/min. So I think we should explicitly specify what units each measurement should use (if it can) and that we should look primarily to SI when possible in defining those expectations.

And we need to accommodate uncalibrated reports - like brightness reported based on an uncalibrated and non-linear sensor which nevertheless has higher (or lower) values for brighter light. We may not be able to translate to lux, but the HA controller could nevertheless use empirically set thresholds for triggers or conditions.

Zeph

@John said:

In your example you are referring to heater, where as i understand an heater could be a collection of possible sensors and actuators, like temperature and switches. In this example it would be best to have an N_HEATER (heater node) sensor node with a S_TEMP sensor and for example a S_SWITCH actuator. from which produces V_TEMP and V_ON/V_OFF. But even then taken this as an example you still do not know if it is only monitoring, etc... which is only something you can assume depending on the reported sensor types. And it would even be better if the user can define the node him/her self of course.

I see that the difficulty of organizing this complexity and diversity is becoming clear to us all.

I would disagree about N_HEATER, ie: a semantic type for a node. I see the node as a transport mechanism, which may contain one or more "variable containers" (or "sensors" where a sensor can have multiple measurements which are closely associated). So for example a node type I'm thinking to put in many rooms would have:

DHT-11 or DHT-22 for temp and humidity (former so cheap all would have at least that)
LDR or digital sensor for brightness (likewise)
PIR motion detector
IR remote control decoder
IR LED
visible LED for signalling
possibly a puzzer or small speaker
battery voltage sensor (divider)
I'm considering sonar range finder in some cases
possibly a door switch

This is one node (I hope). And the physical sensors would be lumped and split into child containers for variables (logical "sensors"). It does not matter to the central controller whether temp and humidity came from one physical sensor or two - the reported variables will be aggregated into several logical groupings of related measurements.

I an live with each container (child "sensor") having a "sensor type" as a semantic hint. But I don't see the node itself as having a semantic type - it's function is addressing and transport. I don't see that there's even any "hint" to be associated with the above node which would be of use to the controller.

Going a step further, if the timing gets too messy, or I want motion sensing and IR decoding in different locations in the room, I might split up these devices into two nodes in some rooms, rather than one. As I see it, that should make no difference to the controller software per se. In the "adapter" function, some local configuration (addressing) would change, but the same set of variables would be reported.

(And here I thought this thread was about cooked)

Damme

I'll try to write my opions about this, First of : Do Not over complicate stuff!

Today we have presentation values and sensor_type values, those two tables are not compatable (Dont understand why). I think this should be only ONE list off things, used both for presentation and value type. We just put more stuff in it..

IF someone needs more complicated stuff it is easy to do in the node code; Just send an empty request to the node and you can get whatever you want in return, I use this code here to get latest relay state from the node if the controller forgets what the last one was. (makes more sense also because saving the value at the controller might be far off the value from the node)

void incomingMessage(const MyMessage &msg) {
	if (msg.type==V_LIGHT) {  // We dont care about other stuff than V_LIGHT , this could be anything!
		if (strlen(msg.getString())==0) { // is there any payload?
			// NO = Get the latest state, construct a message and send.
			gw.send(message.setSensor(msg.sensor).setType(V_LIGHT).set(digitalRead(msg.sensor-1+RELAY_1)?RELAY_ON:RELAY_OFF)); 
		} else { 
			// Here we could check is it 0 / 1 ? or something else, could be 'STOP_60' and we could program a timer to shut down the relay in 60 minutes.
			digitalWrite(msg.sensor-1+RELAY_1, msg.getBool()?RELAY_ON:RELAY_OFF);
			gw.saveState(msg.sensor, msg.getBool());
		}
	} 
}

The current form of address in my mqtt broker is 'prefix/nodeid/sendorid/v_type' -> 'MyMQTT/074/002/Light'

In this example client can send 0/1 or nothing to the broker, if nothing it will publish the latest state back to client. but payload can be anything.

marceltrapman

@hek said:

We should move unit information/conversion back to the controller. All sensor data is send as SI-units. This will make the sketches even simpler but it puts some extra workload on the plugin developers. I'll discuss this further with @marceltrapman and @john who is the first to port/create 1.4 plugins for their respective favorite controllers.

So, what you mean is that we send information based on the metric system (if any) and receive information about temperature etc. SI based with the actual conversion solely happening in the controller?

Basically any system is fine with me as long as I know about it :)

John

@hek said:

We should move unit information/conversion back to the controller. All sensor data is send as SI-units. This will make the sketches even simpler but it puts some extra workload on the plugin developers. I'll discuss this further with @marceltrapman and @john who is the first to port/create 1.4 plugins for their respective favorite controllers.

I could not agree more. And, to be honest, it would even require less work then keeping track per sensor node. Having a sensor network wide setting is much more comfortable. And nonetheless this solution means less cycles spend on sensor node side saving power.

@zeph said:

I would disagree about N_HEATER

Me too, it was solely an example which could be used as a group definition to make it clear what this sensor would do in remark of sensor-node typing and should be done controller side. In the following up example you provide you mention a nice multi-purpose sensor node which clearly produces a "User knows best" case which is absolutely true in case of the flexibility provided with MySensors and is the perfect example why there should be done as much as possible done on the controller and let the hardware do nothing more then deliver only the most basic types and their data.

I an live with each container (child "sensor") having a "sensor type" as a semantic hint. But I don't see the node itself as having a semantic type - it's function is addressing and transport. I don't see that there's even any "hint" to be associated with the above node which would be of use to the controller.

I totally agree until the "hint" part. I personally find it very useful if the table was more organized in the type specifics. I'm seeing device types and sensor/actuators in the same table, which agreeable can cause confusion. I personally would leave out the device types which should be user definable.

In my personal opinion the most optimal solution with the tables would be still provide two tables, but one with sensor/actuator types, and one with datatypes and let the user decide in the controller what a sensor-node device represents. A door/window sensor could be replaced with a reed switch/hall effect to detect open/close, which could be integrated in one sensor type defining an open/close (0/1,true/false) state. This would i think be the simplest solution.
And if the controller is smart enough, group specific sensor types together.

@hek said:

char variables[] = {V_FORECAST, V_TEMP, V_PRESURE, V_TEMP}
gw.present(<id>, S_BAROMETER , variables);

I think this could be possible and helps generalizing MySensors by decoupling address and type. But this would require some extra work on the controller side. Is it possible that the position of the sensor presentation in the array can be the sensor's address? It would also require some extra work for the programmer of the sensor node where he/she needs to remember which sensor is at which position.
The controller would see based on the char array's character position that V_FORECAST is address 0, V_TEMP = 1, V_PRESSURE = 2, and V_TEMP = 3. Because the sensor developer knows which V_TEMP sensor is for what. The first is for outside, and the second is for inside, he/she can define this on the controller.
In this way you would decouple the the sensor type and address definition. Or is this making it too complex?

Zeph

We continue to mostly agree.

Warning: this turned out to be very long indeed, and I need to move on - and I didn't want to lose this thinking and describing, so here it is. This should probably be condensed into a document.

@John said:

@hek said:
char variables[] = {V_FORECAST, V_TEMP, V_PRESURE, V_TEMP}
gw.present(<id>, S_BAROMETER , variables);
I think this could be possible and helps generalizing MySensors by decoupling address and type. But this would require some extra work on the controller side. Is it possible that the position of the sensor presentation in the array can be the sensor's address?

We are talking about multiple levels of hierarchy here. In the current system, there is:

node (no semantic typing)
"sensor" or child (with a S_* type)
variable (with a V_* type)

The array hek was using as an example (of possible future approaches) was for variables within a "sensor", not sensors within a node.

I find the above heirarchy still useful, but I think we are asking for trouble when we label the middle level "sensor". Structurally it looks like a group of one or more logically or physically related variables which are reported or assigned (where the reports measure the real world and the assignments may change the real world).

What do we want to call a "sensor" (apart from the current usage)? Is a DHT-11 one sensor or two? Is a light sensor which also can also report internal temperature two sensors? If we go with the "those are multiple sensors" then is there any distinction between a "sensor" and a "variable" or is there a one to one mapping?

For purposes of this discussion, I'm going to go with "SensorGroup" for that middle level of the hierarchy (currently "sensor"). Where I say "SensorGroup" you can read "the addressable container formerly known as a 'sensor' " :-)

The MySensors network can have one or more end nodes
A node can have one or more SensorGroups, each with a child id by which it can be addressed
*. A SensorGroup can have one or more Variables.

The purpose of a SensorGroup is to provide an addressable container for related measurements and controls (represented as variables). So far, not very different from MySensors 1.4 and earlier, except the name.

Where we begin to evolve the design is in hek's suggestion that a SensorGroup could enumerate the variables it contains. Currently, the central control doesn't really know what variable types any given SensorGroup might report. It might take a wild guess based on the presentation type of the SensorGroup (the current S_* enums), but we've had lots of examples of how flawed any such assumptions might be. So heck proposed (very much in line with the rethinking going on here) that a SensorGroup might report what Variables it contains.

@hek said:

char variables[] = {V_FORECAST, V_TEMP, V_PRESURE, V_TEMP}
gw.present(<id>, S_BAROMETER , variables);

This is very interesting because there are two V_TEMP values in the list. The current addressing hierarchy is node/SensorGroup/variable_type (where SensorGroup is represented by a numeric child id). A given SensorGroup should have only one V_TEMP variable, because we otherwise have no way to distinguish "which V_TEMP".

So the above implies another numeric address component - the variable id (or index into that array). You could now identify the first V_TEMP as (node address)/(child id)/1 and the second as (node address)/(child id)/3 (if they are zero based like the array (note 1)).

So: (node address)/(child id)/(variable idx) is the implied new addressing scheme. And passing the array in hek's example does two things: it reports how many variables the given SensorGroup has, and it tells the type of each. So the controller (or some adapter function) knows that variable 2 in that SensorGroup is type V_PRESSURE.

This is very much along the lines that I have been thinking as well (for a different wireless network design).

If we follow this concept, the world actually starts to sort out. The "atoms" in this chemistry are Variables - they can be reported as measurements or set to cause an action or configure a device. These can be combined into logical SensorGroups. And SensorGroups are combined into Nodes for addressing.

Now we can talk about the flavor of those atoms, the Variables. That is, a given Variable can have various attributes. We have one major attribute already - the V_* types. Those give the variable's values some semantic interpretation - like interpreting the numeric value 45 as meaning 45 degrees Celsius. Each V_* type would imply a standard unit to be used (if meaningful).

Another obvious attribute of a given Variable is the format or structure: eg: one/two/four byte signed/unsigned int, or floating point, or string, or boolean. For some purposes like weather in degrees C, a one byte signed int could be sufficient for V_TEMP, but for a reflow oven we might need two bytes or a float, while still being semantically V_TEMP.

I'm going to add another attribute: UNCALIBRATED. This is optional, but if attached to a Variable we know that it is NOT reporting in the standard units. (If it's not attached, then standard units should be used for reporting). For example an uncalibrated LDR might report values which are not in lux and probably very non-linear. However, the value should follow the same slope as the standard units - if a higher number means "brighter" in the standard units, then the uncalibrated values should also rise for brighter. So we can set thresholds for "brighter than" even if we don't have a lux value.

Earlier examples have pointed out that degrees C could be used for a thermostat or a temperature sensor, and for free air temperature or contact-with-device temperature or even dewpoint. Even air temp could be "intake temp" or "exhaust temp". Some of these concepts could be reported as additional attributes associated with a given Variable.

Another attribute would be "name". Gee - if the variable has a meaningful name, at least a human might more easily set up the mappings and configuration files. Knowing that node 7, child 1, variable 2 is a V_TEMP and is named "IntakeTempB" might be useful. A simple extension would be to allow a short "Description" attribute as well.

One of the ongoing areas for clarifying is when to define additional attributes (classes and the members of each), and when to just let the name.and/or description serve. This is a classic question of controlled vocabularies (pre-defined enums) vs free tagging (just a string or set of strings). If done right, a mixture of these can work well.

At this point one might feel this is getting way too complicated, but really I think it's coming into focus. You can always omit most of these attributes (except structure and V_* type perhaps), and you will default to something like the current situation - except that it has the hooks to do more in a consistent way when you are ready.

I will note that these attributes can be stored in PROGMEM and need not be very large, really, for most nodes. The predefined attributes might be as simple as two bytes (vocabulary and term) to allow up to 256 vocabs with up to 256 terms each). If you run out of PROGMEM, omit the less valuable attributes.

AND - note that these attributes can also be associated with a given Variable in the configuration of the adapter function. For example, there could be a simple text file which links node 7/child 3/var 2 to a set of attributes (encoded as a text string with simple delims). The advantage of putting them into the node is that they will aid in self-documentation and discovery and reduce the effort and errors of configuration, but conceptually the attributes (except structure) are not needed until they get to the controller adapter, so they could be reported over the air when a node joins (or is queried), or they could be pulled from a file or other memory in the central system.

(Yes the controller adapter can be implemented as a plugin of the controller).

What I picture is that in the sensactuator node code, one would typically define the types of variables within each child SensorGroup statically, and that would include storing the attributes in PROGMEM as a byte array.

A node would only automatically report the minimum when joining the network. The hub could query it to find out:

 Node name (optional)
 number of SensorGroups
 for each SensorGroup:
     sensor type hint (optional) (S_*)
     Sensor name (optional)
     number of variables
     for each variable:
          variable structure (size and signedness, or string or bool, etc)
          basic variable type (V_*)
          variable name (optional)
          optional attributes (sequence of vocab/term bytes)  (optional)

Maybe add optional "Description" to some of these levels?

It would not do this querying with a single packet! For example there might be a separate query for each variable name, and for each variable's list of optional attributes. And the hub would not be required to query for and use the optional parts. The required parts would tell it everything it knows in the current MySensors architecture.

And finally (for now), this is one of the places where I could see a smart wireless hub doing much of the legwork. Probably using a Mega or Due or Teensy 3.x (or maybe even a RPi). This device could handle querying new sensactuator nodes to create a copy of all this informaion in local RAM. Or reading some of it from config files (eg: SD card) and combining that with the bare skeleton reported over the air. In either case, the adapter to a given controller would just have all this info (the hierarchy of SensorGroups and Variable with all names and attributes. That info would be part of the API which the adapter gets to use in accessing the sensor network.

OR - a dumb wireless hub could let a smart adapter (eg: running on the home automation controller) handle all this polling for names and attributes. That has implications but could be the best solution in some cases.

(note 1: I might start numbering the variables within a sensor node with 1, reserving 0 for other purposes)

John

@Zeph

The array hek was using as an example (of possible future approaches) was for variables within a "sensor", not sensors within a node.

Yes you are right, i should have used the correct naming.

This is very interesting because there are two V_TEMP values in the list.

I should have better displayed that the second V_TEMP was an addition i made to the remark of what heck posted, my excuse for this. It also does not have to mean there should be an extra address parameter for the specific enum type. Currently, if I've understand correctly the variable type is being send, this then could be replaced with the variable char array position. This because the proposed present already has defined which variable is at what position.

Let's say it is possible to do the below

#define ID 1 /// Barometer child id
char variables[] = {V_FORECAST, V_TEMP, V_PRESURE, V_TEMP} /// Last temp added by me (john in addition to heck's example)
gw.present(ID, S_BAROMETER , variables);

The sendVariable would then look like this:

v_temp_var_at_pos_1 = analogRead(somePin * (somecalculations)); //// Read an attached temp sensor
v_temp_var_at_pos_3 = analogRead(someOtherPin * (somecalculations)); //// Read another attached temp sensor.
gw.sendVariable(ID, 1, v_temp_var_at_pos_1);
gw.sendVariable(ID, 3, v_temp_var_at_pos_3);

Because the controller knows at what position what variable is send, there would be no need to alter the sendVariable to much. This is then completely different from the current variable update notification because it then has to be clear it is targeting a position instead of type presentation.

I hope i clarified my post.

Each V* type would imply a standard unit to be used

And each standardized unit like the V_TEMP example should also stay with a fixed datatype float will be fine because of decimals.

UNCALIBRATED

Would be more like: The reported value is not the value you would expect because when your thermometer shows 12°C i'm reporting 14°C so you need to calibrate me. And not reporting a different unit.

Damme

I am sorry to say this, but I do not really see the gain in this quite complex way?

Still I don't even see the idea of having to present a sensor type and a sensor variable.
Everyone agrees on the following:

The gateway talks to one or many nodes
The node can have one or many sensors

Today we use:

The node has a name (I_SKETCH_NAME) and a version (I_SKETCH_VERSION)
We present every sensor with a ID, an S_TYPE.
We send data with a ID and a V_TYPE.

You are talking about:

The Node present one or many S_TYPE with one or many V_TYPE. (In this setup, The node might handle the ID itself?)
The Node (still?) sends one Value with sensorID and V_TYPE. Here we are missing S_TYPE. (longer address - smaller payload) (How do we handle multiple S_TYPE? How do we separate one from another? And if we have multiple v-temp in one sensor? witch is what and who know about it? v_temp1 v_temp2?)

No, NO! :bomb: This just messes things up imo. I already think its messed up already with this S_ and V_ allocations. We should stick with ONLY . One type-array. And let controller configuration handle groupings...
OR if someone wants this more complicated way of work, he can program it in the node code himself by constructing a payload with more stuff in it, example ;
Today we use ID and V_TYPE in the address field, no one forces anyone to do this. The user can program an ID and S_TYPE in the address instead and the payload could be constructed as "v_type:VALUE"...

Me myself is using the sketchname for keeping track of what type of node (sensor group in your view) it is. But this could be sensor ID0 reserved for this. Everyone is free to program whatever they want! You dont even have to use V_TYPE.. You could use this as an adress as well, so you can actually address 256x256 sensors on every node..

Today you could even present every ID with a S_TYPE (id 1+2+3+4 = S_WEATHER) and then just send data with ID and V_TYPE.. Let the controller keep track of eveything, this will pose a problem for me tough in my MQTT broker. I dont have the space of storing this type of data. So I'll be might start publishing the Presentation data, This data is just discarded of today. But because I know people using stuff are full of ideas they will work around this problem.

I dont think the progress in here is actually leading for a simple sensor network at all. I see tons of problems from a controller and gateway programmer's view. Sorry guys..

//Damme

John

@Damme

Hi,

The node might handle the ID itself?

I used the reference on the top of this page: http://www.mysensors.org/build/sensor_api

What i'm trying to put in the example is for a S_TYPE to supply multiple V_TYPE'S and the possibility to have the same V_TYPES based on fixed positions defined by the example char array. Just because the current setup is working with S_TYPES and V_TYPES.

I have not thought about multiple S_TYPE's of the same kind... good point..... But would this not just send multiple presentations like:
gw.sendSensorPresentation(ID1, S_DOOR);
gw.sendSensorPresentation(ID2, S_MOTION);

Or am i wrong... ?

I agree absolutely with what your saying and that everything that can be done with data handling like the metrics and grouping etc. should be done on the controller.

The current setup where there is just one group level is from what i think quite clever, of course on controller side it can be discarded, but it gives a nice multiple single purpose, multi sensor setup. An example with S_TYPE "Door":

With the possibility to announce in advance what kind of V_TYPES it can send, lets say: V_LOCK_STATUS at position 0 in the char array and V_OPEN_STATUS at position 1 in the char array (just made up). With the possibility to announce in this way it is possible to auto create this "door" device and will of course be a one time request.
When handling the V_LOCK_STATUS data on controller side when mysensors would send, as current is the case, the V_LOCK_STATUS it can send the array position of this V_LOCK_STATUS. Because the controller knows in advance that position 0 is V_LOCK_STATUS it can bound to that.

Yes it does complicate stuff more on controller side, in my case it would be a simple extra mapping. On the hardware side it would require the programmer to keep track of it's V_TYPE's position.

The driver i'm currently writing does not care how a "device" is defined. It can handle a sensor node as a device where the S_TYPES are groups which can contain multiple V_TYPES. Or it can create a "device" based on the S_TYPE which contains the V_TYPES. My example builds further on @hek 's example, which would give me the opportunity and i think also others to automatically create multi-V_TYPE S_TYPE groups.

Me myself is using the sketchname for keeping track of what type of node (sensor group in your view) it is. But this could be sensor ID0 reserved for this. Everyone is free to program whatever they want! You dont even have to use V_TYPE.. You could use this as an adress as well, so you can actually address 256x256 sensors on every node..

Today you could even present every ID with a S_TYPE (id 1+2+3+4 = S_WEATHER) and then just send data with ID and V_TYPE.. Let the controller keep track of eveything, this will pose a problem for me tough in my MQTT broker. I dont have the space of storing this type of data. So I'll be might start publishing the Presentation data, This data is just discarded of today. But because I know people using stuff are full of ideas they will work around this problem.

With my example I tried to keep the current multi table setup in honor and it currently is the default way to build the sensor network. I think @hek 's example does help to advance the MySensors protocol. And at the moment people are inventing their own idea's with code modifications and change the protocol data position definitions, in which of course they are free to do, they break the default compatibility.

P.S.
I also think this thread could better me renamed to "Advancing MySensors" then "Generalizing MySensors" ;) because it is so extremely difficult or even impossible to generalize at this current moment with all those hundreds of protocols, data- and sensor definitions used. Also with MQTT there is no generalization because everyone is free to create their own hierarchy.

But, eventually, i will be fine with any implementation, as long is i know about it :).

Generalizing MySensors

7

11.7k

11.2k

113.0k