2.0 Discussion: Units, sensor types and protocol
-
@Yveaux said:
[dissing start] Did you think about sending the sensor type only once, durig presentation? There really is no need to send it with each set/req message as it is static for the duration of the connection. For Vera and the like this probably means you should buffer these values in the gateway... [dissing end]
Yes, it would save a byte. But it could be very useful for a simpler controller (which can determine sensor type from each and every message) and when displaying sniffing data ;).
This would only leave 3 COMMAND types left. REQ, SET and STREAM. Where STREAM-data probably could be handled by S_NODE (internal). This means just 1 bit is needed for a SET/REQ flag. But @ToSa would have to feedback on this.
-
@Yveaux said:
[dissing start] Did you think about sending the sensor type only once, durig presentation? There really is no need to send it with each set/req message as it is static for the duration of the connection. For Vera and the like this probably means you should buffer these values in the gateway... [dissing end]
Yes, it would save a byte. But it could be very useful for a simpler controller (which can determine sensor type from each and every message) and when displaying sniffing data ;).
This would only leave 3 COMMAND types left. REQ, SET and STREAM. Where STREAM-data probably could be handled by S_NODE (internal). This means just 1 bit is needed for a SET/REQ flag. But @ToSa would have to feedback on this.
@hek said:
when displaying sniffing data
Yeay yeah, blame it on the sniffer :facepunch:
Making stuff 'easier' to sniff can never be a reason to always send one extra byte...This means just 1 bit is needed for a SET/REQ flag
Looks like the protocol discussion is shifting towards this topic. Better not reserve a bit to indicate in a set/req message which of the two it is, but use different message indicator values. The actual message structures can still be the same.
-
@hek said:
when displaying sniffing data
Yeay yeah, blame it on the sniffer :facepunch:
Making stuff 'easier' to sniff can never be a reason to always send one extra byte...This means just 1 bit is needed for a SET/REQ flag
Looks like the protocol discussion is shifting towards this topic. Better not reserve a bit to indicate in a set/req message which of the two it is, but use different message indicator values. The actual message structures can still be the same.
@Yveaux said:
Yeay yeah, blame it on the sniffer
Making stuff 'easier' to sniff can never be a reason to always send one extra byte...:) We're talking about a total of just 8 bytes header here. And if we remove routing just 5 bytes brimful with useful information about the payload.
Better not reserve a bit to indicate in a set/req message which of the two it is, but use different message indicator values. The actual message structures can still be the same.
:question: I do not understand this.
-
@Yveaux said:
Yeay yeah, blame it on the sniffer
Making stuff 'easier' to sniff can never be a reason to always send one extra byte...:) We're talking about a total of just 8 bytes header here. And if we remove routing just 5 bytes brimful with useful information about the payload.
Better not reserve a bit to indicate in a set/req message which of the two it is, but use different message indicator values. The actual message structures can still be the same.
:question: I do not understand this.
@hek said:
Better not reserve a bit to indicate in a set/req message which of the two it is, but use different message indicator values. The actual message structures can still be the same.
I do not understand this.Probably because I didn't understand you then...
I interpret your post as using a single message structure for either set or req messages. A single bit is used to distinguish between the two, right? -
@hek said:
Better not reserve a bit to indicate in a set/req message which of the two it is, but use different message indicator values. The actual message structures can still be the same.
I do not understand this.Probably because I didn't understand you then...
I interpret your post as using a single message structure for either set or req messages. A single bit is used to distinguish between the two, right? -
@hek What I meant was to just have two separate commands for set & req (like it is now in 1.3 & 1.4b) but use the same messgae structure to send these messages. The command value will be used to differentiate between set & req.
No need to reserve a bit in the message this way.
Hope its clear now.... :dart: -
Ok, I've had a solutions bubbling in my head over night. It will add one byte in the MySensors Req/Set Header but solves a lot of the discussions we've had earlier in other threads as well. Its also replaces both INTERNAL and PRESENTATION command.
--- Moved to the first post ---
@hek
First off - I think this taxonomy of sensactuator values is getting better each time, and I appreciate your mixture of creativity and receptivity as the leader in this effort. Good work.This is not dissing, more like reflecting on what I see more than making judgements.
The S_ and V_ enums (controlled vocabularies) are becoming relatively more orthogonal rather than confusing people by appearing to cover similar conceptual territory twice but in different way, as the earlier versions inherited from Vera tended to do. So for example, S_TEMPERATURE has V_LEVEL rather than V_TEMPERATURE.
One result is that to understand what a transmitted value (number) represents, you would now need to look at both the S_ code and the V_code (as a good thing). The "dimension" and "units" concepts for understanding a V_LEVEL value depend on the S_ code associated with the variable. In a sense there would now be a two-byte combined expansion of the concepts often formerly described in the one byte V_ code. (So instead of V_TEMPERATURE, there is V_LEVEL+S_TEMPERATURE and you have to look at both to see that the number is temperature in degrees C).I think this is getting cleaner.
The semantics or meaning of a variable are still somewhat distributed between the S and V codes. For example V_LEVEL and V_ANGLE have very little semantic content, relying almost entirely on the S code to provide that, but V_ARMED vs V_TRIPPED do have semantic content. This is just an observation; I'm still letting it sink in.
A big question is the relationship of node children with S_ code - both are called sensors but they need not be the same thing. For this discussion a "Child" is an addressable subunit of a node, numbered 0,1,2... and each child can contain a set of variables. An S code is a one byte code associated with each variable (along with a V code). In an earlier discussion where I was questioning the value of the "Sensor" level of the heirarchy you made the point that there could be value in grouping several variables that are logically tied to "the same thing". The Child concept does that, it provides an addressable container for one or more variables.
At issue is whether there is a fixed relationship beween the Child and the S_ code, whether all variables within the same Child container must have the same S_ code or not.
-
If all variables in a given Child must have the same S_ code, then as somebody else pointed out this could be sent to the hub once during node integration and stored there (associated with the given node/child), rather than being transmitted with each value. But you cannot include variables for temperature and humidity in the same Child, even if logically that's what you want to convey (say you have two DHT-11's on the same node, you can't just make them two Children each with a temp and humidity).
-
If each variable within a given Child can have its own S code, then the S code and V code become a two byte descriptor of the variable, and Child is a flexible container used to represent "these variables go together" and only that.
Which of these relationships between a Child (addressable subunit of a node) and S code (one byte code that along with V code helps interpret a given variable) did you have in mind?
-
-
To simplify sketches even further we've discussed and decided to let sensors send in standard SI units and let controller convert them to the appropriate format.
This will also remove the need for a unit-config exchange with controller at sensor startup.
The new over-the-air messages has changed quite much with a more logical structure that can handle extension and even a completely different network layer if needed.
The ongoing work can be found here:
https://github.com/henrikekblad/Arduino/blob/development/libraries/MySensors/MyMessage.hWill not bring this into development until it actually compiles.
Feel free to give feedback and report any missed sensor-types.
In parallell we should probably discuss the serial-protocol for 2.0. Is it time for json perhaps?
@hek All these changes are more 2.0 than 1.4.. :)
Edit;
Btw, All these extra bytes S_ and V_ to describe a payload is good for mqtt also.
And this is why we need some more command types (i.e. Stream for OTA update etc. more space for payload smaller header) -
I see the edited version with MIN, MAX and RESET's added. This takes the orthogonality of V and S codes even further, and it seems like a good direction.
And it's good to see a start on better addressing one of the complicated aspects of this framework = accumulation periods (for Max/Min/Average/Total).
But is V_RESET a variable or a command, conceptually? Is is something sent to the node (probably) or reported by the hub (probably not)?
This brings up another broad conceptual issue in creating a taxonomny like this - the difference between values and command/events. Many software frameworks have both concepts, for example class member variables and class funtions/events. (In electrical terms. this is somewhat like the "Level" vs "Edge or Trigger" distinction).
In the current taxonomy, command/event and value are kind of intermingled. I think it will be important to document which is which.
There are a couple of approaches. One is to define a list of commands (eg; C_RESET) distince from values (V_LEVEL); the other is to have a well defined way of using variables as pseudo-commands.
For one possible example of the latter, one could say that for any variable defined as a command variable:
- when the variable goes from 0 to 1, the node will take the defined action
- the hub will send the node a "Set" command to set the value to 1; if it was 0, then the action is taken
- Setting to 0 has no effect
- the node will set the value to 0 when the action is completed (may be immediate)
- the hub can query the current value of the variable to see if there is any action still pending or not
(This might be exactly what you had in mind, or you might have had a variation in mind - I'm just saying that it should be clearly defined and documented).
Then we could say that V_RESET is a command variable and thus shares the standard dynamics, where the specific "action" is to restart the accumulation period. And there could be other command variables, each documented as such.
(The Event concept is similar to a command in that it's a transition rather than a level, but used as reports from the node to the hub. Not addressed yet)
-
One characteristic of the MySensors system is that nodes are not systematically aware of time. That's good in that it simplifies the node firmware, and it's a limitation in terms of dealing with accumulation periods.
I'm speaking of "accumulation" periods rather than "reporting" periods, because a node may "report" many times in an accumulation period, for example every 10 seconds it may report the accumulated rainfall since midnight.
And there are several kinds of accumulation periods: in particular rolling and fixed start. Fixed start might report accumulation (sum, average, min, max) since a given time like midnight. Rolling might report the last hour or last 5 minutes from now. (I mentioned a rain sensor which does it's own "rain event" splitting based on no-rain intervals).
In some cases, accumulation periods may not be under our control - for example reading an integrated weather station. At best a node could report the accumulation period.
In other cases, we can control the accumulation period, for example the node's sketch could itself count KWH or rain, or it could average temperature or speed or direction, taking readings every N seconds.
One more concept: how to handle a Reset. The simplest approach for a hub-commanded reset would be for the hub to be sure it has all desired recent value reports before sending a Reset command (so the time gap between last report and Reset is short enough that nothing signficant was missed, eg: mm of rain). A more synchronous approach would be for the node to send one last set of accumulated values immediately before zeroing it's counter.
But what about a manually triggered reset (button pushed)? The sensor node could in that case want to let the hub know that it happened.
One way to unify these in a simple and consistent way would be to report V_ACCUM_TIME - the number of seconds since the accumulation period began. That's the period over which values were averaged, summed, or min/max'd. The time would increase until a reset, go to zero and increase again. This handles controllable and not controllable accumulation periods, fixed and rolling accumulation periods, hub triggered and manually triggered resets.
If an accumulation period is reset, the node could potentially send a last report with the ending V_ACCUM_TIME before the reset and then the following report would have time since starting the new period.
If the time is just based on millis() then with the Arduino Pro Micro the time accuracy would be nominally about 0.5% (they have ceramic resonators rather than crystals and that's a typical spec). That's probably good enough, tho it's possible to do better if it mattered.
Where this seems to be going is that any quantity variable could potentially be expanded into an accumulation variable - which means that in addition to (say) V_LEVEL, there is V_LEVEL_MAX, V_LEVEL_MIN, V_LEVEL_TIME, and either V_LEVEL_AVG or V_LEVEL_SUM. This "accumulation enhancement" could optionally be added to temp, humidity, wind, power, fluid/gas flow, whatever - in a defined and consistent way.
-
One characteristic of the MySensors system is that nodes are not systematically aware of time. That's good in that it simplifies the node firmware, and it's a limitation in terms of dealing with accumulation periods.
I'm speaking of "accumulation" periods rather than "reporting" periods, because a node may "report" many times in an accumulation period, for example every 10 seconds it may report the accumulated rainfall since midnight.
And there are several kinds of accumulation periods: in particular rolling and fixed start. Fixed start might report accumulation (sum, average, min, max) since a given time like midnight. Rolling might report the last hour or last 5 minutes from now. (I mentioned a rain sensor which does it's own "rain event" splitting based on no-rain intervals).
In some cases, accumulation periods may not be under our control - for example reading an integrated weather station. At best a node could report the accumulation period.
In other cases, we can control the accumulation period, for example the node's sketch could itself count KWH or rain, or it could average temperature or speed or direction, taking readings every N seconds.
One more concept: how to handle a Reset. The simplest approach for a hub-commanded reset would be for the hub to be sure it has all desired recent value reports before sending a Reset command (so the time gap between last report and Reset is short enough that nothing signficant was missed, eg: mm of rain). A more synchronous approach would be for the node to send one last set of accumulated values immediately before zeroing it's counter.
But what about a manually triggered reset (button pushed)? The sensor node could in that case want to let the hub know that it happened.
One way to unify these in a simple and consistent way would be to report V_ACCUM_TIME - the number of seconds since the accumulation period began. That's the period over which values were averaged, summed, or min/max'd. The time would increase until a reset, go to zero and increase again. This handles controllable and not controllable accumulation periods, fixed and rolling accumulation periods, hub triggered and manually triggered resets.
If an accumulation period is reset, the node could potentially send a last report with the ending V_ACCUM_TIME before the reset and then the following report would have time since starting the new period.
If the time is just based on millis() then with the Arduino Pro Micro the time accuracy would be nominally about 0.5% (they have ceramic resonators rather than crystals and that's a typical spec). That's probably good enough, tho it's possible to do better if it mattered.
Where this seems to be going is that any quantity variable could potentially be expanded into an accumulation variable - which means that in addition to (say) V_LEVEL, there is V_LEVEL_MAX, V_LEVEL_MIN, V_LEVEL_TIME, and either V_LEVEL_AVG or V_LEVEL_SUM. This "accumulation enhancement" could optionally be added to temp, humidity, wind, power, fluid/gas flow, whatever - in a defined and consistent way.
FYI: Sleeping sensors (who also can be woken by a interrupt on a pin) will have a hard time keeping time without clock module or by fetching time from server at each wakeup from pin trigger.
Good idea about a STATUS for RAIN sensor. It will be added and indicates if rain is ongoing.
Will also change to ACCUMULATED for all accumulated values. Average for these makes no sense.So.. I see now that there is need for more than one reset command. We'll need RESET_ACCUMELATED, RESET_LEVEL, RESET_PERCENTAGE. They should also have their respective MAX, MIN, AVERAGE. Could you please give me a use case for V_LEVEL_SUM.
Just to clarify for anyone getting a bit scared of all these new variables. Using them for your own sensor is of course optional!
I feel it's time to move the new stuff to the first post to keep it in focus.
-
FYI: Sleeping sensors (who also can be woken by a interrupt on a pin) will have a hard time keeping time without clock module or by fetching time from server at each wakeup from pin trigger.
Good idea about a STATUS for RAIN sensor. It will be added and indicates if rain is ongoing.
Will also change to ACCUMULATED for all accumulated values. Average for these makes no sense.So.. I see now that there is need for more than one reset command. We'll need RESET_ACCUMELATED, RESET_LEVEL, RESET_PERCENTAGE. They should also have their respective MAX, MIN, AVERAGE. Could you please give me a use case for V_LEVEL_SUM.
Just to clarify for anyone getting a bit scared of all these new variables. Using them for your own sensor is of course optional!
I feel it's time to move the new stuff to the first post to keep it in focus.
I dont know why we have to limit some S_type to a number of V_types. Why not just let a S_type have 'all' V_types... User can be more flexible
If I wanted node could send
S_MOTION V_TRIPPED 1 (motion tripped)
S_MOTION V_ARMED 0 (no longer armed)
S_MOTION V_LEVEL 24 (motion quality 24 of 255) (probobly animal)controller decides S_MOTION V_RESET 1
(order not thought of at all, needs to be fixed)
#DEFINE S_DOOR 1 #DEFINE S_MOTION 2 #DEFINE S_SMOKE 3 #DEFINE S_BINARY 4 #DEFINE S_DIMMABLE 5 #DEFINE S_WINDOW_COVER 6 #DEFINE S_THERMOMETER 7 #DEFINE S_HUMIDITY 8 #DEFINE S_BAROMETER 9 #DEFINE S_WIND 10 #DEFINE S_RAIN 11 #DEFINE S_UV 12 #DEFINE S_WEIGHT_SCALE 13 #DEFINE S_POWER 14 #DEFINE S_HEATER 15 #DEFINE S_DISTANCE 16 #DEFINE S_LIGHT_SENSOR 17 #DEFINE S_NODE 18 #DEFINE S_LOCK 19 #DEFINE S_IR 20 #DEFINE S_WATER_METER 21 #DEFINE S_AIR_QUALITY 22 #DEFINE S_CUSTOM 23 #DEFINE S_DUST 24 #DEFINE S_PH 25 #DEFINE S_SCENE_CONTROLLER 26 #DEFINE S_NODE 255 #DEFINE V_CONFIG1 1 #DEFINE V_CONFIG2 2 #DEFINE V_CONFIG3 3 #DEFINE V_CONFIG4 4 #DEFINE V_CONFIG5 5 #DEFINE V_VAR1 6 #DEFINE V_VAR2 7 #DEFINE V_VAR3 8 #DEFINE V_VAR4 9 #DEFINE V_VAR5 10 #DEFINE V_ARMED 11 #DEFINE V_STATUS 12 #DEFINE V_WATT 13 #DEFINE V_PERCENTAGE 14 #DEFINE V_STOP 15 #DEFINE V_LEVEL 16 #DEFINE V_MAX 17 #DEFINE V_MIN 18 #DEFINE V_RESET 19 #DEFINE V_DEW_POINT 20 #DEFINE V_MODE 21 #DEFINE V_ANGLE 22 #DEFINE V_RATE 23 #DEFINE V_VOLTS 24 #DEFINE V_AMPS 25 #DEFINE V_PRESENTATION 26 #DEFINE V_BATTERY_LEVEL 27 #DEFINE V_RESET 28 #DEFINE V_TIME 29 #DEFINE V_ID 30 #DEFINE V_LOG_MESSAGE 31 #DEFINE V_SKETCH_NAME 32 #DEFINE V_SKETCH_VERSION 33 #DEFINE V_FIND_PARENT 34 #DEFINE V_CHILDREN 35 #DEFINE V_VERSION 36 #DEFINE V_INCLUSION_MODE 37 #DEFINE V_GATEWAY_READY 38 #DEFINE V_STATUS 39 #DEFINE V_IR_SEND 40 #DEFINE V_IR_RECEIVE 41 -
FYI: Sleeping sensors (who also can be woken by a interrupt on a pin) will have a hard time keeping time without clock module or by fetching time from server at each wakeup from pin trigger.
Good idea about a STATUS for RAIN sensor. It will be added and indicates if rain is ongoing.
Will also change to ACCUMULATED for all accumulated values. Average for these makes no sense.So.. I see now that there is need for more than one reset command. We'll need RESET_ACCUMELATED, RESET_LEVEL, RESET_PERCENTAGE. They should also have their respective MAX, MIN, AVERAGE. Could you please give me a use case for V_LEVEL_SUM.
Just to clarify for anyone getting a bit scared of all these new variables. Using them for your own sensor is of course optional!
I feel it's time to move the new stuff to the first post to keep it in focus.
@hek said:
FYI: Sleeping sensors (who also can be woken by a interrupt on a pin) will have a hard time keeping time without clock module or by fetching time from server at each wakeup from pin trigger.
Sleeping nodes may have a hard time averaging in some cases too.. But your point is well taken and it is a real concern.
One of the sleep libraries calibrate the RC watchdog timer against the main clock, so it can make a good guess for adjusting milliseconds upon each wakeup. That could help for a simple sleep, but not for waking up from an interrupt.
So the problem child is a node for which all the following apply:
- sleeps
- is awoken by interrupts at unpredictable intervals
- has no local RTC
- has accumulated variables
One option would just be to ignore the accumulated time feature for the subset of nodes in that category; for them, you have to fall back to where would be be if I had never spoken up about tracking time. For other nodes, the tracking & reporting of accumulation time can be an enhancement, as described in an earlier message.
But suppose we DO want to go further and determine accumulation time for all nodes, even the above.
(Sigh. I acknowledged that there was a simplicity advantage to not dealing with time, and that just using the local node's millis() was the simplest but not only way to handle time; and that does handle most cases but not the above)
System Time approach 1: Master Clock Broadcast to nodes
The hub will have the master clock, using its millis(). Periodically it will broadcast the value of this to all nodes. A node could also request a time be sent. We could apply a crude delta to adjust this according to number of hops and typical transmission time, or not - we don't need anywhere close to millisecond accuracy. Each node would maintain a delta between its own millisecond clock and the broadcast millis and adjust that every time it receives that broadcast. (Smart nodes could even use a computed time scaling factor to adjust for differences in their clock vs the master). When for any reason has a need to estimate the current time (including but not limited to the case of having been powered up from sleeping by an interrupt AND needing to repost accumulations with more precision than it can estimate), the node can either wait for the next broadcast, or request one.
System Time approach 2: Track accumulation times in the hub
Whenever an accumulation period is started, the node will send a message to the hub to that effect (telling it which variable is involved and an incrementing reset count). This could be triggered by a commanded or manual reset, or by the automatic reset of an external weather station or service, or whatever. Then it would send the reset count (V_RESET_COUNT) rather than the accumulated time along with accumulation based values. The hub can look at its current time, and at the recorded time of the last reported reset of that variable, to compute for itself the elapsed time. The reset count makes sure both sides are sync'd as to which reset they are talking about. Both the "Starting accumulation period #n for (variable)" and the "reporting accumulated value for (variable)" messages travel from node to hub, and their travel times will roughly cancel out, at least well enough.
The former approach is more powerful. It better supports rolling averages. And it would provide a flexible "MySensors network time" framework for future features, like reporting event times or periods more accurately.
For example, I've considered putting two motion detectors at angles and with masks such that one could usually detect which way somebody went through a doorway by the trigger times, or at different place in a hallway. I'm not proposing to support that directly at this time, but when it comes up, if each node had the option of knowing the hub's master clock time to within tens of milliseconds or better, that could be a useful base upon which to build.
[Total aside. My ambition for another nRF + Arduino system is more precise than this, possibly even enough to do PWM of 60 Hz mains powered lights without a local zero crossing detector - not yet determined. That's why I connect the nRF24L01+'s IRQ output to Arduino pin 8 = ICP1, even if I'm not using interrupt driven nRF code. If the nRF's receive interrupt is enabled, this allows the ATMega328p to get a very accurate time of reception using Timer 1 if it wants, even in polling mode without interrupts. And if I ever do want to interrupt the ATMega from the nRF, I can use the capture interrupt, while leaving INT0 and INT1 on pins 3 & 4 for other uses. The above first option is a smaller, simpler version of that design, adapted for the needs of a sensor network]
-
FYI: Sleeping sensors (who also can be woken by a interrupt on a pin) will have a hard time keeping time without clock module or by fetching time from server at each wakeup from pin trigger.
Good idea about a STATUS for RAIN sensor. It will be added and indicates if rain is ongoing.
Will also change to ACCUMULATED for all accumulated values. Average for these makes no sense.So.. I see now that there is need for more than one reset command. We'll need RESET_ACCUMELATED, RESET_LEVEL, RESET_PERCENTAGE. They should also have their respective MAX, MIN, AVERAGE. Could you please give me a use case for V_LEVEL_SUM.
Just to clarify for anyone getting a bit scared of all these new variables. Using them for your own sensor is of course optional!
I feel it's time to move the new stuff to the first post to keep it in focus.
@hek said:
Could you please give me a use case for V_LEVEL_SUM.
-
"snapshot in time" values like temperature and humidity and watts logically accumulate as (averages. min, and max) over the given period as their end product. Tuesday the average temperature was 22.3 degrees.
-
"incrementing total" values like KWH, liquid flow, and rainfall logically accumulate as total or (sum) over the given period as their primary end product: Tuesday a total of 20mm of rain fell. (IFF you know the time as well, you can calculate an average as a secondary product)
These two types of values have somewhat different dynamics.
Of course if you report a total AND an elapsed time, the hub can calculate averages for itself, but not min and max. This calculation can be done at the hub even for temperature, tho total degrees C is not meaningful and total mm rain is.
So IF we can rely on S_LEVEL_ELAPSED and S_LEVEL_SUM both always being available (and the latter not overflowing integer formats), we could omit S_LEVEL_AVG. But in the original proposal without a time framework, the node should report S_LEVEL_SUM for rain and S_LEVEL_AVG for temp.
-
-
To simplify sketches even further we've discussed and decided to let sensors send in standard SI units and let controller convert them to the appropriate format.
This will also remove the need for a unit-config exchange with controller at sensor startup.
The new over-the-air messages has changed quite much with a more logical structure that can handle extension and even a completely different network layer if needed.
The ongoing work can be found here:
https://github.com/henrikekblad/Arduino/blob/development/libraries/MySensors/MyMessage.hWill not bring this into development until it actually compiles.
Feel free to give feedback and report any missed sensor-types.
In parallell we should probably discuss the serial-protocol for 2.0. Is it time for json perhaps?
EDIT - there are some good concepts in the next few posts from me (IMHO obviously), but some may turn out to be sidestepped by a more radical change described in response
http://forum.mysensors.org/topic/304/unit-standardization-1-4/57,
so you might want to just read that first.
@hek said:
###S_WIND - Wind sensor
V_LEVEL - Wind level in m/s (average wind speed during last report period)
V_LEVEL_MIN - Min wind level
V_LEVEL_MAX - Max wind level (gust)
V_LEVEL_AVERAGE - Average wind level-
V_LEVEL_RESET - Reset max/min value
V_ANGLE - radians <float or int>First, i still want to convince you that V_ANGLE in "degrees clockwise from true north" is the proper measure here, not radians. (and you need to define if this points to where the wind comes from or where it's moving towards, so is a west wind 90 degrees or 270 degrees - or translate to radians if you want to see ugly)
Look, I'm a math person (or was) and I totally get that e, national logs and exponents, pi and radians are more fundamentally defined values than degrees or using base 10 for decibels or logs. But the point is not scientific purity (which are are not really close to anyway) but pragmatic utility in the field of sensor networks. And degrees simply work better in this context of conveying direction, especially if you allow int representation. Enough on that for now.
Second, notice that you really only have two real measurements here: speed and direction. Those are what I call snapshot values - if you sample them at a given time, you get a value, which can later rise or fall.
The former has additional derived or calculated sub-values, and those subvalues follow a pattern representing "simple time accumulation statistics" and also common to many other snapshot variable types, like humidity.
The thing is - direction is also a snapshot value. If you are going to report average wind over the past hour, you can also report average direction.
Alas, angle differs from continuum measurements in that it wraps, so average/min/max have to be computed carefully and are only meaningful if the source stays within a reasonable range; and sum makes no sense.
But any continuum variable is a potential candidate for accumulation-period based derived sub-variables. I picture that there will be a single "accumulation" library which can be applied to any int or float variable, rather than that the temperature library will have accumulation code and the humidity library will have its own accumulation code.
So in the hierarchy I'd put V_LEVEL_MIN, etc one level indented under V_LEVEL rather than make it look like a parallel and independent measurement.
Let's also be clear that there are two reasons to have accumulation period based derived variables. One is that the node could be getting values from a device which does its own accumulation of some sort (eg: a weather station or some rain gauges). The other is that the device may sample the instantaneous variables more often than it transmits them. If we transmitted every measurement, the hub could potentially do its own min/max/average/sum calculations. No big deal there, but the hub could also accumulate by multiple arbitrary periods, like hour of the day AND since sunrise AND past 10 minutes AND past hour AND since Oct 1st (which is the start of the water year as used by scientists around here). So the most flexible thing would be to report every measurement to the hub and let it handle aggregation any way (as many ways) as it wants. But that could be bandwidth heavy, or might impose too much work on simple hubs (or controllers they report to).
So for one or more of these reasons, sometimes it may make sense to have the sketch running in a node handle accumulation sub-variables. If it reports often enough in a well considered format, the hub or controller or cloud based analytical software can still do aggregation over some of these alternative time periods.
-
To simplify sketches even further we've discussed and decided to let sensors send in standard SI units and let controller convert them to the appropriate format.
This will also remove the need for a unit-config exchange with controller at sensor startup.
The new over-the-air messages has changed quite much with a more logical structure that can handle extension and even a completely different network layer if needed.
The ongoing work can be found here:
https://github.com/henrikekblad/Arduino/blob/development/libraries/MySensors/MyMessage.hWill not bring this into development until it actually compiles.
Feel free to give feedback and report any missed sensor-types.
In parallell we should probably discuss the serial-protocol for 2.0. Is it time for json perhaps?
@hek said:
###S_BINARY - Light (or other) on/off actuator
V_STATUS - 1 - turn on, 0 = turn off###S_DIMMABLE - Dimmable actuator
V_STATUS - 1 - turn on, 0 = turn off
V_PERCENTAGE - Dimmer level 0-100%OK, let's define what happens if you get more than one type of command (probably just making explicit what we are thinking anyway).
S_DIMMABLE:
V_STATUS = 1 is the same as V_PERCENTAGE = 100, either has same effect
V_STATUS = 0 is the same as V_PERCENTAGE = 0, either has same effectIf you set V_STATUS = 1 and then query V_PERCENTAGE it should return 100% (or 0 => 0%)
If you set V_PERCENTAGE = 100 and query V_STATUS it should return 1OK?
The only question is setting V_PERCENTAGE = 1..99 and querying V_STATUS.
Option 1: Return -1 for "neither fully on nor fully off"
Option 2: Return 1 for "at least partially on" (ie: anything above 0%)
OK, what about a Fan with setting Off, Low and High, and another fan with settings "Off, Low, Medium and High?
We could map those to text "OFF, LOW, MED, HIGH"
Or we could map them to 0,1,2 where 2=HIGH for the first fan, and 0,1,2,3 where 3 = HIGH for the secondI'm going to suggest that we primarily map to V_PERCENTAGE, but perhaps also support the latter option as an expansion of binary 0..1 into multi step value 0..N_STEPS.
All fans: OFF = 0% HIGH = 100% Fan 1: Low = 50% Fan 2: Low = 33% Med = 67%If you are sent a V_PERCENTAGE, set the fan to the closest speed to the given percentage. 100% is always the highest speed, 0% is always OFF, in between will approximate. This extends to 4 level fans. And it works for a heater or motor with multiple speeds as well. If you query V_PERCENTAGE the actual value chosen from the list will be returned, not the requested value (ie: requesting 45% but reporting 50% in the case of fan 1).
The above intermediate percentages are the default. However the node is empowered to set other values. We all know of fans or other devices where the speeds are not evenly spaced. I have a heater which has 600, 900 and 1500 watt settings (both elements). The Low and Medium setpoints could be 40% and 60%. Otherwise it works as described above.
And there could be an integer V_SET_STEP (or something), corresponding to V_BINARY* but ranging from 0 to N_STEPS where N_STEPS is per-variable defined. I'm not attached to this, but it's an option, It would work in the obvious way. So V_SET_STEP of 1 for Fan1 would set it to Low=50% and V_SET_STEP of 1 for fan2 would set it to Low=33% (and V_SET_STEP of 1 for the heater example would set it to Low=40%. SET_STEP of 0 is always off, and SET_STEP of N_STEPS is always HIGH or 100%
For completeness, I will mention that there's another way to define this for V_DIMMABLE. It could be required to memorize the last set percentage, and then V_STATUS = 0 means turn to 0% but remember last non-zero percentat, and V_STATUS = 1 could mean return to last non-zero percentage.
Tho I don't favor this dynamcic, it too could work. However I strongly favor defining which way we expect things to work rather than leaving it ambiguous!
-
I dont know why we have to limit some S_type to a number of V_types. Why not just let a S_type have 'all' V_types... User can be more flexible
If I wanted node could send
S_MOTION V_TRIPPED 1 (motion tripped)
S_MOTION V_ARMED 0 (no longer armed)
S_MOTION V_LEVEL 24 (motion quality 24 of 255) (probobly animal)controller decides S_MOTION V_RESET 1
(order not thought of at all, needs to be fixed)
#DEFINE S_DOOR 1 #DEFINE S_MOTION 2 #DEFINE S_SMOKE 3 #DEFINE S_BINARY 4 #DEFINE S_DIMMABLE 5 #DEFINE S_WINDOW_COVER 6 #DEFINE S_THERMOMETER 7 #DEFINE S_HUMIDITY 8 #DEFINE S_BAROMETER 9 #DEFINE S_WIND 10 #DEFINE S_RAIN 11 #DEFINE S_UV 12 #DEFINE S_WEIGHT_SCALE 13 #DEFINE S_POWER 14 #DEFINE S_HEATER 15 #DEFINE S_DISTANCE 16 #DEFINE S_LIGHT_SENSOR 17 #DEFINE S_NODE 18 #DEFINE S_LOCK 19 #DEFINE S_IR 20 #DEFINE S_WATER_METER 21 #DEFINE S_AIR_QUALITY 22 #DEFINE S_CUSTOM 23 #DEFINE S_DUST 24 #DEFINE S_PH 25 #DEFINE S_SCENE_CONTROLLER 26 #DEFINE S_NODE 255 #DEFINE V_CONFIG1 1 #DEFINE V_CONFIG2 2 #DEFINE V_CONFIG3 3 #DEFINE V_CONFIG4 4 #DEFINE V_CONFIG5 5 #DEFINE V_VAR1 6 #DEFINE V_VAR2 7 #DEFINE V_VAR3 8 #DEFINE V_VAR4 9 #DEFINE V_VAR5 10 #DEFINE V_ARMED 11 #DEFINE V_STATUS 12 #DEFINE V_WATT 13 #DEFINE V_PERCENTAGE 14 #DEFINE V_STOP 15 #DEFINE V_LEVEL 16 #DEFINE V_MAX 17 #DEFINE V_MIN 18 #DEFINE V_RESET 19 #DEFINE V_DEW_POINT 20 #DEFINE V_MODE 21 #DEFINE V_ANGLE 22 #DEFINE V_RATE 23 #DEFINE V_VOLTS 24 #DEFINE V_AMPS 25 #DEFINE V_PRESENTATION 26 #DEFINE V_BATTERY_LEVEL 27 #DEFINE V_RESET 28 #DEFINE V_TIME 29 #DEFINE V_ID 30 #DEFINE V_LOG_MESSAGE 31 #DEFINE V_SKETCH_NAME 32 #DEFINE V_SKETCH_VERSION 33 #DEFINE V_FIND_PARENT 34 #DEFINE V_CHILDREN 35 #DEFINE V_VERSION 36 #DEFINE V_INCLUSION_MODE 37 #DEFINE V_GATEWAY_READY 38 #DEFINE V_STATUS 39 #DEFINE V_IR_SEND 40 #DEFINE V_IR_RECEIVE 41@Damme said:
I dont know why we have to limit some S_type to a number of V_types. Why not just let a S_type have 'all' V_types... User can be more flexible
As described at the time I write, the proposed taxonomy of values isn't really as heirarchical as it's presented.
You can also parse the meaning from V code to S code:Ah, we have a V_LEVEL value, that means it represents a one dimensions continuum But what does that continuum represents? Oh, I see the variable is also tagged with S_TEMPERATURE That means the continuum is temperature in degrees CThat is, the V code tells you something of the structural concept, and the S code adds some degree of semantics (like dimension and unit).
So you could also invert the displayed hierarchy and show which S codes are available for a given V code
-
Hold the presses.
I've happily engaged in the collaborative process of cleaning up the variable taxonomies of V 1.4b here.
But we keep coming to conceptual tricky parts. Some of that is inherent, but in thinking deeper about it, i've come to the conclusion that a good part of what makes it hard to resolve cleanly is that we are trying to cram too much information into too few bits of variable metadata which has to be included in every packet.
So rather than reforming S and V codes, I'm thinking a more radical approach is needed, to make the system both more flexible and simpler. Part of that comes from identifying what metadata about a variable is static and what needs to be dynamic, and then moving the static part out of the normal OTA operations. That turns out to make the taxonomy much more tractable and flexible while simplifying the node firmware.
Guess what - even child ID turns out to be metadata in this approach, rather than addressing.
Here's the sketch of the new approach.
http://forum.mysensors.org/topic/308/another-way-of-organizing-variables
It's sufficiently different that I expect it may encounter resistance - from the natural and appropriate fear of breaking too much (I share that concern). But it just may turn out to be sufficiently attractive to be worth it. Have a look. And now would be the time to consider such a break - with RadioHead and OTA programming support also in the air.
-
@Yveaux said:
[dissing start] Did you think about sending the sensor type only once, durig presentation? There really is no need to send it with each set/req message as it is static for the duration of the connection. For Vera and the like this probably means you should buffer these values in the gateway... [dissing end]
Yes, it would save a byte. But it could be very useful for a simpler controller (which can determine sensor type from each and every message) and when displaying sniffing data ;).
This would only leave 3 COMMAND types left. REQ, SET and STREAM. Where STREAM-data probably could be handled by S_NODE (internal). This means just 1 bit is needed for a SET/REQ flag. But @ToSa would have to feedback on this.
@hek said:
This would only leave 3 COMMAND types left. REQ, SET and STREAM. Where STREAM-data probably could be handled by S_NODE (internal). This means just 1 bit is needed for a SET/REQ flag. But @ToSa would have to feedback on this.
I switched the bootloader code between using stream and using simple internal messages - at the end doesn't matter as long as it's simple enough to dis-samble and assemble a packet without a lot of additional code overhead.
-
@hek said:
This would only leave 3 COMMAND types left. REQ, SET and STREAM. Where STREAM-data probably could be handled by S_NODE (internal). This means just 1 bit is needed for a SET/REQ flag. But @ToSa would have to feedback on this.
I switched the bootloader code between using stream and using simple internal messages - at the end doesn't matter as long as it's simple enough to dis-samble and assemble a packet without a lot of additional code overhead.