2.0 Discussion: Units, sensor types and protocol
-
@hek said:
V_HUM - Humidity percentage 0-100% <int>
No reason to limit to int, can also be float.
Why do you mention the data types for some and not for most? Mysensors doesn't really care....
-
@hek said:
V_WIND - m/s (average wind speed during last hour)
V_GUST - m/s (highest wind speed during last hour)Why average/highest over last hour? My weather station reports wind speed every minute and I like to see the minute detail instead of averaging everything over an hour.
-
-
@hek said:
V_DUST_LEVEL - ug/m3
g/m3 if you standardize on SI
Same for distance BTW. Might be more...
-
@hek said:
V_DIRECTION
Maybe just return angle ( in radians and call it V_ANGLE) then textual indication. This way higher resolutions can also be supported.
Sorry for all the separate messages. Works more convenient on a tablet...
-
I would like a way to configure a sensor from the gateway, to e.g calibrate a sensor or store some ID. Maybe a V_CONFIG can be added, where childid indicates the actual variable written. Or define a few V_CONFIG0,V_CONFIG1 etc.
What do you think?
-
@Yveaux said:
g/m3 if you standardize on SI
Same for distance BTW. Might be more...I choose to scale the SI units to something relevant for thing getting measured. E.g. g/m3 would give a VERY small small value for dust.
Distance is mainly for the distance sensor now, which only has a small range.
Rain - measure in meters would also render very small numbers.
So, until we have a scaling factor in the protocol or something similar we'll have select a proper scaling at the source and agree on it between sensor and controller.
-
-
-
@Yveaux said:
Why average/highest over last hour? My weather station reports wind speed every minute and I like to see the minute detail instead of averaging everything over an hour.
I'll change the definition to "over the last report period".
-
V_ANGLE sounds good. With a bit of (floating point) maths, all can be calculated.
-
@ServiceXp said:
V_RELAY_STATUS 1=Active, 0=Deactive
Not sure what differs this form V_LIGHT_STATUS.
Maybe we need a better word than LIGHT that describes a thing-that-can-be-turned-on-or-off?
-
@hek said:
So, until we have a scaling factor in the protocol or something similar we'll have select a proper scaling at the source and agree on it between sensor and controller.
When values are sent as floats it doesn't matter. Scaling is implicitly store in the floating point value when sent as binary float. When transfered as ASCII you're right, but that isn't required with 1.4 anymore...
-
-
@a-lurker said:
Electrical impedance and resistance are not the same thing- suspect resistance is intended here?
V_IMPEDANCE --> V_RESISTANCEI think the impedance-variable comes from the RFXTRX plugin. And it is related to body fat measurements for body scales.
http://code.mios.com/trac/mios_rfxtrx/browser/trunk/L_RFXtrx.lua
If I understand it correctly Impedace is used for AC and resistance for DC. Not sure what would be most useful in this setup.
-
@Yveaux said:
When transfered as ASCII you're right, but that isn't required with 1.4 anymore.
Most users still use the serial protocol. Which means strings like "0.000003423" must be generated and transferred to the controller from gateway.
-
-
@hek Moisture or Water detecded... Might be 2 different ?
Thinking different applications: Moisture in ground. (I.e. Water needed? Ground dry?) And Water level, hmm, if you have a fluid level meter to detect overfill. or leakage. Fluid flow
-
@hek said:
strings like "0.000003423" must be generated and transferred
Ok, but is this a problem? You could even pass it as 3.423e-6....
You don't want this string on air if you can transfer the same thing in 4 bytes, that's true.
I only want to state that if you transfer it purely in SI units, without scaling, you always know what the actual value is.
-
I'm still in doubt if we really need all these different value types...
In principle a (light) switch, water level sensor, burglar alarm, heater status <fill in your own binary switch> all just return or take 0 or 1. V_BINARY should suffice for all of them.
For other values you see the struggle between defining them as bare units (V_AMPS, V_VOLTS, V_CUBIC_MEASURE) and function (V_FLOW, V_WEIGHT, V_HUMIDITY).
Can't we just live with transfering values by specifying their units only and leave the interpretation of it to the controller (e.g. V_AMPS, V_BINARY, V_PERCENTAGE) ?
I'm not familiar with Vera, but e.g. the OpenHAB/MQTT combination doesn't care at all what type of value is exchanged. It's up to the configuration of OpenHAB how a value is interpreted.
-
@Damme said:
Moisture or Water detecded... Might be 2 different ?
For water detected I would probably use V_TRIPPED.
Hmm.. Soil Moisture.. I have no idea how to represent this. Help me! The ebay sensors we're using isn't super accurate and comes uncalibrated.
-
@hek I know, ist more 'need moisture' or not
Hmm... V_Moisture, (relative) 0-255 / 0-100% / True | False
-
@hek Can we also pass battery level as just a sensor value? There's no real need to pass this as an internal message, is it?
-
First off, this is a very positive step forward!
Now some details
@hek said:
> I think the impedance-variable comes from the RFXTRX plugin. And it is related to body fat measurements for body scales.I think that we need to be clear about the core concept of what we intend to measure.
Resistance or impedance generally don't make much sense, as they are usually not the conceptual thing being measured but an intermediate value used in calculating it.
This is also true of Volts and Amps, except that sometimes we really do want volts and amps (for power monitoring).
But consider a Light Dependent Resistor in a bridge circuit.
We could report the value as ADC readings 0..1023.
Or we could convert that to volts 0..3.3 or 0..5.0 or whatever.
Or we could convert that to LDR resistance (knowing the other resistor value) in ohms.
Or we could do some calcs and try to report the result in Lux.Since the conceptual thing we are trying to measure is light intensity, the best option would be to report it in Lux if possible. Not as ohms, volts, or ADC readings.
So any time we see something reporting as resistance or impedance, we should ask if that's the core concept we need to report, or an incompletely processed proxy that will be hard for the server to understand.
Likewise with volts or amps - are we really trying to report temperature or humidty or what? If we are monitoring a battery or the AC mains, then maybe volts it exactly right, but we need to think it out first.
So who in a home automation context cares about resistance or impedance as the core thing being measured?
-
@hek said:
V_TEMPERATURE - Degrees celsius <int or float>
V_HUMIDITY - Humidity percentage 0-100% <int or float>Can any of the values be int or float? Or just these two?
V_DIMMER - Percentage 0-100% (used for lights, blinds and curtains)
The core concept here is V_PERCENT. In one home automation system there might be only one included concept, the Dimmer, and you'd have to use that for blinds as well as dimmers. In another home automation system there might be separate UI widgets for blinds and light dimmers. The sensactuator node's code should not need to change either way. Mapping a given V_PERCENT variable to UI concepts of blinds or dimmers is conceptually part of the configuration for a given (version of a) home automation system.
V_FORECAST Whether forecast. One of "stable", "sunny", "cloudy", "unstable", "thunderstorm" or "unknown"
Difficult one. That is very definitely not the controlled vocabulary used by all forecasting sources, and mapping into this list is going to be difficult. "Scattered showers" and "partly cloudy" and "light rain" and "foggy" are not going to be easily represented. What about stable/sunny versus stable/rainy? Are stable/unstable mutually exclusive choices with the others, or optional additional attributes for any of the other choices?
Maybe this has to be an open vocabulary with a text string? What text they will get depends on their source.
V_RAIN - mm
V_WIND - m/s (average wind speed during last report period)
V_GUST - m/s (highest wind speed during since last report)
V_WATT - Watt <int>
V_KWH - kWh <int or float>What is the report period and what resets it? Last minute, last hour, last 24 hours, since midnight, since the reset button was pushed, or using automatic episode detection (I was just looking at a rain sensor which uses 8 hours of no rain to separate rain accumulation events). Without some knowledge of the reporting period these values cannot be meaningfully interpreted.
V_RAIN could be a forever incrementing total if the host wants to do its own deltas and define it's own report periods (maybe several of them).
V_WIND is a bit more difficult. If we had guaranteed delivery, we could just average since the last report from the node. Of course, some weather stations may do their own averaging, or the node could average many readings. But how does the host know?
From my perspective these "time period sum or average" values would ideally involve:
- an OTA format which loses no useful information available from the sensors or peripherals
- a flexible gateway (or adapter) which reformulates into time units meaningful to the HA controller software
Let me make up an example. Suppose the sensor reported wind speed averages over 5 minute periods and the HA controller wanted to report it over one hour periods. The gateway could average 12 reports - for either the last 60 minutes, or for on-the-hour, as needed. Or "new rainfall" could be reported in 10 minute increments, but accumulated since midnight by the gateway, if that's what the controller prefers. Different controllers might use different accumulation periods; and what is sent to a controller could vary from what was uploaded to a cloud service. The node should not be adapting to the controller, it should send all available and useful info and the gateway or adapter should adjust - sort of like degrees F but more complicated because it might involve keeping running totals for averaging.
V_ANGLE - radians <float or int>
Here's one of those "generic vs specific" ones. Like V_PERCENT suggested above, V_ANGLE has no implications for what is being measured, only the units. It could be wind direction, it could be the plantation shutter position, it could be the solar panel tilt. How does the host know?
(This is the kind of thing that prompted my suggestions on another thread to separate the concept of units from what is being measured.)
An int for radians doesn't make much sense (unless we have a scaling factor).
I think degrees is a reasonable compromise - it's more precise than any sensor we're going to afford (or than the noise in the data itself would make moot), so it loses no meaningful accuracy. And virtually no sensor is going to report in radians. Some report in degrees. If it reports in 16 compass points or such, that can be easily converted to degrees, without even using floating point.V_WEIGHT - kg <int, float>
V_DISTANCE - Cm <int, float>
V_IMPEDANCE - OhmSee other message about intermediate calculations and the true conceptual thing being measured, (with impedance as an example). What does a distance of 240cm mean?
V_ARMED - Armed status of a security sensor. 1=Armed, 0=Bypassed
V_TRIPPED - Tripped status of a security sensor. 1=Tripped, 0=UntrippedHere the underlying concept is "V_BINARY", with a variety of associated labels for 1 and 0 based on what is being reported or controlled. On/Off, Open/Closed, Armed/Unarmed, Tripped/Untripped, Locked/Unlocked, Enabled/Disabled, etc. Do we create an ever expanding set of binary options as different variable types? Or is it functionally one variable type (or a small number with definably different dynamics from each other) with additional attributes (like the labels used for 1 and 0)?
V_SCENE_ON - Scene number <int>
V_SCENE_OFF - Scene number <int>Now we are conflating events or commands with statuses or values. We turn on a light by setting it to 1 (or 100%) and we turn it off by setting it to 0 (or 0%). This leverages the same "set value to" underlaying functionality, and allows a simple and consistent query concept (what is the current value of the light?) What is the advantage of treating a Scene differently?
Scenes are a bit confusing, because they may or may not be mutually exclusive. They are neither obviously "checkboxes" or "radio buttons".
V_HEATER_MODE - Heater mode. One of "Off", "HeatOn", "CoolOn", or "AutoChangeOver"
How do we set the themostat? What about Fan?
V_HEATER_STATUS - 1=On, 0=Off. Heater power switch
What about CoolOn mode? If it's cooling, is V_HEATER_STATUS on or off?
.V_LIGHT_LEVEL - 0-100% <int>
Is this a sensor (alternative to V_Lux) or an actuator (different than V_DIMMER/V_PERCENT above)?
V_UP - command which modes blinds ful up. Use DIMMER to specify percentage
V_DOWN - command which moved blinds to full down. Use DIMMER to specify percentageUse V_DIMMER (or V_PERCENT) instead of or in addition to this? Why not just set to 100% instead of having a "V_UP" command?
V_STOP - stop blinds in the middle of motion.
This is a bit tricky too - it's time dependent and latency becomes important, if you want to set a given position manually. So maybe it's just about canceling any previous command whereever the blinds are, eg: for safety. I'm not really sure how this is used in a script.
V_LOCK_STATUS - 1=Locker, 0 = Unlocked
Again, is this different than other binary set/query dynamics, or just different labels?
V_DUST_LEVEL - ug/m3
One of the issues I mentioned in the other thread is uncalibrated values. That comes up with some sensors. For example, the sensor's values may not be convertable to ug/m3, it might measure particle count without mass, or size without density to compute mass, or only particles within a given range. (Light is like this too, by the way, as a sensor's true output is not lux, but lux-within-a-given-wavelength-range-with_weighting, but we skip past that).
V_VOLTS - Volt <int or float>
V_AMPS - Ampere <int or float>Discussed in another message. OK if the thing we are really measuring IS voltage or current.
Anyway, this is moving forward despite my questions, and it's meeting my desire to have each of these defined more meaningfully. Thanks!
-
I see four concepts involved in describing variables types here.
-
Simplest is format, like int or float (or what size of int, signed or not). OK, that's getting separated out and is pretty well understood.
-
Another is what I'm going to call dimension - what kind of physical or abstract thing is being measured, like temperature or position or speed or particle density.
-
A related one is units.
The units usually imply the dimension, so it becomes very tempting to conflate dimension and units into just the latter. However sometimes we don't know or need to know standard units. For example, we might have an uncalibrated light dependent resistor, for which we know only two things: we get readings from 0 to 1023, and higher is brighter (or lower is brighter) - but we don't know the lux and don't have the info to compute it. In this case we DO know the dimension (brightness) and the direction. We could record values over time and graph them, we could know when it's getting brighter or dimmer and when it peaks, we could empirically set meanngful thresholds ("when value > 350 at night the lights are still on") - all without knowing the value in Lux. If we cannot report brightness in lux, should we ignore the readings,, lie about the lux, or have a concept of uncalibrated values whose demension and direction are known but whose units are not?
Also, the same units may be used for conceptually different things - like temperature, dew point, and wind chill. So units don't always imply the dimension.
- Meaning or semantics. Take angle - is it a wind direction or a servo position or what? Even something like temperature gets a bit confused - we are all familiar with air temp, but concepts like temp at 3 meters and at grould level come up. Or input and output temperature for a heating system. Or measured temperature versus thermostat setting. Or how about absolute vs relative humidity?
Now when we come up with a defined list of variable types, it's all too easy to mix and match among these concepts when deciding where to lump and where to split )(and how to name and describe). Avoiding a hodgepodge of concepts is the challenge.
Let me give a small example. Here's one way to slice things:with just dimension and units:
V_SPEED m/s V_ANGLE degreesAnd here's a different one, mixing in semantics:
V_WIND m/s V_WIND_DIRECTION "from direction" in degrees cw from true northWe currently don't have either, we have V_ANGLE and V_WIND, some from conceptual framework. Not criticising, just observing. I'm very sympathetic to how this happens.
Also notice that angle for wind also has an implicit or explicit orientation (where is 0 degrees/radians and which way increases), which is missing from the abstraction of angle.
The problem with combining semantics is that your controlled vocabulary of types can get so large.
My best solution so far is to handle dimension as a mutually exclusive type choice (choose one), and semantics as a tag or attribute (choose zero or more of many). If the semantics are not understood by a given system ("what's manifold water temperature mean, I've not been programmed to understand that?" then at least it understands the temperature dimension and the units. This doesn't make it all easy, but it seems to make it more tractable than just having one mutually exclusive list for to represent all aspects (except format #1 above)
-
-
@Zeph Most of the concepts and ideas you touch have already been mentioned in the posts above... Did you read them?
-
Ok, I've had a solutions bubbling in my head over night. It will add one byte in the MySensors Req/Set Header but solves a lot of the discussions we've had earlier in other threads as well. Its also replaces both INTERNAL and PRESENTATION command.
--- Moved to the first post ---
-
@hek said:
Where does this fit?
V_DEW_POINT - Degrees Celsius <int or float>Either it doesn't fit in (let the controller calculate it) or place under S_HUMIDITY.
I'm not completely convinced we need to store it this way and use another byte for each req/set message (which are most common...). I'll let it rest for a while so I might diss your solution later
-
Could someone please explain to me why we have S_ types at all? I really dont get it.. I Only think it makes things more complicated. (my conclusion - I dont like it)
-
@Damme said:
Could someone please explain to me why we have S_ types at all? I really dont get it.. I Only think it makes things more complicated.
Read my post again and explain how you would distinguish a thermometer from a uv sensor on the controller side without it?
-
@hek said:
distinguish a thermometer from a uv sensor on the controller side
IMHO this all depends on the controller used, as I already explained. I'm not familiar with Vera, but e.g. for OpenHAB/MQTT the actual type of a sensor is defined by the OpenHAB configuration. It just receives a value and you need to tell it that node x sensor y is a temperature sensor.
Possibly Vera and other systems are more of a plug-and-play kind and are able to add a sensor depending on its type automatically.I (for a long time) agree with @Damme but I can imagine there are systems that do require this information.
[dissing start] Did you think about sending the sensor type only once, durig presentation? There really is no need to send it with each set/req message as it is static for the duration of the connection. For Vera and the like this probably means you should buffer these values in the gateway... [dissing end]
-
@hek I've been thinking and thinking and even might have an other solution.. but I'll wait with that.
so every piece of data has a S_type and V_type.. V_ must be a fixed unique table (not starting from 0 on every S_type)
I'm more acceptable now than 10 minutes ago..
-
@Yveaux said:
[dissing start] Did you think about sending the sensor type only once, durig presentation? There really is no need to send it with each set/req message as it is static for the duration of the connection. For Vera and the like this probably means you should buffer these values in the gateway... [dissing end]
I think for the controllers view it would be best for the node to send every message with sensor id, sensor type, value type and payload. Noone needs to remember anything
The way back to the node the rest can be omitted, as long as sensor id is there.
-
@Yveaux said:
[dissing start] Did you think about sending the sensor type only once, durig presentation? There really is no need to send it with each set/req message as it is static for the duration of the connection. For Vera and the like this probably means you should buffer these values in the gateway... [dissing end]
Yes, it would save a byte. But it could be very useful for a simpler controller (which can determine sensor type from each and every message) and when displaying sniffing data ;).
This would only leave 3 COMMAND types left. REQ, SET and STREAM. Where STREAM-data probably could be handled by S_NODE (internal). This means just 1 bit is needed for a SET/REQ flag. But @ToSa would have to feedback on this.
-
@hek S_node could include stuff as
Reboot
ClearEEprom
Read EEprom byte
Set EEprom byte
besides stream data types.
(to be discussed if any of them are useful or not and if they should be hardcoded or not)
-
@hek said:
when displaying sniffing data
Yeay yeah, blame it on the sniffer
Making stuff 'easier' to sniff can never be a reason to always send one extra byte...This means just 1 bit is needed for a SET/REQ flag
Looks like the protocol discussion is shifting towards this topic. Better not reserve a bit to indicate in a set/req message which of the two it is, but use different message indicator values. The actual message structures can still be the same.
-
@Yveaux said:
Yeay yeah, blame it on the sniffer
Making stuff 'easier' to sniff can never be a reason to always send one extra byte... We're talking about a total of just 8 bytes header here. And if we remove routing just 5 bytes brimful with useful information about the payload.Better not reserve a bit to indicate in a set/req message which of the two it is, but use different message indicator values. The actual message structures can still be the same.
I do not understand this.
-
@hek said:
Better not reserve a bit to indicate in a set/req message which of the two it is, but use different message indicator values. The actual message structures can still be the same.
I do not understand this.Probably because I didn't understand you then...
I interpret your post as using a single message structure for either set or req messages. A single bit is used to distinguish between the two, right?
-
-
@hek What I meant was to just have two separate commands for set & req (like it is now in 1.3 & 1.4b) but use the same messgae structure to send these messages. The command value will be used to differentiate between set & req.
No need to reserve a bit in the message this way.
Hope its clear now....
-
@hek
First off - I think this taxonomy of sensactuator values is getting better each time, and I appreciate your mixture of creativity and receptivity as the leader in this effort. Good work.This is not dissing, more like reflecting on what I see more than making judgements.
The S_ and V_ enums (controlled vocabularies) are becoming relatively more orthogonal rather than confusing people by appearing to cover similar conceptual territory twice but in different way, as the earlier versions inherited from Vera tended to do. So for example, S_TEMPERATURE has V_LEVEL rather than V_TEMPERATURE.
One result is that to understand what a transmitted value (number) represents, you would now need to look at both the S_ code and the V_code (as a good thing). The "dimension" and "units" concepts for understanding a V_LEVEL value depend on the S_ code associated with the variable. In a sense there would now be a two-byte combined expansion of the concepts often formerly described in the one byte V_ code. (So instead of V_TEMPERATURE, there is V_LEVEL+S_TEMPERATURE and you have to look at both to see that the number is temperature in degrees C).I think this is getting cleaner.
The semantics or meaning of a variable are still somewhat distributed between the S and V codes. For example V_LEVEL and V_ANGLE have very little semantic content, relying almost entirely on the S code to provide that, but V_ARMED vs V_TRIPPED do have semantic content. This is just an observation; I'm still letting it sink in.
A big question is the relationship of node children with S_ code - both are called sensors but they need not be the same thing. For this discussion a "Child" is an addressable subunit of a node, numbered 0,1,2... and each child can contain a set of variables. An S code is a one byte code associated with each variable (along with a V code). In an earlier discussion where I was questioning the value of the "Sensor" level of the heirarchy you made the point that there could be value in grouping several variables that are logically tied to "the same thing". The Child concept does that, it provides an addressable container for one or more variables.
At issue is whether there is a fixed relationship beween the Child and the S_ code, whether all variables within the same Child container must have the same S_ code or not.
-
If all variables in a given Child must have the same S_ code, then as somebody else pointed out this could be sent to the hub once during node integration and stored there (associated with the given node/child), rather than being transmitted with each value. But you cannot include variables for temperature and humidity in the same Child, even if logically that's what you want to convey (say you have two DHT-11's on the same node, you can't just make them two Children each with a temp and humidity).
-
If each variable within a given Child can have its own S code, then the S code and V code become a two byte descriptor of the variable, and Child is a flexible container used to represent "these variables go together" and only that.
Which of these relationships between a Child (addressable subunit of a node) and S code (one byte code that along with V code helps interpret a given variable) did you have in mind?
-
-
@hek All these changes are more 2.0 than 1.4..
Edit;
Btw, All these extra bytes S_ and V_ to describe a payload is good for mqtt also.
And this is why we need some more command types (i.e. Stream for OTA update etc. more space for payload smaller header)
-
I see the edited version with MIN, MAX and RESET's added. This takes the orthogonality of V and S codes even further, and it seems like a good direction.
And it's good to see a start on better addressing one of the complicated aspects of this framework = accumulation periods (for Max/Min/Average/Total).
But is V_RESET a variable or a command, conceptually? Is is something sent to the node (probably) or reported by the hub (probably not)?
This brings up another broad conceptual issue in creating a taxonomny like this - the difference between values and command/events. Many software frameworks have both concepts, for example class member variables and class funtions/events. (In electrical terms. this is somewhat like the "Level" vs "Edge or Trigger" distinction).
In the current taxonomy, command/event and value are kind of intermingled. I think it will be important to document which is which.
There are a couple of approaches. One is to define a list of commands (eg; C_RESET) distince from values (V_LEVEL); the other is to have a well defined way of using variables as pseudo-commands.
For one possible example of the latter, one could say that for any variable defined as a command variable:
- when the variable goes from 0 to 1, the node will take the defined action
- the hub will send the node a "Set" command to set the value to 1; if it was 0, then the action is taken
- Setting to 0 has no effect
- the node will set the value to 0 when the action is completed (may be immediate)
- the hub can query the current value of the variable to see if there is any action still pending or not
(This might be exactly what you had in mind, or you might have had a variation in mind - I'm just saying that it should be clearly defined and documented).
Then we could say that V_RESET is a command variable and thus shares the standard dynamics, where the specific "action" is to restart the accumulation period. And there could be other command variables, each documented as such.
(The Event concept is similar to a command in that it's a transition rather than a level, but used as reports from the node to the hub. Not addressed yet)
-
One characteristic of the MySensors system is that nodes are not systematically aware of time. That's good in that it simplifies the node firmware, and it's a limitation in terms of dealing with accumulation periods.
I'm speaking of "accumulation" periods rather than "reporting" periods, because a node may "report" many times in an accumulation period, for example every 10 seconds it may report the accumulated rainfall since midnight.
And there are several kinds of accumulation periods: in particular rolling and fixed start. Fixed start might report accumulation (sum, average, min, max) since a given time like midnight. Rolling might report the last hour or last 5 minutes from now. (I mentioned a rain sensor which does it's own "rain event" splitting based on no-rain intervals).
In some cases, accumulation periods may not be under our control - for example reading an integrated weather station. At best a node could report the accumulation period.
In other cases, we can control the accumulation period, for example the node's sketch could itself count KWH or rain, or it could average temperature or speed or direction, taking readings every N seconds.
One more concept: how to handle a Reset. The simplest approach for a hub-commanded reset would be for the hub to be sure it has all desired recent value reports before sending a Reset command (so the time gap between last report and Reset is short enough that nothing signficant was missed, eg: mm of rain). A more synchronous approach would be for the node to send one last set of accumulated values immediately before zeroing it's counter.
But what about a manually triggered reset (button pushed)? The sensor node could in that case want to let the hub know that it happened.
One way to unify these in a simple and consistent way would be to report V_ACCUM_TIME - the number of seconds since the accumulation period began. That's the period over which values were averaged, summed, or min/max'd. The time would increase until a reset, go to zero and increase again. This handles controllable and not controllable accumulation periods, fixed and rolling accumulation periods, hub triggered and manually triggered resets.
If an accumulation period is reset, the node could potentially send a last report with the ending V_ACCUM_TIME before the reset and then the following report would have time since starting the new period.
If the time is just based on millis() then with the Arduino Pro Micro the time accuracy would be nominally about 0.5% (they have ceramic resonators rather than crystals and that's a typical spec). That's probably good enough, tho it's possible to do better if it mattered.
Where this seems to be going is that any quantity variable could potentially be expanded into an accumulation variable - which means that in addition to (say) V_LEVEL, there is V_LEVEL_MAX, V_LEVEL_MIN, V_LEVEL_TIME, and either V_LEVEL_AVG or V_LEVEL_SUM. This "accumulation enhancement" could optionally be added to temp, humidity, wind, power, fluid/gas flow, whatever - in a defined and consistent way.
-
FYI: Sleeping sensors (who also can be woken by a interrupt on a pin) will have a hard time keeping time without clock module or by fetching time from server at each wakeup from pin trigger.
Good idea about a STATUS for RAIN sensor. It will be added and indicates if rain is ongoing.
Will also change to ACCUMULATED for all accumulated values. Average for these makes no sense.So.. I see now that there is need for more than one reset command. We'll need RESET_ACCUMELATED, RESET_LEVEL, RESET_PERCENTAGE. They should also have their respective MAX, MIN, AVERAGE. Could you please give me a use case for V_LEVEL_SUM.
Just to clarify for anyone getting a bit scared of all these new variables. Using them for your own sensor is of course optional!
I feel it's time to move the new stuff to the first post to keep it in focus.
-
I dont know why we have to limit some S_type to a number of V_types. Why not just let a S_type have 'all' V_types... User can be more flexible
If I wanted node could send
S_MOTION V_TRIPPED 1 (motion tripped)
S_MOTION V_ARMED 0 (no longer armed)
S_MOTION V_LEVEL 24 (motion quality 24 of 255) (probobly animal)controller decides S_MOTION V_RESET 1
(order not thought of at all, needs to be fixed)
#DEFINE S_DOOR 1 #DEFINE S_MOTION 2 #DEFINE S_SMOKE 3 #DEFINE S_BINARY 4 #DEFINE S_DIMMABLE 5 #DEFINE S_WINDOW_COVER 6 #DEFINE S_THERMOMETER 7 #DEFINE S_HUMIDITY 8 #DEFINE S_BAROMETER 9 #DEFINE S_WIND 10 #DEFINE S_RAIN 11 #DEFINE S_UV 12 #DEFINE S_WEIGHT_SCALE 13 #DEFINE S_POWER 14 #DEFINE S_HEATER 15 #DEFINE S_DISTANCE 16 #DEFINE S_LIGHT_SENSOR 17 #DEFINE S_NODE 18 #DEFINE S_LOCK 19 #DEFINE S_IR 20 #DEFINE S_WATER_METER 21 #DEFINE S_AIR_QUALITY 22 #DEFINE S_CUSTOM 23 #DEFINE S_DUST 24 #DEFINE S_PH 25 #DEFINE S_SCENE_CONTROLLER 26 #DEFINE S_NODE 255 #DEFINE V_CONFIG1 1 #DEFINE V_CONFIG2 2 #DEFINE V_CONFIG3 3 #DEFINE V_CONFIG4 4 #DEFINE V_CONFIG5 5 #DEFINE V_VAR1 6 #DEFINE V_VAR2 7 #DEFINE V_VAR3 8 #DEFINE V_VAR4 9 #DEFINE V_VAR5 10 #DEFINE V_ARMED 11 #DEFINE V_STATUS 12 #DEFINE V_WATT 13 #DEFINE V_PERCENTAGE 14 #DEFINE V_STOP 15 #DEFINE V_LEVEL 16 #DEFINE V_MAX 17 #DEFINE V_MIN 18 #DEFINE V_RESET 19 #DEFINE V_DEW_POINT 20 #DEFINE V_MODE 21 #DEFINE V_ANGLE 22 #DEFINE V_RATE 23 #DEFINE V_VOLTS 24 #DEFINE V_AMPS 25 #DEFINE V_PRESENTATION 26 #DEFINE V_BATTERY_LEVEL 27 #DEFINE V_RESET 28 #DEFINE V_TIME 29 #DEFINE V_ID 30 #DEFINE V_LOG_MESSAGE 31 #DEFINE V_SKETCH_NAME 32 #DEFINE V_SKETCH_VERSION 33 #DEFINE V_FIND_PARENT 34 #DEFINE V_CHILDREN 35 #DEFINE V_VERSION 36 #DEFINE V_INCLUSION_MODE 37 #DEFINE V_GATEWAY_READY 38 #DEFINE V_STATUS 39 #DEFINE V_IR_SEND 40 #DEFINE V_IR_RECEIVE 41
-
@hek said:
FYI: Sleeping sensors (who also can be woken by a interrupt on a pin) will have a hard time keeping time without clock module or by fetching time from server at each wakeup from pin trigger.
Sleeping nodes may have a hard time averaging in some cases too.. But your point is well taken and it is a real concern.
One of the sleep libraries calibrate the RC watchdog timer against the main clock, so it can make a good guess for adjusting milliseconds upon each wakeup. That could help for a simple sleep, but not for waking up from an interrupt.
So the problem child is a node for which all the following apply:
- sleeps
- is awoken by interrupts at unpredictable intervals
- has no local RTC
- has accumulated variables
One option would just be to ignore the accumulated time feature for the subset of nodes in that category; for them, you have to fall back to where would be be if I had never spoken up about tracking time. For other nodes, the tracking & reporting of accumulation time can be an enhancement, as described in an earlier message.
But suppose we DO want to go further and determine accumulation time for all nodes, even the above.
(Sigh. I acknowledged that there was a simplicity advantage to not dealing with time, and that just using the local node's millis() was the simplest but not only way to handle time; and that does handle most cases but not the above)
System Time approach 1: Master Clock Broadcast to nodes
The hub will have the master clock, using its millis(). Periodically it will broadcast the value of this to all nodes. A node could also request a time be sent. We could apply a crude delta to adjust this according to number of hops and typical transmission time, or not - we don't need anywhere close to millisecond accuracy. Each node would maintain a delta between its own millisecond clock and the broadcast millis and adjust that every time it receives that broadcast. (Smart nodes could even use a computed time scaling factor to adjust for differences in their clock vs the master). When for any reason has a need to estimate the current time (including but not limited to the case of having been powered up from sleeping by an interrupt AND needing to repost accumulations with more precision than it can estimate), the node can either wait for the next broadcast, or request one.
System Time approach 2: Track accumulation times in the hub
Whenever an accumulation period is started, the node will send a message to the hub to that effect (telling it which variable is involved and an incrementing reset count). This could be triggered by a commanded or manual reset, or by the automatic reset of an external weather station or service, or whatever. Then it would send the reset count (V_RESET_COUNT) rather than the accumulated time along with accumulation based values. The hub can look at its current time, and at the recorded time of the last reported reset of that variable, to compute for itself the elapsed time. The reset count makes sure both sides are sync'd as to which reset they are talking about. Both the "Starting accumulation period #n for (variable)" and the "reporting accumulated value for (variable)" messages travel from node to hub, and their travel times will roughly cancel out, at least well enough.
The former approach is more powerful. It better supports rolling averages. And it would provide a flexible "MySensors network time" framework for future features, like reporting event times or periods more accurately.
For example, I've considered putting two motion detectors at angles and with masks such that one could usually detect which way somebody went through a doorway by the trigger times, or at different place in a hallway. I'm not proposing to support that directly at this time, but when it comes up, if each node had the option of knowing the hub's master clock time to within tens of milliseconds or better, that could be a useful base upon which to build.
[Total aside. My ambition for another nRF + Arduino system is more precise than this, possibly even enough to do PWM of 60 Hz mains powered lights without a local zero crossing detector - not yet determined. That's why I connect the nRF24L01+'s IRQ output to Arduino pin 8 = ICP1, even if I'm not using interrupt driven nRF code. If the nRF's receive interrupt is enabled, this allows the ATMega328p to get a very accurate time of reception using Timer 1 if it wants, even in polling mode without interrupts. And if I ever do want to interrupt the ATMega from the nRF, I can use the capture interrupt, while leaving INT0 and INT1 on pins 3 & 4 for other uses. The above first option is a smaller, simpler version of that design, adapted for the needs of a sensor network]
-
@hek said:
Could you please give me a use case for V_LEVEL_SUM.
-
"snapshot in time" values like temperature and humidity and watts logically accumulate as (averages. min, and max) over the given period as their end product. Tuesday the average temperature was 22.3 degrees.
-
"incrementing total" values like KWH, liquid flow, and rainfall logically accumulate as total or (sum) over the given period as their primary end product: Tuesday a total of 20mm of rain fell. (IFF you know the time as well, you can calculate an average as a secondary product)
These two types of values have somewhat different dynamics.
Of course if you report a total AND an elapsed time, the hub can calculate averages for itself, but not min and max. This calculation can be done at the hub even for temperature, tho total degrees C is not meaningful and total mm rain is.
So IF we can rely on S_LEVEL_ELAPSED and S_LEVEL_SUM both always being available (and the latter not overflowing integer formats), we could omit S_LEVEL_AVG. But in the original proposal without a time framework, the node should report S_LEVEL_SUM for rain and S_LEVEL_AVG for temp.
-
-
EDIT - there are some good concepts in the next few posts from me (IMHO obviously), but some may turn out to be sidestepped by a more radical change described in response
http://forum.mysensors.org/topic/304/unit-standardization-1-4/57,
so you might want to just read that first.
@hek said:
###S_WIND - Wind sensor
V_LEVEL - Wind level in m/s (average wind speed during last report period)
V_LEVEL_MIN - Min wind level
V_LEVEL_MAX - Max wind level (gust)
V_LEVEL_AVERAGE - Average wind level-
V_LEVEL_RESET - Reset max/min value
V_ANGLE - radians <float or int>First, i still want to convince you that V_ANGLE in "degrees clockwise from true north" is the proper measure here, not radians. (and you need to define if this points to where the wind comes from or where it's moving towards, so is a west wind 90 degrees or 270 degrees - or translate to radians if you want to see ugly)
Look, I'm a math person (or was) and I totally get that e, national logs and exponents, pi and radians are more fundamentally defined values than degrees or using base 10 for decibels or logs. But the point is not scientific purity (which are are not really close to anyway) but pragmatic utility in the field of sensor networks. And degrees simply work better in this context of conveying direction, especially if you allow int representation. Enough on that for now.
Second, notice that you really only have two real measurements here: speed and direction. Those are what I call snapshot values - if you sample them at a given time, you get a value, which can later rise or fall.
The former has additional derived or calculated sub-values, and those subvalues follow a pattern representing "simple time accumulation statistics" and also common to many other snapshot variable types, like humidity.
The thing is - direction is also a snapshot value. If you are going to report average wind over the past hour, you can also report average direction.
Alas, angle differs from continuum measurements in that it wraps, so average/min/max have to be computed carefully and are only meaningful if the source stays within a reasonable range; and sum makes no sense.
But any continuum variable is a potential candidate for accumulation-period based derived sub-variables. I picture that there will be a single "accumulation" library which can be applied to any int or float variable, rather than that the temperature library will have accumulation code and the humidity library will have its own accumulation code.
So in the hierarchy I'd put V_LEVEL_MIN, etc one level indented under V_LEVEL rather than make it look like a parallel and independent measurement.
Let's also be clear that there are two reasons to have accumulation period based derived variables. One is that the node could be getting values from a device which does its own accumulation of some sort (eg: a weather station or some rain gauges). The other is that the device may sample the instantaneous variables more often than it transmits them. If we transmitted every measurement, the hub could potentially do its own min/max/average/sum calculations. No big deal there, but the hub could also accumulate by multiple arbitrary periods, like hour of the day AND since sunrise AND past 10 minutes AND past hour AND since Oct 1st (which is the start of the water year as used by scientists around here). So the most flexible thing would be to report every measurement to the hub and let it handle aggregation any way (as many ways) as it wants. But that could be bandwidth heavy, or might impose too much work on simple hubs (or controllers they report to).
So for one or more of these reasons, sometimes it may make sense to have the sketch running in a node handle accumulation sub-variables. If it reports often enough in a well considered format, the hub or controller or cloud based analytical software can still do aggregation over some of these alternative time periods.
-
@hek said:
###S_BINARY - Light (or other) on/off actuator
V_STATUS - 1 - turn on, 0 = turn off###S_DIMMABLE - Dimmable actuator
V_STATUS - 1 - turn on, 0 = turn off
V_PERCENTAGE - Dimmer level 0-100%OK, let's define what happens if you get more than one type of command (probably just making explicit what we are thinking anyway).
S_DIMMABLE:
V_STATUS = 1 is the same as V_PERCENTAGE = 100, either has same effect
V_STATUS = 0 is the same as V_PERCENTAGE = 0, either has same effectIf you set V_STATUS = 1 and then query V_PERCENTAGE it should return 100% (or 0 => 0%)
If you set V_PERCENTAGE = 100 and query V_STATUS it should return 1OK?
The only question is setting V_PERCENTAGE = 1..99 and querying V_STATUS.
Option 1: Return -1 for "neither fully on nor fully off"
Option 2: Return 1 for "at least partially on" (ie: anything above 0%)
OK, what about a Fan with setting Off, Low and High, and another fan with settings "Off, Low, Medium and High?
We could map those to text "OFF, LOW, MED, HIGH"
Or we could map them to 0,1,2 where 2=HIGH for the first fan, and 0,1,2,3 where 3 = HIGH for the secondI'm going to suggest that we primarily map to V_PERCENTAGE, but perhaps also support the latter option as an expansion of binary 0..1 into multi step value 0..N_STEPS.
All fans: OFF = 0% HIGH = 100% Fan 1: Low = 50% Fan 2: Low = 33% Med = 67%If you are sent a V_PERCENTAGE, set the fan to the closest speed to the given percentage. 100% is always the highest speed, 0% is always OFF, in between will approximate. This extends to 4 level fans. And it works for a heater or motor with multiple speeds as well. If you query V_PERCENTAGE the actual value chosen from the list will be returned, not the requested value (ie: requesting 45% but reporting 50% in the case of fan 1).
The above intermediate percentages are the default. However the node is empowered to set other values. We all know of fans or other devices where the speeds are not evenly spaced. I have a heater which has 600, 900 and 1500 watt settings (both elements). The Low and Medium setpoints could be 40% and 60%. Otherwise it works as described above.
And there could be an integer V_SET_STEP (or something), corresponding to V_BINARY* but ranging from 0 to N_STEPS where N_STEPS is per-variable defined. I'm not attached to this, but it's an option, It would work in the obvious way. So V_SET_STEP of 1 for Fan1 would set it to Low=50% and V_SET_STEP of 1 for fan2 would set it to Low=33% (and V_SET_STEP of 1 for the heater example would set it to Low=40%. SET_STEP of 0 is always off, and SET_STEP of N_STEPS is always HIGH or 100%
For completeness, I will mention that there's another way to define this for V_DIMMABLE. It could be required to memorize the last set percentage, and then V_STATUS = 0 means turn to 0% but remember last non-zero percentat, and V_STATUS = 1 could mean return to last non-zero percentage.
Tho I don't favor this dynamcic, it too could work. However I strongly favor defining which way we expect things to work rather than leaving it ambiguous!
-
@Damme said:
I dont know why we have to limit some S_type to a number of V_types. Why not just let a S_type have 'all' V_types... User can be more flexible
As described at the time I write, the proposed taxonomy of values isn't really as heirarchical as it's presented.
You can also parse the meaning from V code to S code:Ah, we have a V_LEVEL value, that means it represents a one dimensions continuum But what does that continuum represents? Oh, I see the variable is also tagged with S_TEMPERATURE That means the continuum is temperature in degrees CThat is, the V code tells you something of the structural concept, and the S code adds some degree of semantics (like dimension and unit).
So you could also invert the displayed hierarchy and show which S codes are available for a given V code
-
Hold the presses.
I've happily engaged in the collaborative process of cleaning up the variable taxonomies of V 1.4b here.
But we keep coming to conceptual tricky parts. Some of that is inherent, but in thinking deeper about it, i've come to the conclusion that a good part of what makes it hard to resolve cleanly is that we are trying to cram too much information into too few bits of variable metadata which has to be included in every packet.
So rather than reforming S and V codes, I'm thinking a more radical approach is needed, to make the system both more flexible and simpler. Part of that comes from identifying what metadata about a variable is static and what needs to be dynamic, and then moving the static part out of the normal OTA operations. That turns out to make the taxonomy much more tractable and flexible while simplifying the node firmware.
Guess what - even child ID turns out to be metadata in this approach, rather than addressing.
Here's the sketch of the new approach.
http://forum.mysensors.org/topic/308/another-way-of-organizing-variables
It's sufficiently different that I expect it may encounter resistance - from the natural and appropriate fear of breaking too much (I share that concern). But it just may turn out to be sufficiently attractive to be worth it. Have a look. And now would be the time to consider such a break - with RadioHead and OTA programming support also in the air.
-
@hek said:
This would only leave 3 COMMAND types left. REQ, SET and STREAM. Where STREAM-data probably could be handled by S_NODE (internal). This means just 1 bit is needed for a SET/REQ flag. But @ToSa would have to feedback on this.
I switched the bootloader code between using stream and using simple internal messages - at the end doesn't matter as long as it's simple enough to dis-samble and assemble a packet without a lot of additional code overhead.
-
Sorry, forget I brought this up. We'll keep the STREAM command as-is. Skyped you about it yesterday
-
@Zeph said:
But suppose we DO want to go further and determine accumulation time for all nodes, even the above.
...
System Time approach 1: Master Clock Broadcast to nodes
...
System Time approach 2: Track accumulation times in the hubI'm in favor of approach 1 as well - and an optional addition to the MySensors class to keep track of time. Only specific sensor types would use this feature - interesting mainly for actuators rather than sensors:
Let's assume you use a relay to turn on/off your garden watering while you are on vacation. The network breaks (controller or gateway or whatever node needed to relay messages).
- If the sensor node itself knows about the time and the schedule for watering, it can turn on and off the watering itself. If there is an issue with the communication, the time would not be 100% accurate over time but still good enough.
- If the sensor node relies completely on the controller for time and (worst case) the communication breaks just between the start watering and stop watering command you can imagine what happens...
Knowing about the time heavily reduces the single points of failure. The protocol already covers the "request time" communication so this should be relatively easy to implement...
- controller sends a time broadcast every e.g. 15 minutes
- only the nodes interested in time care about it and update their internal time
- when a node needs the current time it calculates it based on the last timestamp received, the millis() when it was received and the current millis...
Doesn't solve the sleep issue (as long as you don't want to submit a separate "get time request" every time you wake up) but at least for sensors that are not battery powered (relay nodes in most cases are not) it's a feasible approach.
This is probably worth a separate thread
-
Yes, and to extend this even further it would be great to be able to distribute/push simple rules all the way out to the nodes. It might be enough with simple scheduler-commands to start with but this could easily support even advances scenarios. The question is where to set the cuttingpoint...
The most extreme solution would be to allow a graphical rule engine on the controller which automatically generates a sketch, compiles it and sends it out OTA to the node.
Definitely worth discussing in a new thread...
-
@hek said:
Yes, and to extend this even further it would be great to be able to distribute/push simple rules all the way out to the nodes. It might be enough with simple scheduler-commands to start with but this could easily support even advances scenarios. The question is where to set the cuttingpoint...
For the level of protocol under current discussion, the cuttingpoint could just be giving nodes an optional sense of time (using broadcast packets to keep in sync with the master as needed). And making use of that time as part of the (optional) accumulation enhancement for some variables.
What else they do with that time at higher layers would continue to evolve in the future. We don't have to bite it all off now.
-
V_R_PERCENTAGE - Red component % <int>
V_G_PERCENTAGE - Green component % <int>
V_B_PERCENTAGE - Blue component % <int>
V_W_PERCENTAGE - White component % <int>Just a hundred steps might be to low, especially for green in the lower end which the eye sees much brighter.
Please consider floats or per thousand instead.
-
Ok! Didn't know that. But couldn't the sensor code adjust this with some sort of color curve?
-
@hek
That would probably do it, however one would need to be defined and standardized so that all behaves the same. One more clever than me needs to propose how the curve would look tough.
-
S_RGBW - RGBW Light
V_STATUS - 1 - turn on, 0 = turn off
V_R_PERCENTAGE - Red component % <int>
V_G_PERCENTAGE - Green component % <int>
V_B_PERCENTAGE - Blue component % <int>
V_W_PERCENTAGE - White component % <int>Ain't it better to split up S_RGBW to two different types. W is already present in full R, G, and B. For example i have implemented Philips Hue and there is a difference in fixture types some do RGB some only W. If an user would implement such with mysensors the same option can be possible. An S_RGBW declaration for a W only would indicate RGB capabilities and maybe cause confusion.
Just a hundred steps might be to low, especially for green in the lower end which the eye sees much brighter.
Agree, but thousands could be too much (which never is the case of course), maybe 0-255 would be sufficient and widely used (all though not as precise in steps as with thousands)? Also the W_PERCENTAGE should be left out because this is full R,G,B or similar calibrated R,G,B levels (which is quite difficult to do at every possible level) with the end result it then in fact is a brightness level instead of a white component.
An addition could be S_HSB/S_HSV with float,float,float (0.0-360.0, 0.0-100.0, 0.0-100.0)? Also i think the color presentation should be or calibrated on the device or in the controller. Even some of those Philips Hue's fixtures are often controller side calibrated (better said color index modified after color selection)....
-
I actually have RGBW led strips mounted around our house. W is indeed its own component (Warm White in my case). So it will stick
(Warm White using the special W-component)
Cold white (RGB activated)
Crazy
-
But switching to LEVEL and using 0-255 would be ok (if we need it ;)).
-
@hek
Ahhh... understood :)... Splitting those up will not be an option then because it could then create a numerous amount of options....A S_HSB would be nice though, even if it would only be ints....:
V_STATUS - 1 - turn on, 0 = turn off
V_H_DEGREE - Degree (0-360)<int>
V_S_PERCENTAGE - Saturation % <int>
V_B_PERCENTAGE/V_V_PERCENTAGE - Brightness/Value % <int>
V_WATT - Watt <int>
V_WATT_MAX - Max watt value
V_WATT_MIN - Min watt value
V_WATT_AVERAGE - Min watt value
V_WATT_RESET - Reset max/min valueI must say it is a personal request :$....
[Edit]
But switching to LEVEL and using 0-255 would be ok (if we need it
).It would create space to add the HSB/HSV option
-
What's wrong with setting lights at 1.475% Is percentage limited to integer?
Let's be very explicit about that. Is the idea that all values accept only integers unless they say they have the option of using floats? If so that should be made clear in the list.
Another thing that needs to be more clear is how to handle multiple V codes for the same sensacturator (like the RGB or HSV lights).
- Is a sensor expected to report all of them?
- Is an actuator expected to respond to all of them?
- How do both sides stay in sync about subsetting?
By the way, there is a distinction to be made between RGB lights (three blendable primaries able to synthesize many colors in a predictable and fairly standard way) and multichannel control of N lights of arbitrary colors. RGBW is another thing entirely. The W could be cool or warm white. For that matter the fourth color could be yellow or violet. It's not a color space.
If the RGB channels blend I'd implement it as an RGB light plus a monochrome light, because there is no RGBW color space. If they don't blend I'd implement it as 4 monochrome channels, the same way I'd implement a Yellow and Green pair that didn't blend.
-
What's wrong with setting lights at 1.475% Is percentage limited to integer?
It just not represents any absolute bit value for a specific color, it is an approximate. If for example you have a fixture which does 255 steps and you want to set it at step 40. you will need to send 15.682.....%. But what if you would send 15.770... ? Due to the nature of floats it could be rounded down or up to 40 or 41 and especially in the lower steps of a color the difference would be seen if this is a mix of on of the RGB values, and even more as all the RGB values are send in percentages. A percentage would only be usable (But even then it's still an approximate) with 32 bit or analogue blend RGB fixtures. There ain't a lot of these.
@zeph said:
If the RGB channels blend I'd implement it as an RGB light plus a monochrome light, because there is no RGBW color space. If they don't blend I'd implement it as 4 monochrome channels, the same way I'd implement a Yellow and Green pair that didn't blend.
I do agree with you, and you're totally right, but:
@john said:
Splitting those up will not be an option then because it could then create a numerous amount of options....
In other words it could eat up the possible S_TYPE's
@hek
Please disregard this one, it wouldn't :"it would create space to add the HSB/HSV option".
But, couldn't it be an option to add the HSB/HSV variables to the S_RGBW? because if the presentation stays you would only have to send the vars that are supported, and the type S_RGBW could be renamed to something like S_LIGHT_COLOR?
-
The question is where the actual calculation is done between RGB and HSB/HSV. Should it be done on the Arduino side or controller? Nothing stops you from making a fancy HSB/HSV GUI on your controller but the actual values send to the actuator is RGB. But my knowledge of color scales is a bit limited and I'm probably missing something?
-
It depends, if the HSB scale is purely in integers you will have about 3.600.000 options (including brightness steps) which is much less then 16 bit 0,0,0 - 255,255,255 scale. But this scale ain't specific in int's it is an implementation chosen scale. If choosing for ints, yeah keep the RGB, if going to floats, add it next to the RGB.
probably missing something?
The main difference is that with pure 16 bit you will have whole bit numbers and with degrees there is more then just go from 359 to 360/0. you could use 359,00 to 359,99 which gives an extra 990.000 extra red combinations (including saturation and brightness) with absolute 16 bit you stick with whole ints/bytes limiting the possible fixtures light high "resolution" capabilities.
Also because of the nature of HSB with higher color resolution possibilities you will always have the possibility to find the closest 16 bit neighbor or even the exact bit value because of higher to lower "spectrum" resolution.
With the above if you would leave out the brightness you would have a higher color resolution 255,254,255 is one step lower green form maximum, you can't go to 254.9 right? in degrees i think green is 120, you could go to 119.99999. because the degrees are not fixed to a whole as a byte is.
Because of more steps calibrating un-calibrated fixtures/leds (for example with the green mentioned earlier) can be more precise. Also HSB is more human perspective then fixed bits.
Maybe the above shines a light on the difference?
RGB is just a brightness setting of a specific color, HSB is pure color selection, intensity setting, and the brightness of these.
Personally:
HSB is nice to have, but from personal needs i have enough convertion functions (hsb->rgb and vice, hsb -> kelvin, etc...) to convert.Should it be done on the Arduino side or controller?
I would let the controller do the actual conversions and let the device spec it's capabilities.
-
@hek said:
S_RGBW - RGBW Light
V_STATUS - 1 - turn on, 0 = turn off
V_R_PERCENTAGE - Red component % <int>
V_G_PERCENTAGE - Green component % <int>
V_B_PERCENTAGE - Blue component % <int>
V_W_PERCENTAGE - White component % <int>
V_WATT - Watt <int>
V_WATT_MAX - Max watt value
V_WATT_MIN - Min watt value
V_WATT_AVERAGE - Min watt value
V_WATT_RESET - Reset max/min valuesOne of the areas where the spec is very unclear is when a variable is writable and where it's only reportable (read only).
For example, can I set the watts to 20 above (or for S_DIMMABLE) in order to tell the light how much power I want it to use? Seems like a valuable option. How am I supposed to know?
What if we made the write/report/trigger function explicit:
S_RGBW - RGBW Light
W_STATUS - 1 - turn on (100%), 0 = turn off (0%) W_R_PERCENTAGE - Red component % <int> W_G_PERCENTAGE - Green component % <int> W_B_PERCENTAGE - Blue component % <int> W_W_PERCENTAGE - White component % <int> R_WATT - Watt <int> - current instantaneous or short term average value R_WATT_MAX - Max watt value since last reset R_WATT_MIN - Min watt value since last reset R_WATT_AVERAGE - Min watt value since last reset T_WATT_RESET - Reset max/min/avg valueW variables can be written to; the current value can also be queried
R variables can only be reported (automatically or by query)
T variables are triggers, writing to them triggers some actionThis gives guidance to both the sketch writer for the node, as well the controller adapter writer in knowing what they can do with each variable a node presents.
The actual numeric values for R, W and T codes could be interspersed or non-overlapping ranges could be reserved for each (eg: 1-100 for R, 101-200 for W and 201-255 for T)
-
@Zeph it all depends on how skilled programmer and how nerdy you are. if you want to set a lamp power with V_WATT great, thats nerdy! but in not, just dont read any thing from v_watt. .. there is no such thing as a read only variable, all depends on if you listen to incoming messages of that type or not.... I think you over-think this a bit. The atmega328 is a rather small mcu and wont be able to magically do tons of stuff. how to know what is what, well every home automation system should have a manual
Write it down.
-
I can see a case for making HSB the norm.
Think of light types like a class heirarchy, with added functionality
S_BINARY
V_STATUS (1=100%, 0 =0%)S_DIMMABLE
V_STATUS (1=100%, 0 =0%) V_PERCENTAGE (0.0% - 100.0%)S_HSB
V_STATUS (1=100%, 0 =0%) V_PERCENTAGE (0.0% - 100.0%) V_HUE (0.0 - 360.0) V_SATURATION (0.0% - 100.0%)If you set V_STATUS to 0, or V_PERCENTAGE to 0%, any of these lights turn fully off.
If you set V_PERCENTAGE to 100%, S_DIMMABLE will go to 100% (V_STATUS=1)
and S_HSV will go to 100%, in the current hue and saturation
If you set V_PERCENTAGE to 50%, likewise.I think one could make the case that it's more human compatible to go to "yellow, 25%" than to figure out what the RGB values would be. (The user interface could translate yellow to a hue number)
One aspect HSB is that there are a couple of ways of associating colors with angles, so we would need to specify which is our standard, so 40 degrees is the same hue for all nodes. I like the rainbow style.
@hek
I do also get the point that HSB -> RGB conversions could be done on the controller, passing the lower level RGB values to the node. There is an attraction to that as well.This cannot be fully resolved without also considering the question of light curves or gamma factors, which has been raised. The controller needs to know whether or not the node implements light curves for RGB channels, so it can avoid doing that twice; and also it in the changes how the controller should convert HSB to RGB (the stock conversions assume RGB are linear).
-
@Damme said:
Write it down
That's what I'm trying to do.
We could CLEARLY write down that V_WATT is intended only to report values and cannot be set, while V_RESET is a trigger, and V_R_PERCENTAGE is settable - so different implementors don't make different and incompatible guesses about what these mean.
We do not want the situation where one sketch writer interprets V_WATT to be a way to control how much power the lamp will use and another sketch write interprets it as reporting only. And then controller interface writers also guess differently about what it means. That's chaos.
One step in that direction would be naming them according to the intention - writable, reportable, or trigger.
I've done a LOT of implementing from specifications, as many others have. Having a clear specification that avoids huge ambiguities (is this something you set or something that reports is huge) is not over-thinking - it pays off big time.
-
-
I can see a case for making HSB the norm.
I'm not ;). it is just a possible addition of possible value types to the S_RGBW type and if not possible like you stated a S_HSB and to keep clarity V_PERCENTAGE should then be V_BRIGHTNESS. Or both percentages (not possible) or both naming as it is.
One aspect HSB is that there are a couple of ways of associating colors with angles
Off topic, which ones? i only know one, are you sure your not mixing HSB with HSL?
On topic:
I'm fine with only sending RGB if HSB ain't possible to be combined with S_RGBW or added with S_HSB/S_HSV if it is thought of as a logical addition.
-
@John
You are probably way ahead of me in terms of formal color spaces. I'm just a color user.I like using a modified Hue, such the the "rainbow" described here:
-
@zeph said
You are probably way ahead of me in terms of formal color spaces
Not really, i'm no color expert, But did a lot with colors lately because i use this to influence our living environment
Ok got it, Hue with rainbow style is variable due to the amount of colors (real life circumstance like angel etc) on that page an almost perfect rainbow angle and lighting situation is used.
I agree with you, but that's user implementation on the hardware, and something mysensors can not control. You would still need the HSB parameters
The implementation of rainbow or spectrum should be done on the controller<->end device side and the end user should know about it (or at least the developer of that specific controller/hardware implementation), not protocol based, because the parameters are still the same.
-
@Hek globally I don't understand the min/max/average, are they managed by the sensor or the controller ? if it is the sensor in case of reboot it should get it back from the gateway/controller and should be resettable maybe to have a year/month/week/day min/max. For me that should be controller side to handle this logic...
Now for my favorite playground
it needs the calibration value clearly first, then for the discussion some projects (AirPi) avoid the calibration pitfall by sending back the voltage... If we want to have a concentration value from the sensor that is ok.S_AIR_QUALITY - Air Quality Sensor V_LEVEL - ? We need to describe this better V_LEVEL_RESET - Reset max/min value @hek Hmm.. Soil Moisture.. I have no idea how to represent this. Help me! The ebay sensors we're using isn't super accurate and comes uncalibrated. @damme Hmm... V_Moisture, (relative) 0-255 / 0-100% / True | FalseOn both I agree, I have some imaginatronix and yes you need a margin of error... but when it rains it rains... this is where a calibration value will be usefull at least to have several sensors showing the same value at the same environment time... I you want a logic then the true false can be good but not enough... on my imagitronix I have an empty value of 20% and a watered value of 100%, sometimes after the rain it is 80%..., where would be the limit ?
Also, I am thinking of a leaf wetness (we had a lot here this year... and diseases that goes with it) that could be mapped to soil sensors [Article on values(http://www.decagon.com/education/do-you-measure-leaf-wetness/)
-
@epierre said:
globally I don't understand the min/max/average, are they managed by the sensor or the controller ? if it is the sensor in case of reboot it should get it back from the gateway/controller and should be resettable maybe to have a year/month/week/day min/max. For me that should be controller side to handle this logic...
The MAX/MIN/AVERAGE are optional to use. My idea was that sensors would manage them. They can choose to store it in eeprom or throw away values when reboot occurs.
The idea originates from GUST for wind which really is a max value over a certain time. This is just a generalization.
Usecase: Sensor might not report every wind value to controller but just every 10 minute. But internally it can calculate average/max/min for every 10 seconds without pushing this.
-
Please leave HSV/HSL to the controller. Why complicate the node more that necessary?
Using LEVEL would be good, 8 bit per color is good, it's also super easy for the node to scale down if only supporting 7 or 6 bits and it's perfect for those digital LED strips.Please have a pure RGB without W, most strips and RGB leds don't have a W diode so that would need to be simulated with different result in different nodes.
-
Please leave HSV/HSL to the controller. Why complicate the node more that necessary?
That's why i asked for an addition if it would over complicate stuff (and if there is room for it) ;). And well, yeah you're right, S_RGBW with HSB would be complicating things more mixing capabilities with the node naming. There is always S_CUSTOM with V_VAR_1,V_VAR_2,V_VAR_3 combination.
I agree with leaving out the W. But could be a nice addition for those non calibrated fixtures where you have to supply calibrated values in the node to get "proper" white levels.
-
@hek ok I agree. Can we have too an alert that could be sent back to the gateway, or like an armed mode for the air sensors ?
Today, one of my daughter broke a bottle of perfume, saturating the air and making the sensor go to its max. Even if the sensor is not calibrated, on some kind that (IMHO) must raise an alert for I cannot say infinite or over the maximum value...
-
@epierre In 1.4 a binary represetation of a float value can be exchanged with the library. This allows to report e.g. + or - infinite where normally floating point values are sent. Maybe this will serve your purpose?
-
If you do have a strip with RGB and W LEDs (rare), this can be handled as an RGB light and a DIMMER.
A lot of our discussions really hinge on how much smarts to put where in the architecture. My overall philosophy would be to keep the node simple and move functionality to it only when the same thing cannot be easily done centrally. Reasons to move smarts into a node could include:
- Uses local calibration that central doesn't have
- Needs updating more often than reports go to central (eg: wind max)
- Timing, eg: safety features that need to act fast
- Reliability of data links, eg: safety features that should not depend on RF robustness
- Providing some logically consistent interface available to all controllers
- Providing some functionality which cannot be as easily implemented in a controller (eg: closed source)
The latter two would be one-off custom coding at this time, tho @hek has mentioned the idea of pushing some rules down to the node level in a more automated way someday.
This probably argues more for using RGB rather than HSV at the node level. What I liked about the HSB was the unification of on/off, dimmable, and HSB functionality as a nested set of expanding functionality. So a script or rule that sets the brightness could work on a DIMMABLE or a HSB light by setting the V_PERCENTAGE value (or V_BRIGHTNESS as someone suggested). But that could be done at the central level.
I've used the term "central" because I still have the vision that a "hub" or "gateway" could implement some features, rather than having the smarts only in the controller or in the sensactuator nodes. So for example, it could be possible for the gateway to convert HSB to RGB such that the controller doesn't have to implement it. I know that approach is not currently popular so you don't need to register general disapproval. This will either come to seem a worthwhile additional open in coming years or not.
-
@epierre said:
Can we have too an alert that could be sent back to the gateway, or like an armed mode for the air sensors ?
Today, one of my daughter broke a bottle of perfume, saturating the air and making the sensor go to its max. Even if the sensor is not calibrated, on some kind that (IMHO) must raise an alert for I cannot say infinite or over the maximum value...
Let's not bring infinity into this...
How about just returning "the maximum reportable value" (finite). If the node is capable of detecting "above measurable" conditions, have it define the "maximum reportable" as 1 higher than it's internal "maximum measurable".
The idea is to keep a simple scale of results, where for example we can consistently compute "max/min over 24 hours", or set a trigger on "value >= threshold" without needing to deal with infinity math. Some controller software may not have any concept of infinity, so using it brings in unnecessary incompatibilities.
This allows the controller to define the alert as it wishes. For example, suppose a sensor reports uncalibrated values of 25 to 173, and the node uses 174 for a detected "over-range". The controller can implement a trigger or event at "value > 160" to test for high values (without having to test for infinite values too). Or it could test for "value >=174" if it specifically wants to test for overrange (which could be way over the max measured value or just barely above it).
-
@epierre said:
Today, one of my daughter broke a bottle of perfume, saturating the air and making the sensor go to its max. Even if the sensor is not calibrated, on some kind that (IMHO) must raise an alert for I cannot say infinite or over the maximum value...
LOL
-
@Zeph I don't share the same view, since we allow logic in the sensor for min/max interval, why not doing it too for the gas level ? all the more that the limit of the sensor is not a "business logic" but an internal logic linked by the physical capacity of the sensor itself (and which varies with each sensor type...).
My point was more that if you monitor LPG NO2 / NO3 or smoke that could be really harmfull we should provide safeguards anyway... not a big deal, more a safety net...
-
Ok,
I will change create two device-types S_RGB and S_RGBW and use a level instead of percentage for each color component (0-255).
-
For my sound sketch :
S_SOUND - Sound sensor
V_LEVEL - sound level in db
V_LEVEL_MAX - Max sound level
V_LEVEL_MIN - Min sound level
V_LEVEL_AVERAGE - Average sound level
V_LEVEL_RESET - Reset max/min value
-
For my vibration sketch :
S_VIBRATION -Vibration sensor
V_LEVEL - vibration level in Hertz
V_LEVEL_MAX - Max vibration level
V_LEVEL_MIN - Min vibration level
V_LEVEL_AVERAGE - Average vibration level
V_LEVEL_RESET - Reset max/min value
-
@zeph the smoke sensor is an Air Quality sensor in fact so we coud decribe it this way too.
@hek what about the V_GAS_TYPE proposed below ? that would limit extensing the S and V no ?S_AIR_QUALITY - Air Quality Sensor
V_GAS_TYPE - gas type reported by the sensor
V_LEVEL - gas concentration in ppm/ppb
V_LEVEL_MAX - Max gas concentration level
V_LEVEL_MIN - Min gas concentration level
V_LEVEL_AVERAGE - Average gas concentration level
V_LEVEL_RESET - Reset max/min value
V_STATUS - Tripped status of a gas sensor. 1=Tripped, 0=Untripped
V_ARMED - Armed status of a security sensor. 1=Armed, 0=Bypassed
-
@epierre said:
S_SOUND - Sound sensor
Wouldn't it be good to have a uncalibrated variable for S_SOUND also? Like proposed for S_LIGHT_SENSOR. Reported as:
V_PERCENTAGE - 0-100%
-
@hek hopefully all smartphone now have a calibrated microphone that could help calibrate this... what 'Ive not worked out enough is to know if the store sensor is suitable for this...
also I don't know what the semantic of "100% sound" would mean... max sensor capabilities ?
-
@epierre said:
also I don't know what the semantic of "100% sound" would mean... max sensor capabilities ?
Yeah, something like (1023-analogRead(ANALOG_PIN))/10.23
-
@hek hmmm... what about if this is a 3.3V or a 5V arduino board too ? no ? ?
-
Coming from the Can one child-sensor-id have multiple values thread I had a look at the initial post and I am wondering if this is what you can call "the latest proposal".
The rest of this thread seems to be more about which variables but what I am wondering is how all this would be implemented. Looking at the initial post I see:
New header would be:
ChildSensorId | ChildSensorType | ValueType | Payload
My assumption is that there is still a NodeID in the header, so actually:
NodeId | ChildSensorId | ChildSensorType | ValueType | Payload
If so, I think there is too much freedom/overhead in the first three fields.
My expectations as an user is that every sensor, or better "box" is one node and implements one type of sensor with one or more values. In this sense I do not see the use of the ChildSensorId field, so why not use:
NodeId | SensorType | ValueType | Payload
-
Hmm... what if you want to report multiple temperatures from one node?
-
@hek said:
Hmm... what if you want to report multiple temperatures from one node?
Or for actuators, control multiple LEDs/Blinds etc. from a single node...
I think a few bits could at least be saved by limiting the amount of ChildId's per node.
-
what if you want to report multiple temperatures from one node?
Does not sound to logical to me and I would be surprised if more than 1% of the implemented sensors have "double values", so the first question is if we want to carry an field for exceptions.
Multiple temperatures/LEDs/Blinds can still be implemented by adding an additional ValueType V_LEVEL_SECONDARY but a much cleaner approach would be to use another NodeId.
Let's also have a look at this from the perspective on how to represent things at the Controller:
- Is it logical to have two values of the same type reported for a node?
- How does the user know which temperature/LED/blind is what?
-
@daulagari said:
Let's also have a look at this from the perspective on how to represent things at the Controller:
- Is it logical to have two values of the same type reported for a node?
- How does the user know which temperature/LED/blind is what?
This seems to depend on how you conceptualize the system. In @hek 's proposal and the current system, the core concept of an endpoint is the child aka "sensor". (I'm going to cal this the child/sensor; elsewhere it's often called a "sensor' which as you say can be confusing, but what we mean is the conceptual element addressed using a child id within a node)
The node is a device with an RF transceiver (addressed by node id which is part of the transceiver address), which contains and makes accessible one or more child/sensors, each identified by the node id and child id. The controller receives reports from child/sensors and sends commands to child/sensors. The node is just part of the transportation system.
The controller is configured to know what each child/sensor consists of..
In that context, your first question could be split:
Is it logical to have two values of the same type reported for a child/sensor?
Is it logical to have more than one child/sensor supported by a single transceiver+uC (ie: node)?@hek made the point a while back that a child/sensor can be thought of as a semantic container for one (or sometimes more) closely related values. A node can contain one or more of those.
Yes, there would be an alternative, in skipping the child/sensor container layer. One approach would be to support only a single sensor per node, but there is clearly interest in multi-sensor nodes. Another would be to dissolve the child layer of this heirarchy use rules something like:
- a node can report on (or accept control for) more than one sensor, with more than one value
- no multi-sensor node can have more than one sensor which reports the same kind of value
- the controller will be responsible for logically grouping related sensor/control values within a node
I think one could get by with this for now, but I'm not sure it's a clean conceptual architecture to continue to expand for the future. And I wonder if it puts too much onto the controller - not a big deal if there was only one controller type supported (eg: Vera only), but with an expanding list of controllers, any functionality (like conceptual grouping of the values within a node into "sensor" containers) which is moved to the controller, requires that functionality to be programmed into every controller's architecture in whatever languages that requires - a lot of parallel maintenance to keep in sync for any changes. The less functionality is moved into the controllers, the easier it will be to maintain.
- How does the user know which temperature/LED/blind is what?
Exactly. Is this configured per value, or per child/sensor? (If no sensor ever had more than one value, this would be moot).
Currently the "what" is configured per child/sensor.
Suppose next year one wanted to measure the effectiveness of a swamp cooler with intake temperature and output temperature. Does that need to require two nodes, because the architecture was designed such that no node can report two temperatures?
If the goal is just to create an architecture which can get by for most systems today, there's not need for a big rethinking or the labor of rewriting the code - version 1.4 already does that. I think the point of this thread is to define a flexible conceptual base that's likely to be more cleanly extensible for a few years of growth. I think that hek's child/sensor concept is somewhat helpful in creating that flexibility. Perhaps there's an even better approach to serve that function, but just omitting the child/sensor container level doesn't currently strike me as such.
