2.0 Discussion: Units, sensor types and protocol

[Zeph]

@hek said:

###S_BINARY - Light (or other) on/off actuator
V_STATUS - 1 - turn on, 0 = turn off

###S_DIMMABLE - Dimmable actuator
V_STATUS - 1 - turn on, 0 = turn off
V_PERCENTAGE - Dimmer level 0-100%

OK, let's define what happens if you get more than one type of command (probably just making explicit what we are thinking anyway).

S_DIMMABLE:
V_STATUS = 1 is the same as V_PERCENTAGE = 100, either has same effect
V_STATUS = 0 is the same as V_PERCENTAGE = 0, either has same effect

If you set V_STATUS = 1 and then query V_PERCENTAGE it should return 100% (or 0 => 0%)
If you set V_PERCENTAGE = 100 and query V_STATUS it should return 1

OK?

The only question is setting V_PERCENTAGE = 1..99 and querying V_STATUS.
Option 1: Return -1 for "neither fully on nor fully off"
Option 2: Return 1 for "at least partially on" (ie: anything above 0%)

---

OK, what about a Fan with setting Off, Low and High, and another fan with settings "Off, Low, Medium and High?
We could map those to text "OFF, LOW, MED, HIGH"
Or we could map them to 0,1,2 where 2=HIGH for the first fan, and 0,1,2,3 where 3 = HIGH for the second

I'm going to suggest that we primarily map to V_PERCENTAGE, but perhaps also support the latter option as an expansion of binary 0..1 into multi step value 0..N_STEPS.

All fans:
    OFF = 0%
    HIGH = 100%
Fan 1:
    Low = 50%
Fan 2:
    Low = 33%
    Med = 67%

If you are sent a V_PERCENTAGE, set the fan to the closest speed to the given percentage. 100% is always the highest speed, 0% is always OFF, in between will approximate. This extends to 4 level fans. And it works for a heater or motor with multiple speeds as well. If you query V_PERCENTAGE the actual value chosen from the list will be returned, not the requested value (ie: requesting 45% but reporting 50% in the case of fan 1).

The above intermediate percentages are the default. However the node is empowered to set other values. We all know of fans or other devices where the speeds are not evenly spaced. I have a heater which has 600, 900 and 1500 watt settings (both elements). The Low and Medium setpoints could be 40% and 60%. Otherwise it works as described above.

And there could be an integer V_SET_STEP (or something), corresponding to V_BINARY* but ranging from 0 to N_STEPS where N_STEPS is per-variable defined. I'm not attached to this, but it's an option, It would work in the obvious way. So V_SET_STEP of 1 for Fan1 would set it to Low=50% and V_SET_STEP of 1 for fan2 would set it to Low=33% (and V_SET_STEP of 1 for the heater example would set it to Low=40%. SET_STEP of 0 is always off, and SET_STEP of N_STEPS is always HIGH or 100%

---

For completeness, I will mention that there's another way to define this for V_DIMMABLE. It could be required to memorize the last set percentage, and then V_STATUS = 0 means turn to 0% but remember last non-zero percentat, and V_STATUS = 1 could mean return to last non-zero percentage.

Tho I don't favor this dynamcic, it too could work. However I strongly favor defining which way we expect things to work rather than leaving it ambiguous!

[Zeph]

@Damme said:

I dont know why we have to limit some S_type to a number of V_types. Why not just let a S_type have 'all' V_types... User can be more flexible

As described at the time I write, the proposed taxonomy of values isn't really as heirarchical as it's presented.
You can also parse the meaning from V code to S code:

Ah, we have a V_LEVEL value, that means it represents a one dimensions continuum
But what does that continuum represents?
Oh, I see the variable is also tagged with S_TEMPERATURE
That means the continuum is temperature in degrees C

That is, the V code tells you something of the structural concept, and the S code adds some degree of semantics (like dimension and unit).

So you could also invert the displayed hierarchy and show which S codes are available for a given V code

[Zeph]

Hold the presses.

I've happily engaged in the collaborative process of cleaning up the variable taxonomies of V 1.4b here.

But we keep coming to conceptual tricky parts. Some of that is inherent, but in thinking deeper about it, i've come to the conclusion that a good part of what makes it hard to resolve cleanly is that we are trying to cram too much information into too few bits of variable metadata which has to be included in every packet.

So rather than reforming S and V codes, I'm thinking a more radical approach is needed, to make the system both more flexible and simpler. Part of that comes from identifying what metadata about a variable is static and what needs to be dynamic, and then moving the static part out of the normal OTA operations. That turns out to make the taxonomy much more tractable and flexible while simplifying the node firmware.

Guess what - even child ID turns out to be metadata in this approach, rather than addressing.

Here's the sketch of the new approach.

http://forum.mysensors.org/topic/308/another-way-of-organizing-variables

It's sufficiently different that I expect it may encounter resistance - from the natural and appropriate fear of breaking too much (I share that concern). But it just may turn out to be sufficiently attractive to be worth it. Have a look. And now would be the time to consider such a break - with RadioHead and OTA programming support also in the air.

[ToSa]

@hek said:

This would only leave 3 COMMAND types left. REQ, SET and STREAM. Where STREAM-data probably could be handled by S_NODE (internal). This means just 1 bit is needed for a SET/REQ flag. But @ToSa would have to feedback on this.

I switched the bootloader code between using stream and using simple internal messages - at the end doesn't matter as long as it's simple enough to dis-samble and assemble a packet without a lot of additional code overhead.

[hek]

@ToSa

Sorry, forget I brought this up. We'll keep the STREAM command as-is. Skyped you about it yesterday ;)

[ToSa]

@Zeph said:

But suppose we DO want to go further and determine accumulation time for all nodes, even the above.
...
System Time approach 1: Master Clock Broadcast to nodes
...
System Time approach 2: Track accumulation times in the hub

I'm in favor of approach 1 as well - and an optional addition to the MySensors class to keep track of time. Only specific sensor types would use this feature - interesting mainly for actuators rather than sensors:

Let's assume you use a relay to turn on/off your garden watering while you are on vacation. The network breaks (controller or gateway or whatever node needed to relay messages).

• If the sensor node itself knows about the time and the schedule for watering, it can turn on and off the watering itself. If there is an issue with the communication, the time would not be 100% accurate over time but still good enough.
• If the sensor node relies completely on the controller for time and (worst case) the communication breaks just between the start watering and stop watering command you can imagine what happens...

Knowing about the time heavily reduces the single points of failure. The protocol already covers the "request time" communication so this should be relatively easy to implement...

• controller sends a time broadcast every e.g. 15 minutes
• only the nodes interested in time care about it and update their internal time
• when a node needs the current time it calculates it based on the last timestamp received, the millis() when it was received and the current millis...

Doesn't solve the sleep issue (as long as you don't want to submit a separate "get time request" every time you wake up) but at least for sensors that are not battery powered (relay nodes in most cases are not) it's a feasible approach.

This is probably worth a separate thread :)

[hek]

@ToSa, @Zeph

Yes, and to extend this even further it would be great to be able to distribute/push simple rules all the way out to the nodes. It might be enough with simple scheduler-commands to start with but this could easily support even advances scenarios. The question is where to set the cuttingpoint...

The most extreme solution would be to allow a graphical rule engine on the controller which automatically generates a sketch, compiles it and sends it out OTA to the node.

Definitely worth discussing in a new thread...

[Zeph]

@hek said:

Yes, and to extend this even further it would be great to be able to distribute/push simple rules all the way out to the nodes. It might be enough with simple scheduler-commands to start with but this could easily support even advances scenarios. The question is where to set the cuttingpoint...

For the level of protocol under current discussion, the cuttingpoint could just be giving nodes an optional sense of time (using broadcast packets to keep in sync with the master as needed). And making use of that time as part of the (optional) accumulation enhancement for some variables.

What else they do with that time at higher layers would continue to evolve in the future. We don't have to bite it all off now.

[Rasmus Eneman]

V_R_PERCENTAGE - Red component % <int>
V_G_PERCENTAGE - Green component % <int>
V_B_PERCENTAGE - Blue component % <int>
V_W_PERCENTAGE - White component % <int>

Just a hundred steps might be to low, especially for green in the lower end which the eye sees much brighter.
Please consider floats or per thousand instead.

[hek]

@Rasmus-Eneman

Ok! Didn't know that. But couldn't the sensor code adjust this with some sort of color curve?

[Rasmus Eneman]

@hek
That would probably do it, however one would need to be defined and standardized so that all behaves the same. One more clever than me needs to propose how the curve would look tough.

[John]

@hek @Rasmus-Eneman

S_RGBW - RGBW Light
V_STATUS - 1 - turn on, 0 = turn off
V_R_PERCENTAGE - Red component % <int>
V_G_PERCENTAGE - Green component % <int>
V_B_PERCENTAGE - Blue component % <int>
V_W_PERCENTAGE - White component % <int>

Ain't it better to split up S_RGBW to two different types. W is already present in full R, G, and B. For example i have implemented Philips Hue and there is a difference in fixture types some do RGB some only W. If an user would implement such with mysensors the same option can be possible. An S_RGBW declaration for a W only would indicate RGB capabilities and maybe cause confusion.

Just a hundred steps might be to low, especially for green in the lower end which the eye sees much brighter.

Agree, but thousands could be too much (which never is the case of course), maybe 0-255 would be sufficient and widely used (all though not as precise in steps as with thousands)? Also the W_PERCENTAGE should be left out because this is full R,G,B or similar calibrated R,G,B levels (which is quite difficult to do at every possible level) with the end result it then in fact is a brightness level instead of a white component.

An addition could be S_HSB/S_HSV with float,float,float (0.0-360.0, 0.0-100.0, 0.0-100.0)? Also i think the color presentation should be or calibrated on the device or in the controller. Even some of those Philips Hue's fixtures are often controller side calibrated (better said color index modified after color selection)....

[hek]

I actually have RGBW led strips mounted around our house. W is indeed its own component (Warm White in my case). So it will stick ;)

(Warm White using the special W-component)

Cold white (RGB activated)

Crazy

[hek]

@John

But switching to LEVEL and using 0-255 would be ok (if we need it ;)).

[John]

@hek
Ahhh... understood :)... Splitting those up will not be an option then because it could then create a numerous amount of options....

A S_HSB would be nice though, even if it would only be ints....:
V_STATUS - 1 - turn on, 0 = turn off
V_H_DEGREE - Degree (0-360)<int>
V_S_PERCENTAGE - Saturation % <int>
V_B_PERCENTAGE/V_V_PERCENTAGE - Brightness/Value % <int>
V_WATT - Watt <int>
V_WATT_MAX - Max watt value
V_WATT_MIN - Min watt value
V_WATT_AVERAGE - Min watt value
V_WATT_RESET - Reset max/min value

I must say it is a personal request :$....

[Edit]

But switching to LEVEL and using 0-255 would be ok (if we need it :wink:).

It would create space to add the HSB/HSV option ;)

[Zeph]

What's wrong with setting lights at 1.475% Is percentage limited to integer?

Let's be very explicit about that. Is the idea that all values accept only integers unless they say they have the option of using floats? If so that should be made clear in the list.

---

Another thing that needs to be more clear is how to handle multiple V codes for the same sensacturator (like the RGB or HSV lights).

• Is a sensor expected to report all of them?
• Is an actuator expected to respond to all of them?
• How do both sides stay in sync about subsetting?

By the way, there is a distinction to be made between RGB lights (three blendable primaries able to synthesize many colors in a predictable and fairly standard way) and multichannel control of N lights of arbitrary colors. RGBW is another thing entirely. The W could be cool or warm white. For that matter the fourth color could be yellow or violet. It's not a color space.

If the RGB channels blend I'd implement it as an RGB light plus a monochrome light, because there is no RGBW color space. If they don't blend I'd implement it as 4 monochrome channels, the same way I'd implement a Yellow and Green pair that didn't blend.

[John]

@Zeph

What's wrong with setting lights at 1.475% Is percentage limited to integer?

It just not represents any absolute bit value for a specific color, it is an approximate. If for example you have a fixture which does 255 steps and you want to set it at step 40. you will need to send 15.682.....%. But what if you would send 15.770... ? Due to the nature of floats it could be rounded down or up to 40 or 41 and especially in the lower steps of a color the difference would be seen if this is a mix of on of the RGB values, and even more as all the RGB values are send in percentages. A percentage would only be usable (But even then it's still an approximate) with 32 bit or analogue blend RGB fixtures. There ain't a lot of these.

@zeph said:

If the RGB channels blend I'd implement it as an RGB light plus a monochrome light, because there is no RGBW color space. If they don't blend I'd implement it as 4 monochrome channels, the same way I'd implement a Yellow and Green pair that didn't blend.

I do agree with you, and you're totally right, but:

@john said:

Splitting those up will not be an option then because it could then create a numerous amount of options....

In other words it could eat up the possible S_TYPE's

@hek
Please disregard this one, it wouldn't :"it would create space to add the HSB/HSV option".
But, couldn't it be an option to add the HSB/HSV variables to the S_RGBW? because if the presentation stays you would only have to send the vars that are supported, and the type S_RGBW could be renamed to something like S_LIGHT_COLOR?

[hek]

The question is where the actual calculation is done between RGB and HSB/HSV. Should it be done on the Arduino side or controller? Nothing stops you from making a fancy HSB/HSV GUI on your controller but the actual values send to the actuator is RGB. But my knowledge of color scales is a bit limited and I'm probably missing something?

[John]

@hek

It depends, if the HSB scale is purely in integers you will have about 3.600.000 options (including brightness steps) which is much less then 16 bit 0,0,0 - 255,255,255 scale. But this scale ain't specific in int's it is an implementation chosen scale. If choosing for ints, yeah keep the RGB, if going to floats, add it next to the RGB.

probably missing something?

The main difference is that with pure 16 bit you will have whole bit numbers and with degrees there is more then just go from 359 to 360/0. you could use 359,00 to 359,99 which gives an extra 990.000 extra red combinations (including saturation and brightness) with absolute 16 bit you stick with whole ints/bytes limiting the possible fixtures light high "resolution" capabilities.

Also because of the nature of HSB with higher color resolution possibilities you will always have the possibility to find the closest 16 bit neighbor or even the exact bit value because of higher to lower "spectrum" resolution.

With the above if you would leave out the brightness you would have a higher color resolution 255,254,255 is one step lower green form maximum, you can't go to 254.9 right? in degrees i think green is 120, you could go to 119.99999. because the degrees are not fixed to a whole as a byte is.

Because of more steps calibrating un-calibrated fixtures/leds (for example with the green mentioned earlier) can be more precise. Also HSB is more human perspective then fixed bits.

Maybe the above shines a light on the difference?

RGB is just a brightness setting of a specific color, HSB is pure color selection, intensity setting, and the brightness of these.

Personally:
HSB is nice to have, but from personal needs i have enough convertion functions (hsb->rgb and vice, hsb -> kelvin, etc...) to convert.

Should it be done on the Arduino side or controller?

I would let the controller do the actual conversions and let the device spec it's capabilities.

[Zeph]

@hek said:

S_RGBW - RGBW Light
V_STATUS - 1 - turn on, 0 = turn off
V_R_PERCENTAGE - Red component % <int>
V_G_PERCENTAGE - Green component % <int>
V_B_PERCENTAGE - Blue component % <int>
V_W_PERCENTAGE - White component % <int>
V_WATT - Watt <int>
V_WATT_MAX - Max watt value
V_WATT_MIN - Min watt value
V_WATT_AVERAGE - Min watt value
V_WATT_RESET - Reset max/min values

One of the areas where the spec is very unclear is when a variable is writable and where it's only reportable (read only).

For example, can I set the watts to 20 above (or for S_DIMMABLE) in order to tell the light how much power I want it to use? Seems like a valuable option. How am I supposed to know?

What if we made the write/report/trigger function explicit:

S_RGBW - RGBW Light

W_STATUS - 1 - turn on (100%), 0 = turn off (0%)
W_R_PERCENTAGE - Red component % <int>
W_G_PERCENTAGE - Green component % <int>
W_B_PERCENTAGE - Blue component % <int>
W_W_PERCENTAGE - White component % <int>
R_WATT - Watt <int> - current instantaneous or short term average value
R_WATT_MAX - Max watt value since last reset
R_WATT_MIN - Min watt value since last reset
R_WATT_AVERAGE - Min watt value since last reset
T_WATT_RESET - Reset max/min/avg value

W variables can be written to; the current value can also be queried
R variables can only be reported (automatically or by query)
T variables are triggers, writing to them triggers some action

This gives guidance to both the sketch writer for the node, as well the controller adapter writer in knowing what they can do with each variable a node presents.

The actual numeric values for R, W and T codes could be interspersed or non-overlapping ranges could be reserved for each (eg: 1-100 for R, 101-200 for W and 201-255 for T)