OK, here's some background, for some eventual Wiki article or such.

So there are 4 main ways to get a program into the uC:

1. In System Programming (ISP) - uses SPI pins + reset, can program entire flash as well as fuses. This is how we get bootloaders into protected section of Flash. (There are also parallel programmers, and sometimes even additional methods like DebugWire or JTAG).

Using these methods is more cumbersome, so the Arduino folks chose to use "bootloaders" - a small program programmed (via ISP) into a protected section of on-chip Flash, and which can in turn communicate with the outside world and write (program) a compiled sketch into the rest of on-chip flash. The bootloaders are invoked by rebooting (or upon powerup) the uC.

1. Serial bootloader using Arduino bootloader protocol (subset of Atmel's STK500 protocol). A program in a protected section of uC onboard flash that can write compiled sketches into the rest of flash. Optiboot is the name of a particular serial bootloader used in many ATMega328p Arduino derivatives. (Not always used in APM (Arduino Pro Micros) but can be).

2. MysBootloader - similar to the serial bootloader, is a small program in a protected section of on-chip flash which can write compiled sketches into the rest of the flash, but it gets the data to write from special Over The Air (OTA) bootloader packets rather than via the serial port.

3. DualOptiboot - this is another OTA bootloader, but the idea is that you write the new compiled sketch (in binary) into an external flash chip, and then only after you have received and validated the entire new binary do you reboot and let the bootloader copy from external flash to on-chip flash. Called DualOptiboot because it's derived from OptiBoot and still supports serial bootloading as well.

With the bootloaders, once they begin to program the rest of flash, they wipe out the old compiled sketch - the chip can't in general boot into a sketch again until all of the new sketch has been successfully programmed. With a serial bootloader, this is usually pretty straightforward and reliable. But flashing a page at a time via an OTA bootloader like MysBootloader is a little more dicey in that it's easy to miss packets and leave the chip in a "broken" state - with no workable sketch, only the OTA bootloader. The DualOptiboot approach is perhaps in some ways a little safer, in that the current sketch in on-chip flash is not overwritten until there is a validated copy of the new code in an external flash; then copying from external to internal flash by DualOptiboot is again pretty reliable and fast. Also, if all else fails, you can still open up the node and user serial bootloading. However, DualOptiboot only works when you have an appropriate off-chip flash memory connected, which limits your hardware.

I believe that at this time, MysBootloader only works with the nRF24L01+. It basically needs a subset of the nRF's library in the bootloader section (which grows the bootloader from 1/2 KiB to 2KiB I think). Porting it to use the RFM69 would involve using a subset of the RFM69 library in the bootloader, and had not yet been done when this was written.

If OTA bootloading fails, you can try again (you may need to power cycle the node). You can reduce the distance between source and destination. You can open the node and fall back to normal serial bootloading (at least with DualOptiboot, maybe with future versions of MysBootloader). And of course there is the fallback of using ISP (assuming the PCB doesn't do something to the reset & SPI lines that breaks that option).

Since MysBootloader keeps requesting replacements for missing or corrupted OTA bootloading packets, it usually won't fail unless the node is really out of range or there is terrible interference.

Tekka points out below a different failure mode - suppose the bootloading was successful, but the sketch itself is faulty (bugs). A serial bootloader like OptiBoot takes control upon reset (which in modern Arduinos can usually be triggered via the serial connection using an additional wire), and goes into bootloading mode if it receives proper data on the serial port - or if not, starts the currently loaded sketch. This may also happen on power up. For an OTA bootloader there's no additional wire to trigger reset on the target uC. Tekka says that MysBootloader can recover and load another sketch OTA while for some bugs in a sketch, DualOptiboot cannot recover (without a manual reset).

As best I understand it, MySensors was originally developed around the conceptual framing of devices as used by Vera/MiCasaVerde/MIOS; has been remapped by some adapters to work with other systems; and is at the stage where the semantic gaps are becoming obvious enough to raise the question of rethinking the internal conceptual framework to be more general and less Vera specific.

Let's start from the bottom of the hierarchy - what we could call "variables" - each containing a value which may vary over time.

**Format: **
The values could be

• floating point or integer (of various sizes), or
• chosen from an enumerated set of possible values (a common case being binary options like on/off or true/false or armed/unarmed etc)
• a text string

What it measures (or controls):
In the case of a numeric value, this could be temperature or relative humidity or wind speed or water depth or brightness or rate of flow of electricity or water or accumulated flow of electricity or water or loudness of sound, or angle of a servo, etc. Sometimes there may be no conventional dimension or interpretation.

Units and scaling
There may be a translation to a standard unit of measure like degrees C or F, or percent humidity, or feet or meters, or mph or kph or m/s, or lux, or db or angular degrees or radians, etc. Sometimes we may know the dimension being measured (brightness) but have no translation to standard units (not calibrated in lux, just an arbitrary and perhaps non-linear value) so this is optional. And sometimes there may be a scaling factor needed to get to some standard unit, from the value reported.

The dimension and the units are often conflated, but of course a value in degrees C could be a temperature reading or a dewpoint, two different dimensions which use the same units.

---

That's the basic level - a set of distributed variables measured from the real world (or computed from measurements), or controlling some action in the real world. Then we built on top of that.

Grouping of related measurements
We might wish to associate several variables with a given device or sensor. For example a weather station might have temp and humidity and wind speed and wind direction, etc. Or a flow meter might have immediate flow (gph or kw) as well as accumulated flow (gals or kwh).

This is where some things go a little wonky, because there isn't always one obvious way to logically group variables, and there may be little guidance as to the best practicies OUTSIDE OF matching the way a particular controller has chosen to conceptually organize things.

From the top down, MySensors groups sensactuator variables by the wireless node which reports or uses them. This has a lot of practical sense to it, but in some sense it's a transport or addressing concept rather than a sensor concept. Suppose you had some one wire sensors strung around the property. As you add, remove and move wireless nodes around over time, you might find it more convenient to connect the (relatively long) wiring to different wireless nodes for reporting to the controller. In this case, the actual one-wire sensor would not change in any logical way (it's still measuring the same thing in the same location) but the values are reported through a different path, a different wireless node. So grouping of values by wireless node should be done lightly - it's not a primary logical grouping based on what is being measured or controlled, but more of "the current address and transport path".

The child id is also part of the addressing and transport path.

From the bottom up, MySensors group sensactuator variables into Sensors, each of which has a sensor type. This is one of the confusing areas, since the types of sensors seems to overlap semantically with the types of variables which might be associated with it.

In trying to sort through this, I think the Sensor type mostly represents "which user interface object in the Vera system should be used to display and accept user input" for a group of variables". So if you have one or more related sensor variables which can logically be associated with a given Vera UI object, you may use that sensor type. Or you can do some more work and add a new sensor type to Vera, and use that.

The problem is - another controller may not choose the same set of UI objects. I think that "sensor type" may be better conceptualized not at the MySensors generalized node level, but as a mapping of the generalized sensor nodes into the Vera system.

In this approach (which is being brainstormed, and is not fully figured out), the sensor node would not need to understand Sensor Type (tho it would understand Variable formats, dimensions, scaling, units and names). Instead the gateway to Vera would map variables into Vera related groupings, and the gateway to another controller would map the variables into the groupings which makes sense to that system.

Suppose that one controller had a user interface element which displayed weather related information graphically, always using metric values. Another controller has a "temp and humidity" user interface object but no weather station, tho it does have a separate flow concept to which wind speed could be mapped; the second one allows the user to select metric or imperial units. The first controller might display wind direction in degrees clockwise from true north, the second might use NNW type notation.

The same physical sensors and associated variables (at the node level) might be grouped into "sensor UI objects" differently in the two controllers. But the nodes shouldn't care whether wind speed is shown inside the same box with temperature (a "sensor" UI element) or in separate boxes, or even omitted from a controller which doesn't have a target for some measurement. They just report values for variables, and the gateways alone handle grouping variables into user interface elements as used by the controller they support.

In this model, more than one controller could be used with the same sensor network, so long as they don't issue conflicting commands. The sensors might always report temperature in C, and each controller + gateway would translate if it so needs. The sensors might report wind direciton in degrees and each controller + gateway would translate to compass points if so needed. If one of the controllers doesn't have a way to display "microcontroller temperature" or "H Bridge back-emf" then it can omit that variable, without precluding another controller from accepting such variables.

Knowledge about the controller and its concepts would be removed from the wireless node code, and concentrated in the gateway to that controller. Nodes just report (or accept) one or multiple sensacturator variables. The gateway could package up groups of sensactuator variables as needed for the specific type of controller it serves. Suppose a given controller had a "temp & humidity" object but no weather station object. As described above, you could map wind speed into some other controller element for now. But then when a weather station "sensor type" with additional variables is defined next year, the gateway could regroup sensor variables into that package, without any changes in the wireless nodes.

---

Splitting the gateway and the hub

The concept of a gateway may need to be separated from the concept of the wireless hub.

The wireless hub understands how to communicate with the wireless nodes, up through the protocol level of transferring sensactuator variable values back and forth. Like the nodes, it doesn't need to know what controller(s) are attached or their concepts, it just moves values. It could use a different radio chip, or RS-422.

The gateway function translates from this general format to a specific controller's needs. (Cloud storage could be another type of "controller", by the way). The gateway function doesn't have to worry about ack and routing tables for the nRF24L01+.

There is a general API between the MySensors wireless hub code and the MySensors gateways. They can run on the same processor (or not, if there is a transport option for the API).

When you add a new controller, you write a new gateway, but can use the same wireless hub code.

And a message manager like a MQTT broker could be inserted between the wireless hub and one or more gateways.

That's the architecture which appeals to me. If it has enough appeal to others, including and especially hek, then maybe that will be version 1.5 or 2.0 of MySensors.

This is where I was generally heading before discovering MySensors (which is WAY ahead of me in terms of practical implementation).

As I understand it:

The packet on the left would be from the node, the packet on the right would be to the node. Besides swapping sender and destination, the difference is that in 1.4 the gateway does not have the information to binary encode the value, so it sends the value as a string (payload type 0).
EDIT: added alternate MQTT respresentation of V_code as numeric

I don't see how this technique saves any resistors or battery drain. I see there being two different measurements - input to a voltage regulator and input VCC to the uC, which are measured differently.

By "this technique" I mean using VCC as the analog reference (as usual) and reading the internal 1.1V internal source. That gives you 1.1v as a fraction of VCC, so you can easily calculate VCC..

The two measurements:

1. You want to measure regulator INPUT voltage. You typically use a voltage divider to reduce the regulator INPUT voltage to less than VCC, which is used as the ADC reference. The technique describe here cannot be used for that measurement, so it doesn't save resistors or battery drain.

2. You want to measure regulator OUTPUT voltage - or raw voltage if there is no regulator - as applied to the uC for VCC power. The technique described is the way to go, and a voltage divider is inappropriate.

If you used a voltage divider on VCC (case 2) and measured it against the normal VCC reference, all you'd get is a fixed value determined by the resistors, not reflecting VCC.

Summary: if (1) you want to measure regulator input, use a voltage divider, but if (2) you want to measure VCC then use this (well known) technique. Neither approach substitutes for the other. And you can use both, if you want both measurements (eg: if you are concerned about regulator drop-out).

Side note. There are yet other approaches. You could use the 1.1v source as REFERENCE for the ADC instead of VCC, and on the ADC input use a voltage divider from VCC or from regulator input, with a large enough ratio to bring the measured voltage down to less than 1.1v. If you had a 5v processor fed via regulator from a 7.2v source, you could measure the former with a 4:1 resistor ratio and the latter with an 6:1 or greater ratio. This is not commonly done.

If you DO use a voltage divider for measuring voltage (when appropriate), you can reduce the current wasted by using large resistors in the hundreds of Kohm range, AND putting a cap across the lower resistor. You need the cap to reduce the effective impedance into the ADC. If you just use large resistors, the internal capacitance of the ADC circuitry will be charged too slowly during measurement.

(see new links at end of this post)

(see earlier thread about nRF24l01(no plus) substitutes - not quite the same topic as this thread but related; from a comment below)

(see newer thread on finding the best nRF24L01+)

First off, it appears that most of the cheap nRF's have been fakes (not genuine Nordic chips). There is some indication that they may use somewhat more power and be somewhat less sensitive, than the real thing - and may be more sensitive to power supplies. Not a big problem, since that level of performance is what we're used to (many of us have probably never used a Nordic nRF24L01+).

However, there are even cheaper fakes hitting the market, with inferior specs and outright incompatibilities. In particular, the Si24R1 got the ACK bit inverted (following an error in the datasheet), so it's incompatible with the real nRF24L01+ (and good clones) in ESB mode. (The Si24R1 is often falsely labeled as nRF24L01+). In some cases you can build a network of such variants that work OK together, but a node with the other chip won't work right (at least in ESB mode). There are several other clones too, with whatever changes they happen to have.

And clone chips weren't enough, there is a new cheap module with the chip under an epoxy blob (no biggee per se), which also omits a number of the passive components in the Nordic reference design used by most modules (caps, resistors, inductors). This seems to have problems with the ACK bit due to the chip itself (see avove), but due to module design also probably has reduced actual transmit power, less power filtering, etc. (And would even if it had a good chip in it).

I'm mentioning this because chips that don't work right in some circumstances (depending on which features are used), or which are electrically marginal and thus have intermittent problems, can lead to serious hair loss. And this is probably going to be an increasing problem in our niche, as our vendors switch to the cheapest suppliers.

If you are having odd problems, sometimes it may be that you have one of the clone chips or cost reduced modules.

Perhaps we need to find a good supplier who knows we want only "genuine" high quality clones and who will not suddenly switch to less compatible ones. In the DIY christmas lighting community many people patronize one particular Ali Express vendor (Ray Wu) who has an overall good reputation within that community, who is aware that many people in that community communicate frequently, and who wants to keep a good reputation. He gets enough volume from that community to have incentive to keep up his quality and to fix problems (at least in relative terms among Ali Express vendors).

Some background:

http://hackaday.com/2015/02/23/nordic-nrf24l01-real-vs-fake/

http://ncrmnt.org/wp/2015/03/13/how-do-i-cost-optimize-nrf24l01/

Updates:

List of related chips (if not mislabeled as Nordic these would be clones, derivatives or compatibles, not fakes per se, but I'll list them here as some fakes may really be one of these chips or closer to it): http://sigrok.org/wiki/Protocol_decoder:Nrf24l01

Info about Si24R01 and power: https://github.com/solarkennedy/equail/tree/master/Libraries/RF24

I've been gone for a while, but this seems like a nice next step. Feel free to move these comments to the forum if that make more sense. If it's too late for this round, perhaps they can be useful for the next revision.

1. I still think the controller (or if need be, the gateway) should do any neccessary conversion between metric and imperial units, rather than pushing it down to every node; it seems like some kind of violation of partitioning to push that UI customization concept (which in many controllers the user can toggle freely or even display both ways) down to the node level, where it adds unneeded complication. But I'll let that go.

2. It would be good to augment this with a systematic description of the underlying value type (int32, float32, string), and implied units if appropriate (potentially two units, for metric and imperial). For example, is wind speed in m/s or km/hr (assuming metric)? Is pressure in mmHg or hPA?

3. Aside from the type and implied units, there are a few hints given already - what 0 and 1 mean, an enumerated list of valid string values, color given a strings of hex digits, position as a multivalued structured string, etc. Keep those as notes in augmented docs as described in #1, and add more a needed (eg: IR commands)
   2A) Is there any logic to when enumerated values are encoded as integers, and when they are encoded as strings from a controlled vocabulary? Could we get by with one or the other, rather than this hybrid?

4. How do the semicolons internal to V_POSITION interact with the semicolons in the message field structure? If the same character is used for both, some note about that is needed. Or use commas in the former (substructured payloads) and semicolons in the former (overall message structure).

5. I'm glad to see that a distinction has been made between sensed light levels in Lux and uncalibrated light levels. Generalizing, I can see four types of possible reporting of such things. 1: uncalibrated results in arbitrary units (eg: ADC values of a LDR+resitor voltage divider), 2: scaled results 0-100%, 3: reporting in predefined units understood by both sides, like Lux, 4: reporting in uninterpreted units which can only be displayed but are not parsed. In the first case, I think the only thing to define for a given V_variable is whether brighter should be reported as higher or lower. This standard chooses 0-100% for V_LIGHT_LEVEL. Could't that be reported as V_PERCENTAGE instead? That is, use V_PERCENTAGE for any sensor reporting in scaled percentages.

6. If the protocol is going to allow for both uncalibrated and calibrated values (the latter in implied units), as with light levels, then I think it makes sense for the value with implied units to be the more specific variable; that is, V_LIGHT_LEVEL should be in lux, and V_LEVEL (or V_PERCENTAGE) would be used for the generic values, without units.

5A) V_LEVEL currently seems a little convoluted. It appears that the implied units depend upon which S_sensor the variable is associated with, which kind of breaks the pattern that units (if any) are implied by V_variable. And for S_DUST, S_AIR_QUALITY) there are no units. So for a calibrated light level in lux OR an uncalibrated dust level, use V_LEVEL but for an uncalibrated light level use V_LIGHT_LEVEL. V_LEVEL seems to be a grab bag of conceptually unrelated magnitude values, some with per-S_sensor implied units and some without. This could use rethinking. How about if V_LEVEL was used for any values which do not have implicit units (see #1 above), and V_PERCENTAGE was used if the value scales to 0..100%. Other V_variables could have pre-specified implicit units (metric and imperial).
5B) V_UNIT_PREFIX is really more of a suffix than a prefix. But this can work well with 5A above. If a value is reported as V_LEVEL then there are no interpreted or parsed units involved and the controller is not expected to be able to interpret or convert units. However, V_UNIT_SUFFIX could optionally supply a free text string with can be displayed after the value. So if you are reading a LDR and reporting the raw ADC value (or inverted value to ge the direction right), use only V_LEVEL. If you are reporting something in units not understood by the system, then use V_LEVEL and describe the units in V_UNIT_SUFFIX. (And if you are reporting percentage, use V_PERCENTAGE).

1. Semantically, there are two kinds of outputs: set-current-value which sets a new state to be retained, and trigger/command which performs some action once. It's kind of like edges and levels in hardware logic. Setting the same value twice is harmless, but issuing the same command twice may have different effects than sending it once. Most of the outputs are about set-current-value. However, some are commands - like sending an IR code.
   6A) What do V_UP, V_DOWN, and V_STOP mean? Unless they contain payloads to configure a (retained) movement speed or something, they seem more like commands/triggers rather than set-current-value. In that case, it makes more sense to have a V_SHADES variable which is set to 0,1,2 for stop, up and down (which are mutually exclusive), than to have three variables which could be set in parallel. Likewise with V_SCENE_ON and _OFF. These should be handled with one V_variable, much as sending a IR command. This is also easier to extend for an additional command to the same device - just define additional values to action associations, rather than needing to add a new V_variable.
   6B) (A shade controller which could set/report percentage opening could use V_PERCENTAGE)

2. Likewise there are two kinds of inputs: report-current-value and trigger-event. Mostly this protocol is about report-values. However the trigger functionality comes up in terms of V_TRIPPED, which is often implemented as a trigger. It should be made explicit what resets a triggered state: some systems use autoreset where after one report a device goes back to untriggered (trigger/event); other systems will keep reporting triggered forever until separately reset by some other means (report-current-state). We should be clear which mode this protocol expects.

It's good to see the ongoing refinement of MySensors and related initiatives! These are definitely meant to be helpful, not as criticism! Your work is much appreciated.

Great work indeed!

However, the source link (at this time) fails...

@jendrush said:

Maybe small usb wall charger for 1,4$, and regulator from 5V to 3.3V for 0,1$ would be good solution?

For some projects, using cheap chargers can be a good option. However, I'd open any such cheap charger and have a look before using it. Some appear to be acceptably designed, and some appear to be dangerous designs, from the teardowns. For example:

http://www.righto.com/2012/10/a-dozen-usb-chargers-in-lab-apple-is.html

He rated 2 out of the batch as downright dangerous.

Even with one which appears to have a reasonable design with enough separation, I would not be poking around in the sensors or PCB while they are currently powered from 230V with a cheap charger. Unplug first. And if you need power for programming or testing, use your normal USB supply (eg: a laptop or a high quality power supply) or batteries. Maybe that's just me, but this seems like a reasonably easy additional precaution. Even if the PCB has good separation, the insulation between windings of a transformer could be marginal, for example.

Most readers here probably already know this, just thought it was worth noting for anybody coming to this hobby more recently.

@hek said:

Yes, and to extend this even further it would be great to be able to distribute/push simple rules all the way out to the nodes. It might be enough with simple scheduler-commands to start with but this could easily support even advances scenarios. The question is where to set the cuttingpoint...

For the level of protocol under current discussion, the cuttingpoint could just be giving nodes an optional sense of time (using broadcast packets to keep in sync with the master as needed). And making use of that time as part of the (optional) accumulation enhancement for some variables.

What else they do with that time at higher layers would continue to evolve in the future. We don't have to bite it all off now.

@Yveaux said:

Off-topic: why the heck did @hek add support for this temperature sensor to mysensors 1.4 library??? IMHO It doesn't belong there at all!

I think the internal ATMega temperature can be useful in some circumstances.

Obviously if your goal is to measure weather, there are much better external sensors. But if your sensor is primarily detecting motion or open doors or controlling relays whatever, you may not always have an external temperature sensor included - but you always have a internal one, for free - albeit low accuracy.

Or if you do have an external temperature sensor, the internal temp could be used as a sanity check - if in your physical setup the processor should be close to the same temperature as the external sensor, but the two are way off, something needs to be checked. A bit of monitoring both could tell you the normal range of divergence. Again, this is free - no extra board space or cost.

Another use case is processor monitoring, The internal sensor is actually better at telling you the die temperature, if there's any chance of it overheating (whether or not there is an external sensor), If the board is sealed, just how hot does it get inside the box? If you are putting out a lot of current on a lot of outputs, is the processor heating up? (For weather you'd have the external sensor outside the box and wouldn't know the inside temp).

Yeah, it can be optional, but but the internal temperature is not worthless.

What is W3P? Is this some sort of CANBUS?

Some thoughts.

The last sleep study I had was simple - they sent me home with a recording device attached to two finger sensors; one was for heart rate and oxygenation, one was supposed to detect REM sleep (both had lights aimed at the skin or fingernail, did not appear to be accelerometers). I was surprised that they could detect REM sleep in that manner, but it would be worth looking into.

I wonder if one could detect movement with a cheaper BLE wristband, like the Xiomi MiBand? Not sure if anybody has cracked the OTA protocol. (OTA meaning generic Over the Air, not FOTA = Firmware/Flash Over the Air). I used to have one (before some firmware update from Xiomi bricked it), and it had a very long battery life and was small).

Building one's own wearable device - homebuilt and yet sturgy seems likely to be bigger and clunky; but maybe others can build tiny wearables.

Doppler Radar - I don't think my turning over in my sleep would result in a lot of motion towards/away the sensor from if mounted on the ceiling or high on the wall.

Antenna Effects (RCWL-0506) - might work if mounted close enough. Perhaps multiple devices. Worth considering. (These are NOT radar despite advertising, but do detect local movement).

One thing which might also work is a camera near the ceiling, attached to some image processing (NOT to a recorder!). Upside - it might be able to guess about movements of two people. In our case, we'd need to filter out cats who come and go all night (smaller areas?).

I recall reading a calculation which suggested that the LNA+PA version of the nrF24L01+ would not have a huge increase in range for a two-way connection with a regular transceiver; mainly because the receive side would not be much enhanced after SNR was taken into account (apparently the LNA was not that low noise, as the limiting factor for receiving). For one-way transmission from the PA unit or two way between two LNA+PA units, the improvement was much better . Was that post here?

Anyway, a higher gain antenna should help, as it increases signal strength and SNR in both directions. But is it true that higher gain is (almost??) always at the expense of a narrower emission pattern (kinda like using mirrors and lenses with a flashlight - the same power can be emitted weakly in all directions, or more strongly the smaller the focused beam)? And likewise for receive. For the example antenna in this thread, I believe the gain pattern is sort of toroidal - best at right angles to the antenna, less than unity along the axis. To get 9 db this would need to be a fairly flattened toroid (a stepped-on bagel). So in general, antenna gain is not an unambiguous "win", but rather involves a tradeoff which may or may not be worthwhile for a given system.

That can work well for a ranch style house which is mostly 2 dimensional, but for a multi story house the low transmit/receive gain along the antenna axis sometimes needs to be taken into account.

@tbowmo I agree with keeping things simple to implement, particularly on the Node & Gateway side. And it doesn't hurt that this would likely also keep things simpler to implement on the Controller side (ie: simpler plug-ins/adapters/drivers to write, which is a positive even when resources are not so tight.).

We might be able to nevertheless implement some kind of multi-value packet concept tho. I added a comment to the 3.0 doc about a simple and low overhead way to allow homogeneous arrays. That might or might not be worthwhile.

Robosensors's use case would involve some kind of tagged sub-unit approach, with multiple id/value pairs, yet another concept worth considering.

@tbowmo I made some related comments some while back when 2.0 was being developed, but then I disappeared from MySensors for many months (life does that). So I understand that it's too late for changes to that, and I am only planting seeds for the future, accepting that many will not sprout soon, or ever. The current spec is already a good effort.

The comments about more specific documentation can still be implemented, tho.

Perhaps I can help with comments on the binary protocol. The problem is that it has to map nearly 1:1 into the serial protocol, right? Or not?

I continue to be impressed with the quality of this community.

I've been gone for a while, but this seems like a nice next step. Feel free to move these comments to the forum if that make more sense. If it's too late for this round, perhaps they can be useful for the next revision.

1. I still think the controller (or if need be, the gateway) should do any neccessary conversion between metric and imperial units, rather than pushing it down to every node; it seems like some kind of violation of partitioning to push that UI customization concept (which in many controllers the user can toggle freely or even display both ways) down to the node level, where it adds unneeded complication. But I'll let that go.

2. It would be good to augment this with a systematic description of the underlying value type (int32, float32, string), and implied units if appropriate (potentially two units, for metric and imperial). For example, is wind speed in m/s or km/hr (assuming metric)? Is pressure in mmHg or hPA?

3. Aside from the type and implied units, there are a few hints given already - what 0 and 1 mean, an enumerated list of valid string values, color given a strings of hex digits, position as a multivalued structured string, etc. Keep those as notes in augmented docs as described in #1, and add more a needed (eg: IR commands)
   2A) Is there any logic to when enumerated values are encoded as integers, and when they are encoded as strings from a controlled vocabulary? Could we get by with one or the other, rather than this hybrid?

4. How do the semicolons internal to V_POSITION interact with the semicolons in the message field structure? If the same character is used for both, some note about that is needed. Or use commas in the former (substructured payloads) and semicolons in the former (overall message structure).

5. I'm glad to see that a distinction has been made between sensed light levels in Lux and uncalibrated light levels. Generalizing, I can see four types of possible reporting of such things. 1: uncalibrated results in arbitrary units (eg: ADC values of a LDR+resitor voltage divider), 2: scaled results 0-100%, 3: reporting in predefined units understood by both sides, like Lux, 4: reporting in uninterpreted units which can only be displayed but are not parsed. In the first case, I think the only thing to define for a given V_variable is whether brighter should be reported as higher or lower. This standard chooses 0-100% for V_LIGHT_LEVEL. Could't that be reported as V_PERCENTAGE instead? That is, use V_PERCENTAGE for any sensor reporting in scaled percentages.

6. If the protocol is going to allow for both uncalibrated and calibrated values (the latter in implied units), as with light levels, then I think it makes sense for the value with implied units to be the more specific variable; that is, V_LIGHT_LEVEL should be in lux, and V_LEVEL (or V_PERCENTAGE) would be used for the generic values, without units.

5A) V_LEVEL currently seems a little convoluted. It appears that the implied units depend upon which S_sensor the variable is associated with, which kind of breaks the pattern that units (if any) are implied by V_variable. And for S_DUST, S_AIR_QUALITY) there are no units. So for a calibrated light level in lux OR an uncalibrated dust level, use V_LEVEL but for an uncalibrated light level use V_LIGHT_LEVEL. V_LEVEL seems to be a grab bag of conceptually unrelated magnitude values, some with per-S_sensor implied units and some without. This could use rethinking. How about if V_LEVEL was used for any values which do not have implicit units (see #1 above), and V_PERCENTAGE was used if the value scales to 0..100%. Other V_variables could have pre-specified implicit units (metric and imperial).
5B) V_UNIT_PREFIX is really more of a suffix than a prefix. But this can work well with 5A above. If a value is reported as V_LEVEL then there are no interpreted or parsed units involved and the controller is not expected to be able to interpret or convert units. However, V_UNIT_SUFFIX could optionally supply a free text string with can be displayed after the value. So if you are reading a LDR and reporting the raw ADC value (or inverted value to ge the direction right), use only V_LEVEL. If you are reporting something in units not understood by the system, then use V_LEVEL and describe the units in V_UNIT_SUFFIX. (And if you are reporting percentage, use V_PERCENTAGE).

1. Semantically, there are two kinds of outputs: set-current-value which sets a new state to be retained, and trigger/command which performs some action once. It's kind of like edges and levels in hardware logic. Setting the same value twice is harmless, but issuing the same command twice may have different effects than sending it once. Most of the outputs are about set-current-value. However, some are commands - like sending an IR code.
   6A) What do V_UP, V_DOWN, and V_STOP mean? Unless they contain payloads to configure a (retained) movement speed or something, they seem more like commands/triggers rather than set-current-value. In that case, it makes more sense to have a V_SHADES variable which is set to 0,1,2 for stop, up and down (which are mutually exclusive), than to have three variables which could be set in parallel. Likewise with V_SCENE_ON and _OFF. These should be handled with one V_variable, much as sending a IR command. This is also easier to extend for an additional command to the same device - just define additional values to action associations, rather than needing to add a new V_variable.
   6B) (A shade controller which could set/report percentage opening could use V_PERCENTAGE)

2. Likewise there are two kinds of inputs: report-current-value and trigger-event. Mostly this protocol is about report-values. However the trigger functionality comes up in terms of V_TRIPPED, which is often implemented as a trigger. It should be made explicit what resets a triggered state: some systems use autoreset where after one report a device goes back to untriggered (trigger/event); other systems will keep reporting triggered forever until separately reset by some other means (report-current-state). We should be clear which mode this protocol expects.

It's good to see the ongoing refinement of MySensors and related initiatives! These are definitely meant to be helpful, not as criticism! Your work is much appreciated.

@gregl said:

isn't the key difference of the two that one 32u4 has a built in USB controller, whereas the 328p doesn't and relies on an external USB controller (FTDI usually.)

That's important, but there are many other differences vs ATMega328p. Offhand, there's an extra 0.5K RAM. Second 16 bit timer (Timer3) instead of 8 bit Timer2; plus high speed 10 bit Timer 4. SPI and TWI on different pin #'s (as seen by Arduino). More ADC pins, and a 10x/200x differential amplifier; different voltage ref (2.56v vs 1.1v). And various changes to interrupts or registers based on these. So some less-used libraries need porting, as well as changes to example code. Not sure how these differences affect the core of MySensors; you might have to try it out. There's a good chance that it would work, or that the changes needed would be minor.

The Leonardo is gone from the Arduino.cc product line, but the Micro is still sold so presumably the chip continues to be supported.

I was thinking to migrate to the Raspberry Pi (and try out various controller software).

However, I am hearing about problems with wear on SD cards; apparently the cards can fail after too many writes. There is no TRIM control as with SSD's, so there is limited ability to wear-level. So there are reports of periodic SD failures with 24/7 software running on a RPi. That would not be ideal for a home controller!

Has anybody had problems with this? Is some controller software better than others in this regard? (The problem can be reduced by using RAM disk (tmpfs) for highly volatile files).

@flopp said:

12v can be handle by the Arduino Pro min on RAW pin

Cars go well above 14V during operation (as well as having noise glitches on top of that). If the regulator is only good up to 12v, operation could be marginal. Hence the suggestion from @tbowmo of a couple of diodes in the power feed, to drop a bit of the voltage as seen by the regulator.

OK, let me see if I understand your overall flow.

You have your node connected to switched 12V power in the car. When you turn on the car, the uC gets power, when you turn off the car it loses power. When the car is cranking, the voltage drops enough that it tends to reset again (or else the cranking position on your key switch actually cuts off auxiliary power used by the uC).

So if you are at home and in range, loop() is called by the Arduino runtime just once; at the end of loop() the uC goes into an infinite busy loop rather than return, staying there until power is lost again. If you are away from home or the packets are lost for other reasons, resend() will cause a return from loop(), so that loop() will be called again by the Arduino runtime, so it will just keep trying until either power is lost, or it makes contact with your wireless network again.

Meanwhile every time it starts, the first byte of eeprom is incremented in setup(), and then loop() tries to fetch VAR1, and send VAR1+eeprom(0) back. If it succeeds, then the first byte of eeprom is zeroed.

Right?

That sounds reasonable. My immediate concern would be inconsistent operation during cranking, if the input voltage went just low enough to make the uC unstable. If your car's wiring cuts the power entirely, not a problem. Likewise there appears to be time for the uC to boot, and fully update eeprom, before losing power during cranking. (It's best not to be writing eeprom as voltage drops). It appears that those are not problems for your system tho.