About the architecture: thin vs thick gateways
-
I wanted to start fresh with my current thoughts on this, without reviwers needing to read through an old thread with many false trails later revoked.
First a term: the child/sensor is the functional unit (generally a sensor or actuator) which is addressed by a child id within a node, Often just called a "sensor" but that term is ambiguous.
I see the current architecture as being basically a way for a controller to access a set of child/sensors. Each child/sensor can report its type on startup, from a list originally derived from the Vera implementation. Each child/sensor can have several values (eg: int or floating numbers) which are reported to the controller (typical of sensors), or set by the controller (typical of actuators); each value within a child/sensor has a type which is sent in each report or set command (you could consider this a hybrid of metadata and another layer of addressing below the child id level).
The current concept of a gateway is of a minimalist translator between Over The Air (OA) RF packets and some serial or MQTT packet analog which is understood by the contoller (perhaps specifically by some plugin or extension of the controller.
This is the part which most concerns me, because a lot of the functionality of the wireless sensor network needs to be implemented for each controller. But controller code is written in a variety of languages by a variety of people as a lot of independent products, which are going to be hard to keep maintained in parallel. If past experience is a guide, the person or team implementing a given controller's MySensors interface may not remain intact and available, so some controller interfaces are likely to fall behind, get out of sync. It's not easy for volunteer developers to jump between controllers because each has its own architecture (and language) with start up costs, and people will be less motivated to maintain controller interface code for controller software (or hardware) they are not using. So keeping multiple controller interfaces in sync as the system evolves is one of the key challenges.
For this reason I tend to favor a thicker (less minimalistic) gateway. The concept is there all of the functionality common to all implementations and which does not inherently need to be implemented in a controller, should be moved to this thick gateway, and can be implemented once rather than being reimplemented inside each controller.
That could include node assignment and network functionality. It could include unit conversion if need be (F and C - that was previously external to the controller but moved all the way to the end node, then more recently moved to the controller, but handling it in a thick gateway would be another option). Is there any other common logic which needs duplicated in each controller, but which could share a single implementation in a thick gateway?
With a thick gateway, substantial protocol changes could sometimes be implemented in just the (thick) gateway and the sensactuator nodes, without need to modify every controller's code as well. These could be done in C/C++ for the Arduino environment and within the MySensors architecture (not needing knowledge of the internals for each controller).
The controller interface would only be responsible for the functionality which inherently needs to be handled within the controller. For example, mapping child/sensors and values to controller concepts and objects.
A thick gateway might require another $10 - $15 per system in more capable hardware (or it might not). That's not going to be the main cost factor in the overall sensor network. (Keeping the protocol simple enough to be implemented on cheap uC's at the nodes is still a good thing, because there could be a lot of them)
I'm not in any way insistent on this being adopted - I just want to put it out there clearly. As MySensors evolves AND is interfaced to multiple controllers by diverse volunteers and weakly maintained legacy controller interface code accumulates, we'll all see in a few years whether the approach suggested here starts looking better or not.
-
@Zeph I am pretty sure you are re-doing (an) earlier discussion(s) here but...
Although I could agree on your argument about maintaining code of a controller plugin (I myself can't be as fast as I want to be) I am confident that with the growth of MySensors you will see more volunteers adding code to their plugin of choice as long as the plugin is open source.
-
Yes, i'm trying to summarize earlier long threads in one simple distillation that will be easier to read when and if there comes a time when this approach needs to be revisited.
This history of open source is full of partially maintained or unmaintained projects, as the creators of something like a plug-n move on. Many of the users here are not plug-in developers.
And maybe you are right, and there will forever be active development teams for the plugins for every controller interfaced to MySensors. Time will tell, and I will be happy if that happens. This thread was meant to be an easy to digest summary of an alternate approach if that doesn't happen.
-
@Zeph: I have been busy implementing a Controller and must say things are not too difficult, apart from ... mapping the current node-id, child-sensor-id, message-type and presentation to the device - value structure of the Controller. See in this light also my contribution to the Unit Standardization (2.0) thread.
While in the past I saw something in a "thick gateway" I currently do not see how that can help reducing the implementation effort on the controller.
-
OK, that's good feedback. Good luck with that.
-
I've just implemented a controller for fhem and must say a 'thick' conjtroller would have made implementation noticibly easyer. It's not about parsing the protocol and pulling the values into suitable entities, it's about implementing the low-level details (managing ID-requests, inclusion-mode, acknowlede and retransmit, ping etc....). And only half of this effort was due to things being not propperly documented.
On the other hand a really 'thick' gateway only makes things really easy when implementing a standard interface (like mqtt) that allows to just access sensor-data without having to deal with any of the node-management stuff.
- Norbert
-
Yes, know there is a bit of lacking documentation
Inclusion mode is optional for controller to make use of. Presentation messages is always passed to controller. If controller wants it can choose to only allow adding device while inclusion is running.
If gateway should handle id-handout by itself we would also need some kind of reset/clear functionality in the serial protocol to free up ids etc. The id statuses must be stored in gateway eeprom.
Handling multiple parallell retransmits automatically by gateway (and nodes) could make it impossible to run it on a ATMega329 [speculation]. But if you got some neat ideas I'm open for suggestions (or pull requests) .
If you decide to help out with something please use the development-branch.
