Modular sketch to be configured with JSON (idea)

[monte]

I've never used FOTA with mysensors, but have experience with uploading firmware over WiFi to esp8266 in some of my project and realize the benefits of this. I remember reading discussions on nRF51/52 that there is no way currently to OTA updates and people were tying to come up with some other ways, which doesn't require special bootloader.
Today it came to my mind, that it is possible to write modular firmware for Mysensors nodes with some specific functions, i.e. Relay, Dimmer, Temp/Humidity sensor, basically anything that's in the examples section. And configuration would be done with JSON. This way it will be easier for newbies to make simple Mysensors nodes and will help with updating nodes without reflashing them once they are installed in remote locations.
I don't know if it would be easier than writing needed bootloader to do thing more common way, but maybe this method is something we can consider as way of improving the framework overall.
Would like to hear what you think about this.

[mfalkvidd]

@monte with the large flash space on the nrf5, what you are suggesting should be quite feasible. I have no experience with NodeManager, but from what I've understand it is a nice abstraction. Maybe NodeManager can be used to also switch between different features in a formware? Or maybe it is possible to draw inspiration from the NodeManager to create something new, without having to start from scratch.

[monte]

@mfalkvidd I neither have any experience with NodeManager. I shall look into it, but I expect it will be too big addition to merge.But who knows.
Do you know who maintains this project, maybe we can hear his thoughts on the matter?

[mtiutiu]

@monte

Are you suggesting to implement something like esphome has? If that's the case then I would prefer yaml instead of json as it's easier to read and maintain imho.
The idea to have code generated from a configuration file is pretty neat and it simplifies tedious and repeating tasks a lot besides abstraction.
NodeManager should help as it seems to be pretty modular.

[mfalkvidd]

NodeManager was created by @user2684

[user2684]

@monte first of all I like the idea of having a multi-purpose sensor which can embody different "personalities" on the fly. NodeManager came in exactly for the purpose of providing a simple to configure, modular firmware with a set of predefined sensor ready to use.

BUT this is defined at compilation time not at runtime as you are suggesting. Reason is simple: since there are 60+ ready-to-use sensors in NodeManager, accommodating everything especially in a small arduino is not feasible, also considering the dependencies some sensors bring in. I see usually up to 3-4 sensors can fit the flash, no more. But I expect in different environments the situation could be different or at least for a subset of those.

All of this to say I believe NodeManager is a good starting point for what you are intended to do since you already have modular code for a good number of sensors, hooks for the different phases of the lifecycle and a way for communicating with the node through a "service" channel if you need to send something and act upon it. So feel free to fork the project (preferred way) or start from there.
All of NodeManager's details can be found on https://github.com/mysensors/NodeManager
Thanks

[kimot]

Super...
You add software for relay, switch or new temperature sensor through FOTA...
And then go to your node, remove it and connect this devices to it in fact.
No, you really do not need FOTA for this.
But for repairing some bugs in your program yes.
Other case is for example node ID, mqtt topic, some calibrations, offsets, Domoticz sensor IDX etc.

[boozz]

@kimot @monte
It appears to me as what has been described by @monte is almost completely covered by the Tasmota project: https://github.com/arendst/Tasmota

[monte]

@boozz yes, exactly I had this in mind, but with specifics of Mysensors framework. Not everyone wants to rely on WiFi with their home automation.

@mtiutiu I mentioned JSON because I've used JSON library for Arduino and know that it exists, it well may be YAML if it's more suitable.

@user2684 said in Modular sketch to be configured with JSON (idea):

BUT this is defined at compilation time not at runtime as you are suggesting. Reason is simple: since there are 60+ ready-to-use sensors in NodeManager, accommodating everything especially in a small arduino is not feasible, also considering the dependencies some sensors bring in. I see usually up to 3-4 sensors can fit the flash, no more. But I expect in different environments the situation could be different or at least for a subset of those.

We are talking about nRF52 use case, which has abundance of available flash space. And my thought was a bit different then you described.
Imagine you have base firmwares for most sensor types, or it can be generated with NodeManager. But key parameters are configured with externally loaded JSON (or YAML) file, so it can be modified after flashing initial firmware.
I'm not completely sure how much use does this solution has, that's why I started discussion here, to see if anyone thinks it would be a good idea.
Thanks for your reply

[Michał Szura]

I personally would love that
I still begging my friend for code to some simple nodes where I want to avoid esphome. I use mysensors only through cables with serial and planning on rs485 or maybe pjon if tests are finished.
The yaml support will be nothing new for homeassistant user and probably.not much learning for others.

[user2684]

@monte said in Modular sketch to be configured with JSON (idea):

Imagine you have base firmwares for most sensor types, or it can be generated with NodeManager.

I don't see specific technical limitations to have this working. On the NodeManager side would be:

• Include in the sketch all the sensors you want, which is already there
• Have a logic to configure sensors remotely, which is mostly there through SensorConfiguration (https://github.com/mysensors/NodeManager/blob/master/sensors/SensorConfiguration.h). If then the configuration is through individual messages or json, it makes little difference. There might be some configuration settings missing since not all are available OTA
• Implement a logic for configuring only specific sensors. Shouldn't be that difficult, probably best is to have a enable/disable switch for each sensor (https://github.com/mysensors/NodeManager/issues/500), having all the sensors disabled by default and then dynamically enabling them

Bottomline not something to put in place in a few minutes but all the building blocks are there. I do not have at this time spare cycle to spend unfortunately but feel free to start looking at it if you like.

[user2684]

@monte, I've implemented in PR https://github.com/mysensors/NodeManager/pull/517 something going in the direction you pointed out. Before explaining just a simple assumption first: NodeManager is intended to run on a number of different boards, most of them with limited memory so this capability has to take this constraint into consideration (hence no json parsing, reuse of existing communication mechanism, capability disabled by default, etc.)

Apart from this, I found a sort of compromise to enable/disable sensors, even remotely and optionally persisting the status across a reboot. All the implementation details are within the PR (down below, the PR also include other enhancements) feel free to provide comments here or on Github. Hope it could be useful to avoid reimplementing the entire logic from scratch