@alowhum Sorry, I have abandoned this project. It didn't garner the interest I hoped it would, and I’ve moved on to other microcontrollers/radios/networks since then. This is hosted on GitHub’s free static hosting, so it should continue to work as long as they are in business, and it should work on devices as it was 5-6 years ago. But I won’t be supporting it and won’t continue to build on it. Glad to hear that you like it!

@user2684 Sorry, I'm not at a computer. IIRC, the issue I've been facing is that when I used the latest version of NodeManager with the latest version of MySensors dev branch, with signing and encryption, the gateway would crash on startup. This has been preventing me from proceeding.

Sorry for the lack of detail - I can't verify things at the moment. Looking forward to 1.6!

MAX_SENSORS defines the maximum number of child ids

How can I know how many child ids exist in total? Is there a way to arrive at it from the sensors? Sorry for my ignorance.

Fixed most of the other bugs you pointed out. Thanks!

@user2684 We will need to decide which version of MySensors to use. I've been using the latest version from the development branch. In fact, the ascii-art splash screen is the latest commit, as of 5 days ago. Since you aren't seeing the ascii-art, you are probably on a slightly older commit.

I'm not sure if we can start supporting from the current stable 2.1.1 onwards? I don't know about the advantages that the new driver for RFM69 brings. If it's any effort at all to get it to work with older versions, it's probably not worth it. I don't mind waiting till 2.2 is released.

I've simply copy-pasted the MY_RFM_* defines from somewhere in the MySensors examples. They are currently hardcoded :D and probably are just the defaults. I'll clean that up.

Thanks for pointing out the other issues. I'll get on it. Thanks!

@user2684 Thanks. That helped identify the problem. I've changed the key switch to inputSwitch to avoid clashes with C keywords, which is causing the issue. I had done this long ago, when I wasn't caring about data migrations. Looks like your data still has that old bit in it.

Sorry for the trouble. The easiest thing to do is to delete that value, and start afresh. I'll be more diligent about making modifications in the future.

@user2684 Thanks. If you haven't already deleted the app_data key, could you please share the value of that key with me? If you've already deleted it, do you remember which sensors you had configured across your nodes in your settings?

This was an error in the migration process, where the old data is ported over to the new format, if I make changes in the format of the data. I've obviously done something wrong in that porting logic.

@user2684 Woops, if you only see the loading message, I've dun goofed. Can you please look in your JS console (Ctrl+Shift+I) and tell me what is the error you see?

To fix the problem, you can delete your existing data. You can do that within the devtools itself. Click on the Application tab in the devtools, and in the left pane, under Storage, select Localstorage > https://rakeshpai.github.io, and delete the key called 'app-data'. Please make sure you copy your error messages before you do this though, so that I know what went wrong.

Sorry for the lack of detail. You can get the latest gateway sketch from the app itself at https://rakeshpai.github.io/mysensors-network-manager/

The sketch is just a barebones sketch, wiring up the before, presentation, etc. functions to node manager. The config.h is as follows, though nothing stands out to me as being out of place.

#ifndef config_h
#define config_h

// Message signing
#define MY_SIGNING_SOFT
#define MY_SIGNING_SOFT_RANDOMSEED_PIN 7
#define MY_SIGNING_REQUEST_SIGNATURES


/**********************************
 * Sketch configuration
 */

#define SKETCH_NAME "Gateway"
#define SKETCH_VERSION "1.0"
#define MY_REPEATER_FEATURE


/**********************************
 * MySensors node configuration
 */

// General settings
#define MY_BAUD_RATE 9600
//#define MY_DEBUG


// NRF24 radio settings
#define MY_RADIO_NRF24
#define MY_RF24_ENABLE_ENCRYPTION
#define MY_RF24_CHANNEL 76
#define MY_RF24_PA_LEVEL RF24_PA_LOW
//#define MY_DEBUG_VERBOSE_RF24


// Serial gateway settings
#define MY_GATEWAY_SERIAL


/***********************************
 * NodeManager configuration
 */

// if enabled, enable debug messages on serial port
//#define DEBUG 1

#define POWER_MANAGER 0
#define BATTERY_MANAGER 0
// if enabled, allow modifying the configuration remotely by interacting with the configuration child id
#define REMOTE_CONFIGURATION 1
// if enabled, persist the configuration settings on EEPROM
#define PERSIST 0
// if enabled, a battery sensor will be created at BATTERY_CHILD_ID and will report vcc voltage together with the battery level percentage
#define BATTERY_SENSOR 0
// if enabled, send a SLEEPING and AWAKE service messages just before entering and just after leaving a sleep cycle and STARTED when starting/rebooting
#define SERVICE_MESSAGES 0

// Enable this module to use one of the following sensors: SENSOR_ANALOG_INPUT, SENSOR_LDR, SENSOR_THERMISTOR, SENSOR_MQ, SENSOR_ML8511, SENSOR_ACS712, SENSOR_RAIN_GAUGE
#define MODULE_ANALOG_INPUT 0
// Enable this module to use one of the following sensors: SENSOR_DIGITAL_INPUT
#define MODULE_DIGITAL_INPUT 0
// Enable this module to use one of the following sensors: SENSOR_DIGITAL_OUTPUT, SENSOR_RELAY, SENSOR_LATCHING_RELAY
#define MODULE_DIGITAL_OUTPUT 0
// Enable this module to use one of the following sensors: SENSOR_DHT11, SENSOR_DHT22
#define MODULE_DHT 0
// Enable this module to use one of the following sensors: SENSOR_SHT21
#define MODULE_SHT21 0
// Enable this module to use one of the following sensors: SENSOR_SWITCH, SENSOR_DOOR, SENSOR_MOTION
#define MODULE_SWITCH 0
// Enable this module to use one of the following sensors: SENSOR_DS18B20
#define MODULE_DS18B20 0
// Enable this module to use one of the following sensors: SENSOR_BH1750
#define MODULE_BH1750 0
// Enable this module to use one of the following sensors: SENSOR_MLX90614
#define MODULE_MLX90614 0
// Enable this module to use one of the following sensors: SENSOR_BME280
#define MODULE_BME280 0
// Enable this module to use one of the following sensors: SENSOR_SONOFF
#define MODULE_SONOFF 0
// Enable this module to use one of the following sensors: SENSOR_BMP085
#define MODULE_BMP085 0
// Enable this module to use one of the following sensors: SENSOR_HCSR04
#define MODULE_HCSR04 0
// Enable this module to use one of the following sensors: SENSOR_MCP9808
#define MODULE_MCP9808 0

#endif

I'm trying to compile a serial gateway, no sensors, against the development branch of NodeManager, which works fine. However, when I open the serial monitor, I get the following output:

$ pio device monitor --port /dev/ttyUSB2
--- Miniterm on /dev/ttyUSB2  9600,8,N,1 ---
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
 __  __       ____
|  \/  |_   _/ ___|  ___ _ __  ___  ___  _ __ ___
| |\/| | | | \___ \ / _ \ `_ \/ __|/ _ \| `__/ __|
| |  | | |_| |___| |  __/ | | \__ \  _  | |  \__ \
|_|  |_|\__, |____/ \___|_| |_|___/��

...and it hangs.

Any idea what the issue might be?

PS: As a self-confessed connoisseur of ASCII-art, I tip my hat in your general direction. Well done, good sir.

Pretty big update: Added the ability to select the board that the node/gateway is based on. Supported boards are: Pro Mini (5v and 3.3v), Nano and Uno for the atmega328 boards, and NodeMCU and D1 Mini for the esp8266 boards. Please let me know if your favourite board isn't listed here.

The selection of the board effects the list of available pins for sensors. The sense pin dropdown for sensors now shows the appropriate pins (analog/digital/interrupt pins) based on the board selected for the node and the type of sensor being configured.

The selected board also affects the platformio.ini file generated, which makes PlatformIO integration even more enjoyable to use. No need to specify the board during the build.

This was a pretty big update, including changes to the underlying internal datastructure itself, so it's also an acid test for the data migration and versioning system implemented before. If you don't see any errors when you refresh after the update, it means everything worked transparently, and that's great news! (The migration mechanism hasn't been tested so far, so please let me know if you see errors when you get the update.)

Also implemented a reporting interval setting for analog sensors, to bring it in line with the latest changes in NodeManager.

We're getting closer to being release-ready!

EDIT: Also, this project crossed 100 commits in GitHub already! Wow!

@user2684 The clear separation between sleeping and reporting definitely makes things easier to understand and use. Thanks for doing this.

I've now published a release with the changes for sleep time and reporting time in place. Could you please give it a spin? https://rakeshpai.github.io/mysensors-network-manager/ You may need to delete your existing sensors first.

• Sleep config is as before. Nothing's changed there.
• New reporting interval fields added. Only applies to analog sensors ATM.
• There's additional messaging for the reporting interval fields to clarify what happens when the node is sleeping.
• The generated code has been updated.

I haven't implemented a global reporting time. I felt it would make the UI complicated. I might be mistaken, but it isn't relevant for all types of sensors, isn't it? For eg. there's not much to sense by polling a motion sensor, or a switch. So, I felt like it's best if we ask for it only where relevant, and not out of context. I'm open to change this though. Please let me know what you think.

Just want to be sure I understand the changes, so please bear with me.

When a node is awake, each sensor can have its individual reporting times (or a global reporting time), which need not have any relation to anything else. However, if a node is sleeping, its reporting time has to be a multiple of the sleep interval of the node. Is this understanding correct?

If so, I'm thinking of the following UI:

• If a node is not battery powered, the UI can let you set the reporting interval on individual sensors. Even though setting reporting interval on every sensor gets repetitive, it might be the easiest to use. The alternative would be to have a shared global setting influencing the reporting interval, but that gets tricky to convey in the UI.
• If a node is battery powered, the user will have to set a sleep interval. We could then use one of two approaches: (1) The user can't set the reporting interval for sensors at all, in which case the sleep interval is used as the reporting interval by default, or (2) We can let the user set a reporting interval, but only if it's a multiple of the sleep interval. Either one works for me, the second seems most flexible.

Is my understanding correct, and would this kind of a UI implementation do the job?

Sorry for the silence. I didn't realise I wasn't getting notifications about this thread. Thanks for the updates. I'll go over your comments and issues and start making changes accordingly. I'll update here once I have something. Thanks again. Woohoo!

Sorry for the long silence. it might look like I've been slacking off (and that's mostly true), but I've also been rolling out some updates:

• Code can now either be downloaded for the Arduino IDE or for the PlatformIO IDE. The PlatformIO code generates additional config files to use their dependency manager automatically. This means, you don't have to download and install libraries manually. Just open in PlatformIO and build the project. It's magical.
• Also, you can now download a backup zip of everything. It contains code for the entire network and all it's nodes, and a network.json file. I haven't implemented a 'restore' functionality yet (coming soon), but the network.json file contains everything needed for a restore. The network.json is the internal state data-structure used by the app.
• Couple of minor performance improvements, especially to help with the first-ever-visit scenario. (All subsequent user actions are blazing fast anyway.)

To clarify, while backups are a great idea, you don't need to save anything. Stuff is still automatically saved in the browser's storage anyway during regular use, and is persisted even if you close the browser. Backups are for the assurance of having a neat little zip containing everything on your hard drive, just in case things go horribly wrong.

Very little has changed visually. To find these new features, launch the app, and click the 'hamburger menu' at the top right of the page.

https://rakeshpai.github.io/mysensors-network-manager/

@user2684 Thanks for the update. Much appreciated.

The thing is, I think it'll be really great if this project can come to a point where I can confidently tell people that they can flash the output code to their devices, at least in a test setup, so that I can start to get some usage-based feedback. So far, I've very vocally said that the output isn't ready, and that's definitely limiting usage.

So, I wouldn't say it's urgent, but user-feedback is important, so I'd like to start making releases that I can ask people to try.

@dbemowsk Thanks! It's very encouraging to know that this is useful.

I've been thinking about the libraries problem. The easiest-to-use solution is to put it in the zip, so I'll probably just ultimately choose that. Problem is the way Arduino looks up dependencies. It has to be globally installed, or put in a flat directory structure in the local folder. Both of these are unappealing solutions.

I'll be adding the ability to compile with PlatformIO soon, which has a built in dependency manager, but that's really just an alternative, not a solution.

This tool is built on top of NodeManager, so if you need a new sensor, NodeManager is the right place to contribute. There will be a lag between when NodeManager implements something and I make it available here, but I could prioritise based on demand. I haven't considered the ability to add custom sensors by yourself directly into the UI. I'll think about it. I suspect it might make things too complicated though.

Fields for notes and stuff is a great idea! Thanks for suggesting. I'll add this to the UI soon. Adding an ability to import and export all your data is on the cards. I completely agree that having an archivable dump of everything is very important for backups. (Would things like Dropbox integration be interesting as well?)

Noted about the floor plan thing. I'll wait for more votes for it. I've always thought it would be cool too, but I've felt that it's a lot of effort to implement, then a lot of effort to configure correctly, and finally too little to gain from it. But then, if there's interest, why not!

(PS: Speaking of diagramming, I've given some superficial thought to having a feature where it generates a schematic of the circuit, based on your configuration. What do you think?)

@pjr Noted. Thanks. Are there a lot of people using RS485? (I have no idea, and I'm constantly surprised by how people use MySensors.) My only concern is that I don't want to add too many features that people won't use, since it would make the UI confusing. But if it's a popular feature, it's definitely on the to-do.

@Anticimex Haha. No, it doesn't come with the IDE, and you have to install it separately. It's required if you want to use their Cloud IDE. Besides, it only validates and executes commands from their server, and not from anywhere else.

You still need to have a lot of trust in their server when using their cloud IDE - they appear to indirectly effectively have local shell access with your user privileges when using their cloud IDE, and their installation process sets up additional udev rules for serial access.

Also, I'm no security expert, so don't take my words at face-value - I don't want to blame them incorrectly. I might very well be wrong in my assessment.

I'll be stripping out the code I've written, so here's a screenshot for posterity. Serial comms in a browser!

tl;dr: Arduino Create Agent is only for the Arduino Create site. Everyone else is locked out. We're back to square one, and we'll need to implement a native app on the desktop.

I've hit a dead-end with Arduino Create Agent.

Firstly, I got bad vibes when I was using their API. It felt very kludgey, and it was very hard to do very simple things. API troubles aren't that big a deal though, and soon enough, I had a decent reliable wrapper on top of their API. I was able to successfully list serial ports, connect to one, and talk over serial to it. I also found a lot of undocumented reliability issues along the way, which I had to fix by trial and error, but in the end it was pretty solid. It felt great to see MySensors' Serial.print messages in the browser! :metal:

However, I hit a complete road-block when trying to upload firmware. They require that every upload be signed, and they verify signatures. About signatures, their readme says that the signature should be:

(...) signed with the private key that matches the public key contained in the config.ini of the arduino-create-agent

However, the config.ini doesn't have any field allowing for public keys to be put in there. Also, their code for verification uses a hard-coded public key anyway. Presumably the private key lies on their servers.

If I understand correctly, this basically locks out everyone but Arduino from using Arduino Create Agent to upload firmware.

Aside: It's the upload 'commandline' that requires signatures, not the firmware itself, which is very strange. Also, the upload command-line is literally the shell command used to invoke avrdude. It doesn't sound like a good idea at all to take a shell command as input from a web server, via a browser, and execute it locally, signed or not. See what I mean when I say their API is kludgy? This isn't the most massive security loophole though, since they cryptographically verify the command's signature, and this is all over SSL anyway so MITM isn't possible, but I don't think I would have implemented it like this.

Can't fault Arduino for the agent not working with our site. There's no reason they would have designed for third-parties using their app to flash Arduinos. But it does mean that we are locked out.

Now, I can file bugs, and if they are interested they could address them, but even if they agree that this is a good idea, it's unlikely that this would be a priority for them. We can't wait for them, but if/when they are ready, we could move over to using their plugin. Until then, we are on our own.

Onwards!