Browser-based firmware generator
-
@rakeshpai I should mention that when I look at the zip, it is quite big (235KB) but when I open it from a explorer window it appear empty. If I try another tool like 7-zip, it opens. But the root folder is "_" and I suspect windows does not like that. Under that folder the gateway code is. So I think it is just a bug in the directory tree fabrication which is windows zip incompatible.
@Anticimex Ah, that makes sense. Based on this, I've rolled out an update. I don't have a Windows box to test, so I'd be awesome if you could have a look once again. Don't forget to do the double-refresh or the hard-refresh I mentioned above.
-
@Anticimex Ah, that makes sense. Based on this, I've rolled out an update. I don't have a Windows box to test, so I'd be awesome if you could have a look once again. Don't forget to do the double-refresh or the hard-refresh I mentioned above.
@rakeshpai Works just fine now. Great work! :+1:
-
Also, it looks like you generate the beta-personalizer, so just switching to PERSONALIZE_ATSHA204A instead of PERSONALIZE_SOFT should do the trick. If you decide to add the ATSHA "checkbox" option that is.
The only other change needed would be to set the appropriate signing flags and requirement flags. I could not see that you enable any such in the gateway it generated. Only encryption. So no signing functionality would be enabled. -
Yes, the browser downloads the SecurityPersonaliser from the development branch of GitHub. The choice of the development branch was accidental ;) but if that's the way things are headed, I'd rather stick with it.
There's very few modifications made to the personalizer. The changes are made here. Basically, the three keys are set, and PERSONALIZE_SOFT is defined. I say this just so that there are multiple eye-balls looking at the code. :)
I haven't worked with ATSHA, and I don't forsee myself doing so soon, so I'll need to understand the process before I make changes. If I understand correctly, I won't need to generate a device ID at all. The user will have to download and flash the personalizer on the arduino, and optionally (if they want whitelisting) read the generated key from the serial output. Is that correct?
I realise now that I haven't thought about adding whitelisting features to this yet. If you have any suggestions about how this can be made easy to use, preferably to the point of being an entirely transparent process, that would be awesome. Meanwhile I'll keep staring at your article on signing, hoping to soak more of it in. :)
Yes, I also haven't added any of the signing defines. That's why I have the huge disclaimers. :) Here's what I glean at the moment, please correct me if I'm wrong:
#define MY_SIGNING_SOFT // or MY_SIGNING_ATSHA204 #define MY_SIGNING_SOFT_RANDOMSEED_PIN 7 // or MY_SIGNING_ATSHA204_PIN 4 #define MY_SIGNING_REQUEST_SIGNATURES // optional whitelisting #include <MySensors.h>Is this correct? Thanks for looking at this.
-
Yes, the browser downloads the SecurityPersonaliser from the development branch of GitHub. The choice of the development branch was accidental ;) but if that's the way things are headed, I'd rather stick with it.
There's very few modifications made to the personalizer. The changes are made here. Basically, the three keys are set, and PERSONALIZE_SOFT is defined. I say this just so that there are multiple eye-balls looking at the code. :)
I haven't worked with ATSHA, and I don't forsee myself doing so soon, so I'll need to understand the process before I make changes. If I understand correctly, I won't need to generate a device ID at all. The user will have to download and flash the personalizer on the arduino, and optionally (if they want whitelisting) read the generated key from the serial output. Is that correct?
I realise now that I haven't thought about adding whitelisting features to this yet. If you have any suggestions about how this can be made easy to use, preferably to the point of being an entirely transparent process, that would be awesome. Meanwhile I'll keep staring at your article on signing, hoping to soak more of it in. :)
Yes, I also haven't added any of the signing defines. That's why I have the huge disclaimers. :) Here's what I glean at the moment, please correct me if I'm wrong:
#define MY_SIGNING_SOFT // or MY_SIGNING_ATSHA204 #define MY_SIGNING_SOFT_RANDOMSEED_PIN 7 // or MY_SIGNING_ATSHA204_PIN 4 #define MY_SIGNING_REQUEST_SIGNATURES // optional whitelisting #include <MySensors.h>Is this correct? Thanks for looking at this.
@rakeshpai
So far so good.
Yes, you don't need a device ID for atsha. It already has one.
The process of personalization is identical for the rest. Only the PERSONALIZE_SOFT flag has to be replaced.For whitelisting, you will have to somehow collect all nodes ID:s (therefore these should be statically set and not assigned during runtime) as well as the device ID of all nodes. Then you can fabricate a whitelist for the GW (or node) in question.
I think in addition to that, it would be prudent if the whitelist could be customized. So that you actually get a list of all the nodes from which you just can pick/check the nodes you want in the whitelist of a particular node (or GW).In it's simplest form, only the GW would have a whitelist, as the GW is "trusted", and the nodes are not.
It is only in very rare occasions that a node would need to speak signed messages directly to another node, and I also don't see a usecase for a node having to use a whitelist to trust a GW.
The message is already signed with a preshared key so a "rogue" GW should not be able to communicate as it wouldn't know the key. But in case someone stole a node, and turned it into a GW this might still be desired to have. But you would probably notice a second GW showing up and starting talking to your nodes :)The pins (random seed and atsha204) probably need to be customizable in the UI as it is HW dependent.
-
Yes, the browser downloads the SecurityPersonaliser from the development branch of GitHub. The choice of the development branch was accidental ;) but if that's the way things are headed, I'd rather stick with it.
There's very few modifications made to the personalizer. The changes are made here. Basically, the three keys are set, and PERSONALIZE_SOFT is defined. I say this just so that there are multiple eye-balls looking at the code. :)
I haven't worked with ATSHA, and I don't forsee myself doing so soon, so I'll need to understand the process before I make changes. If I understand correctly, I won't need to generate a device ID at all. The user will have to download and flash the personalizer on the arduino, and optionally (if they want whitelisting) read the generated key from the serial output. Is that correct?
I realise now that I haven't thought about adding whitelisting features to this yet. If you have any suggestions about how this can be made easy to use, preferably to the point of being an entirely transparent process, that would be awesome. Meanwhile I'll keep staring at your article on signing, hoping to soak more of it in. :)
Yes, I also haven't added any of the signing defines. That's why I have the huge disclaimers. :) Here's what I glean at the moment, please correct me if I'm wrong:
#define MY_SIGNING_SOFT // or MY_SIGNING_ATSHA204 #define MY_SIGNING_SOFT_RANDOMSEED_PIN 7 // or MY_SIGNING_ATSHA204_PIN 4 #define MY_SIGNING_REQUEST_SIGNATURES // optional whitelisting #include <MySensors.h>Is this correct? Thanks for looking at this.
Thanks for the comments, @Anticimex. I've now deployed a new build with the signing changes we discussed about, without the whitelisting. You can choose ATSHA or SoftSigning, and can specify the signing pin. You'll need to refresh and delete your existing network for the changes to take affect.
Sorry about the trouble with refreshing and deleting. I'll smooth out this experience once this has more users.
Regarding whitelists, I must add, if there's interest, I would like to develop a browser extension to flash the device directly from the site. If we have a browser extension that could talk to the device, we could read the key off serial. If we combine that with instructions to flash the gateway last, we might have simplified whitelist creation.
Also, I was hoping that if a user specifies that the node isn't battery operated, it would be configured to run as a repeater. I'm not sure how whitelists would work in such a scenario.
As an aside: The flash-directly-from-the-browser thing would require a server-side thing to do compilation - can't run avr-gcc in the browser yet. (Someone should port avr-gcc to llvm, so I can run it as WebAssembly in the browser! Very soon, I'm sure. :))
-
Nice but are you sure that "security signing" is enabled? In sources I've found only
#define MY_RF24_ENABLE_ENCRYPTIONnot
#define MY_SIGNING_ATSHA204
#define MY_SIGNING_REQUEST_SIGNATURES@bilbolodz I just updated the UI. Please do a hard-refresh (ctrl+r in chrome), delete your existing network, and try again.
In general, you shouldn't trust the output code yet. I haven't even run it on an arduino yet. The idea is just to give you a feel for the goals of the app, and find out if people find this useful/interesting.
Bugs reports are definitely welcome though. A comment on this thread works just fine.
-
Thanks for the comments, @Anticimex. I've now deployed a new build with the signing changes we discussed about, without the whitelisting. You can choose ATSHA or SoftSigning, and can specify the signing pin. You'll need to refresh and delete your existing network for the changes to take affect.
Sorry about the trouble with refreshing and deleting. I'll smooth out this experience once this has more users.
Regarding whitelists, I must add, if there's interest, I would like to develop a browser extension to flash the device directly from the site. If we have a browser extension that could talk to the device, we could read the key off serial. If we combine that with instructions to flash the gateway last, we might have simplified whitelist creation.
Also, I was hoping that if a user specifies that the node isn't battery operated, it would be configured to run as a repeater. I'm not sure how whitelists would work in such a scenario.
As an aside: The flash-directly-from-the-browser thing would require a server-side thing to do compilation - can't run avr-gcc in the browser yet. (Someone should port avr-gcc to llvm, so I can run it as WebAssembly in the browser! Very soon, I'm sure. :))
@rakeshpai Great!
Repeaters are completely signing agnostic. If they receive a signed message not addressed to them, they just forward it as is. So they don't need to carry whitelist entries to validate messages that are not directed directly to them. -
Thanks for the comments, @Anticimex. I've now deployed a new build with the signing changes we discussed about, without the whitelisting. You can choose ATSHA or SoftSigning, and can specify the signing pin. You'll need to refresh and delete your existing network for the changes to take affect.
Sorry about the trouble with refreshing and deleting. I'll smooth out this experience once this has more users.
Regarding whitelists, I must add, if there's interest, I would like to develop a browser extension to flash the device directly from the site. If we have a browser extension that could talk to the device, we could read the key off serial. If we combine that with instructions to flash the gateway last, we might have simplified whitelist creation.
Also, I was hoping that if a user specifies that the node isn't battery operated, it would be configured to run as a repeater. I'm not sure how whitelists would work in such a scenario.
As an aside: The flash-directly-from-the-browser thing would require a server-side thing to do compilation - can't run avr-gcc in the browser yet. (Someone should port avr-gcc to llvm, so I can run it as WebAssembly in the browser! Very soon, I'm sure. :))
@rakeshpai Looks good with the ATSHA option. One thing to consider is that we have "official" HW that comes predefined (the SenseBender GW for examples). Currently, it seem your tool (which is incomplete, I know) only seem to support AVR boards from looking at the pin options. At some point, you could perhams have a HW device list to pick from to specify what kind of HW the gw or node execute on. And in the case of the SenseBender GW (there could be others) the ATSHA pin is already set by the Arduino environment and should not be overridden by user config. Consider this just FYI right now. The tool looks really promising for people uncomfortable with modifying code.
-
Noted. Right now, the user will have to specify the ATSHA pin correctly for the SenseBender, so I guess it's not the end of the world, but I understand that it can be made easier.
There's actually a little bit of an understanding of the board already - if you choose an ESP8266 gateway and decide to add sensors to it, you'll see pin numbers matching the ESP8266's pinout. That's pin configuration in a different sense though. Let me think about how I can incorporate this.
Thanks for the kind words. :)
-
Noted. Right now, the user will have to specify the ATSHA pin correctly for the SenseBender, so I guess it's not the end of the world, but I understand that it can be made easier.
There's actually a little bit of an understanding of the board already - if you choose an ESP8266 gateway and decide to add sensors to it, you'll see pin numbers matching the ESP8266's pinout. That's pin configuration in a different sense though. Let me think about how I can incorporate this.
Thanks for the kind words. :)
@rakeshpai ah, ok. But the list of pins does not really make sense for the SenseBender gw since it is based on SAMD and not AVR.
-
@rakeshpai I personally think this is just amazing! When I wrote NodeManager I tried to make the world of MySensors just a step closed to a user who is not necessarily a programmer, this is doing the other 99 steps ;-) I'm sure there will be a lot of interest around this project
-
@rakeshpai I personally think this is just amazing! When I wrote NodeManager I tried to make the world of MySensors just a step closed to a user who is not necessarily a programmer, this is doing the other 99 steps ;-) I'm sure there will be a lot of interest around this project
Thanks, @user2684. @hek tells me that this is not the first attempt at making such a tool, but those attempts didn't end up succeeding. However, NodeManager might be changing that, making such a tool even possible. So, all compliments should be directed to you.
I'll need to bother you soon, because I'll need your help to validate the generated code, since it directly uses NodeManager. Still working on it - I'll keep you posted. -
Thanks, @user2684. @hek tells me that this is not the first attempt at making such a tool, but those attempts didn't end up succeeding. However, NodeManager might be changing that, making such a tool even possible. So, all compliments should be directed to you.
I'll need to bother you soon, because I'll need your help to validate the generated code, since it directly uses NodeManager. Still working on it - I'll keep you posted.@rakeshpai sure, I'll be more than happy to double check any code generated for NodeManager! And I'll do a better job documenting future changes to the API since may have an impact on your project as well.
-
Update: Not much has changed functionally, but I've now added a UI notification for when an update is available, so I don't have to tell you to do the double-refresh/hard-refresh ;) and people are always on the latest version.
I've also started writing some tests so that I can start to trust the generated code. I hope these tests are (at least somewhat) human-readable - they might be the easiest way to find out what the generated code is doing.
For eg. this file tests the NodeManager parts of config.h, and this file tests the sensor-specific code for NodeManager. The tests aren't exhaustive yet - still work in progress.
@user2684 Not sure if you'd prefer to read these tests. The alternative would be to change stuff on the UI, download zips, and verify the contents, which can get painful fast.
-
Update: Not much has changed functionally, but I've now added a UI notification for when an update is available, so I don't have to tell you to do the double-refresh/hard-refresh ;) and people are always on the latest version.
I've also started writing some tests so that I can start to trust the generated code. I hope these tests are (at least somewhat) human-readable - they might be the easiest way to find out what the generated code is doing.
For eg. this file tests the NodeManager parts of config.h, and this file tests the sensor-specific code for NodeManager. The tests aren't exhaustive yet - still work in progress.
@user2684 Not sure if you'd prefer to read these tests. The alternative would be to change stuff on the UI, download zips, and verify the contents, which can get painful fast.
@rakeshpai I like the idea of the tests but I'm just afraid it will take more for you to write tests for all the possible combinations more than the time it took already to write the entire sketch generator :-) I'm wondering, is it something easy by chance having in console.log() the resulting code or the change performed upon any action for troubleshooting purposes? If it is too much effort, since it would be for myself only probably, let's explore a better alternative :-)
-
Disclaimer: This is a proof-of-concept. It doesn't work. Things will change, and will break. In fact, I'm sure many things are broken right now. It would be pretty stupid to use this for anything important. I'm just putting this in the public to gauge interest. You have been warned.
Link to the firmware generator.
If you are like me, you want to put approximately half a million MySensors nodes around the house. You've probably already built the first few nodes, so it feels like you are most of the way there. You love the soldering and the building. However, the coding isn't exactly fun. It requires reading through docs, then trying and failing, and rinse and repeat until you get it right.
This tool doesn't solve that problem yet, but it aims to. It aims to provide a simple browser-based UI for configuring your network, and then generating the firmware for the node automatically, using whatever's the best-practice for writing the firmware. This tool writes your code for you!
The good:
- You don't have to write code! This web page does it for you!
- Security built in by default.
- Uses the awesome NodeManager by @user2684 under the hood.
- The web-page, once loaded, works completely offline. You do not need an Internet connection to use this tool! (Try it!)
- No sign up, no registration. In fact, there's no server, or no central database. Your data doesn't even leave your browser.
- There's no server, so there's no network communication and everything's local, so it's blazing fast!
- That bears repeating: The firmware is generated inside your browser, without using any cloud that will rain down on you. Isn't it awesome what browsers can do these days?
The bad:
- Needs a pretty modern browser. Tested on Chrome and Firefox. Untested on IE.
- Doesn't work on mobile devices (yet). Might work in Chrome/Android, but it's untested. Other browsers would need a server-side component to get them to behave.
- I have Google Analytics on the site, just to gawk at some graphs. All GA data is anonymous.
The ugly:
- Well, it looks ugly.
- The code it generates hasn't been tested. This is just a proof of concept. DO NOT depend on this.
- The code, especially the security bits, require review. In fact, I'm pretty sure I've done it wrong.
- They say, 'if you aren't ashamed of it, you've released too late'. I'm definitely releasing too early.
With that summary out of the way, I'd love it if you could poke around and let me know what you think. You don't need to download or install anything - it's just a web page - so just click the link above. I'm trying to gauge interest in the community for this, to decide if it's worth pursuing.
Would you like me to spend time on making this better?
The icon to download firmware is on the top-right of the page, and you can see it on the pages for either the nodes you create, or on the page for the gateway. You can create nodes by selecting your network in the left hand side navigation, and clicking on the
+icon at the top right of the page.The firmware is downloaded as a zip, which contains a SecurityPersonalizer with your security settings, and the main sketch which runs on your node. Currently, the firmware might not even compile.
Hope this gives an idea of what I have in mind, even if it doesn't actually work.
All manner of suggestions and feedback is welcome.
@rakeshpai The idea of creating a generator is really cool. There are many non-programmers out there, just wanting their sensor-network up and running.
I bet you finish this project, and make many people happy. Keep up the good work rakeshpai.
The awesome MySensors generator -
@rakeshpai The idea of creating a generator is really cool. There are many non-programmers out there, just wanting their sensor-network up and running.
I bet you finish this project, and make many people happy. Keep up the good work rakeshpai.
The awesome MySensors generator@core_c Thanks! That reminds me...
I need a name for this project, and I'm horrible at naming things. If some creative folks can chip in, that'd be awesome.
I've been referring to this as 'this', or app, or tool, or site, or things like that. None of them are good enough. A nice label would be great. Something catchy, maybe?
