Started with MySensors and about to give up (some feedback)

[mickecarlsson]

I started with MySensors, Pro Mini and NRF24. I had tons of problems. I borrowed from work our WiFi scanner equipment and checked the 2.4 GHz. It was a total mess. Our DECT-phone almost killed the channel that was used for MySensors. And I discovered, as many has, the NRF24 was all counterfeit.
So, i trashed all NRF24 and moved to RFM69. That was nearly three years ago. My gateway is a Raspberry Pi.
My controller is Domoticz.
Conclusion, and a note to all MySensors users out there. Move away from NRF24 and go with RFM69, you will never look back.

[sundberg84]

Or you can do both (RFM/NRF), which is another positive thing about Mysensors. I have also noticed how dependent NRF is on other traffic. I sometimes lose some nodes during high peak wifi hours, but only the nodes that are on "the edge" of their coverage range. So - its quite easy and cheap to build another RFM gateway, dont have to throw the NRF gateway since I have alot of indoor nodes working with that one. The garden nodes and some basement nodes are running RFM protocol - and that is a big upside with MySensors in my world.

How i see this thread is that MySensors team or any other interested and dedicated person needs to make this clearer and maybe in some newbie guide or on the main homepage start this discussion/ state this information. In this way we can avoid all these things... For example changing the chapter "Radio" in https://www.mysensors.org/about/components so it not only shows the NRF module but also other options, and also have this "common issues" stated.

[BearWithBeard]

I agree with @sundberg84. Let's just make clear that none of this implies that MySensors is fundamentally flawed. My understanding is that the criticism / objection is solely directed at the documentation and how it is presented for new users. If I may try to summarize the consensus in this topic so far:

• Having a proper documentation is very important.
• MySensors used to be simple and straight forward to setup, because options were few.
• The ever-increasing feature set makes it harder for new users to find out which components might be the best choice for their use case ...
• ... and some of the options or their documentations are currently outdated, deprecated or insufficiently tested. They should at least be labeled as such.
• If you are new to a project and follow an introductory guide, the expectation is that you can succeed without seeking information in additional sources ...
• ... or else new users who run into problems may quit in silence and one may never find out, if there is an issue with the documentation.
• There is a desire for a simple step-by-step guide with a "core" set of hardware (and software), both for beginners and people who are simply looking for a stable, reliable setup without hours of fiddling, debugging, searching and swapping components.
• NRF24 might be widely adopted in other Arduino-related projects and the "go-to" radio in MySensors for historical reasons, but many are seemingly frustrated by its comparatively poor range and stability, so RFM69 may be pushed more ...
• ... and there is a lack of explanation, which transport method new users should pick up. A list of pros and cons may be helpful.

Honestly, I think one of the greatest features of MySensors is the fact, that you have tons of options to build your network with. Whether you decide to use Arduinos or other beginner friendy dev boards and premade modules, bare AVRs, ESPs, ARM or NRF5x chips and design your own boards, 2.4 GHz, sub-GHz or wired, lots of controllers to choose from... but I see how this variety can cause problems if they're not well documented, because every component has a different set of quirks and requirements.

Propably the best way so far to get issues related to the guides fixed right away, would be to comment on a guide directly, for example Building a Raspberry Pi Gateway, and mention the problem in there. Suggestions to improve the overall guide could also be discussed in there and a staff member could update it in a timely manner.

But I agree that the process for new users could be more streamlined. Maybe we should collect some ideas how this situation could be improved?

Short term:

• Compile a list of guides which contain outdated or otherwise flawed information. Mark the respective sections as such with a note in a colored box for the time being, if they can't be fixed / reviewed right away.
• Colored boxes may be used in general for asides / cross-references, to emphasize important bits or refer to related information.
• Explain the differences and (dis-)advantages of every available transport method in a separate article: "How to choose the right transport method"
• Add a "Build your first MySensors network step by step" guide in the Getting Started section, which focuses on a single set of hard- and software. The user may learn how to configure and connect to a controller, sending and receiving messages, using two Arduinos with USB interface (Uno, Nano) and NRF24 radios - one Arduino as a serial gateway, attached to a computer running Home Assistant and the other one as a node with a simple button and a relay attached to it. (The selected hardware should just serve as an example here.)

Mid / long term:

• Review all guides. Best practices need to be added and / or mentioned more prominently. Test if the components are still supported; fix or remove them.
• Add feedback buttons to every page like "I made this" and "Didn't work for me". Clicking the latter could open a modal and ask for the users input (without registration). It could also serve as a public popularity indicator (besides analytics, I guess).
• Users currently have to choose between each a bunch of 1) controllers, 2) gateway hardware options, 3) ways to connect to the controller, 4) ways to connect nodes and 5) microcontrollers for the nodes and we may want to find a way to suggest different paths which are known to work well together, for users with different levels of expertise or different needs. Possibly by adding additional step-by-step walkthroughs as mentioned above. One that focuses on simplicity, one on flexiblity, or range, low cost, fully wired,...
• Or implement some sort of configurator where the user can select components or features they want to use and the system suggests which hardware / software would be a good fit by internally weighing them.
• Maybe turn the currently solely staff-curated pages into some kind of Wiki format or GitHub repo, where everyone can participate and submit changes (moderated, of course).

Thoughts? Other ideas?

Of course, I don't expect the devs and mods to do all the work. Everyone who has the time and expertise to contribute is invited to help out if any of this is welcome.

---

@Jens-Jensen The other wireless transports aren't configured for max TX power either. I guess they are set a little bit conservative to work with a wider variety of hardware and power sources out of the box. I don't have actual numbers for common NRF24 modules at hand right now, but there are modules like the CDEbyte E01-2G4M27D which can draw up to half an amp with RF24_PA_MAX. RFM69 radios draw about 30 mA during TX with the default +10 dBm setting, about 45 mA with +13 dBm (max. for modules without PA) and > 130 mA at +20 dBm (PA). I think this approach is fine, but I agree that this setting could be documented better (Is it even? Can't find it outside of example sketches and apidocs right now).

About NRF24 range in general: I have a gateway and a repeater with PA-LNA modules with RF24_PA_MAX in our house (reinforced pumice stone walls and concrete ceilings) and cover all four floors and even some places in the garden with them. None of my sensor nodes need to go above RF24_PA_HIGH, many work fine with RF24_PA_LOW, increasing the battery life, which is especially nice for nodes running off CR2032 coin cells. But I understand that RFM69 might be the better choice for a sensor network and it should be considered to push it more or even make it the default / recommended radio for MySensors in the future, even though they might be harder to use for beginners.

[rejoe2]

Starting with MySensors in the 2.0.0-beta days with nRF24 (and no technical or IT educational background), I completely agree with a lot of the things already mentionned here.
It was a lot easier then and most of the transceivers (even the most likely couterfeied ones) worked with some caps added. But I saw a lot of people struggling when trying out themselfes, be it counterfied chips, struggling with what the controller sw (FHEM in my case) does or (often) - using the Arduino framework to get combined nodes.
So I started doing some wiki work with focus on FHEM (MySensors Starter Guide), collecting also the experience from some other users there.

As atm some new users became interested especially in wired solutions, I was very happy to see the PJON integration making good progress. Imo this is a very good and easy approach, as no additional hardware is required. This makes this transport layer imo the best choice to get into the basics on MySensors and it's interaction with the controller sw as well. Choice for controller in most cases might already been done when joining the community here, so imo this could really help to "join this train here" more easy, as testing doesn't need much wiring or additional hardware.

The only disadvantage is the lack of RF. So starting with PJON as "learning level transport layer" and adding then RFMxx als wireless option might be considered as primary recommendation for the first steps towards wireless then?

[scalz]

Hi there,

so far the main "culprits" I identified are:

• add a picture of a genuine nrf24 module (eg cdebyte) near the cheapo nrf24, and why not a rfm69/95 module too, in the Getting Started/Component/Radio
• add a comparison of pros/cons of nrf24 vs rfm69 in Build/Connect radio, before the wirings
• remove en28j60 from doc, and keeps only w5x00. Let the community handle en28j60 if this is really needed. I don't think anyone in the team is using en28j60 for gw. The fact it's not uncommon in rpi community, doesn't mean it's common in mysensors community (more based on arduino than rpi)
• update store links and try to remove cheapo nrf24 clones
• rpi section

else I don't see so many outdated infos.

Regarding howto connect a controller to mysensors, I've no idea if this should be on mainsite. As controller choice is very subjective. For example, I've already tried Openhab and others, even HomeAssistant for months too (imho having to configure yaml etc is not noob friendly, I couldn't recommend that to noobs and non-computer people, same for nodered as automations tool ) and I still prefer Jeedom..

Other notes:

• it's hard for the team to support and guarantee what we do not use.. regarding all the options available in MySensors (gw, hw platforms..). Example: I'm not sure, but I think rpi as gw is not widely used in the team for example.
• a reproducible howto get started, with 100% success, would be nice sure.
  Still it's important to remember that will always need to learn new skills in mid-longterm. Not because you're a dev, that you know electronics, rf recommendations, the framework etc
  It's hard to follow and compile all forums infos in one place, with limited resources. This would need a book!
• I'm also in favor of subghz but I can understand it's maybe less plug&play for noobs. I also think when you want to get started, it's easier to get success with Serial GW.

[Sergio Rius]

@scalz After the "cleaning" there should be an example on each supported endpoint platform. I mean, if rj45+mqtt is supported, rfm should have a working example with all the components (level adjusters, etc...)

The big problem with rf24, said that it's so dependant on the environment, is that different countries have different regulations. And could be that what seems perfectly ok for some or all developers, doesn't work for other people in their houses.

Documentation doesn't say anything about checking country's wireless regulations as it's neglected.

[scalz]

@Sergio-Rius
sure these are good points. especially country regulations :+1:
That said this needs some research, personally I don't know rules for all countries & frequencies.
In that case, we should also mention that anyone making&selling his board/device with a rf module is maybe out of the rules without fcc/ce certifications. So many things to say on regulations, you're right.

The trickier point is docs for all possible usecases. There are so many different combo. I don't know if educating and covering "all" electronics possibilities, arduino coding, handling third parties libs etc, is the goal of a software framework here, I don't know exactly the original goals.

"If I was in charge and alone for all this work" (personal point of view), I would remove from howtos all things that I don't use, and would let this stuff live in the community hands, on the forum for example.
Like I said, supporting/maintaining/documenting stuff that you don't use, for free, in spare time, is pita.
Franlky, there are many platforms+mysensors that we don't even use. It has been added to the framework, mostly because it's fun to learn/discover new platforms etc, not because of an urgent need at home.

The problem is the more you show&provide, the more users may want and expect, whereas we hope it will be community driven.
It's like you provide for free young plants to someone, and he expects you'll always feed&water it :)

Unfortunately, regarding documenting the stuff I don't use, I've no time for the moment..busy on other projects, hobby in my spare time must be fun.

I hope we'll find a way to improve docs so it's 100% success each time someone new get started. Hopefully I think there are lot of people who already got success with MySensors and helping on the forum

[monte]

@alex28 said in Started with MySensors and about to give up (some feedback):

Problem 2. The Raspberry Pi gateway.

Yes you need to add overlay to /boot/config.txt to use both SPIs. I have working gateway which utilizes both SPI ports and works without issue.
So have you achieved working GW on raspberry?

@alex28 said in Started with MySensors and about to give up (some feedback):

Problem 3. NRF24 range is bad. Really bad.

I am using this NRF24 PA LNA module on my gateway and the cheapest NRF24 modules on my nodes. This works pretty good. I suggest you buy some good quality PA LNA module for your gateway as they require proper shielding and the cheap ones don't have it. CDEBYTE modules are considered to have original Nordic chips and has many good reviews on this forum. And yes, you need to ensure you have good power supplied to radios. Also 2.4GHz can be easily blocked by concrete or metal obstacles if it is placed closed to them, for example I have outdoor light node with concrete base, and I had to stick radio module above concrete part to achieve stable connection, same goes for metal enclosures and boxes. My GW is places in metal distribution box, so I mounted external antenna on its door to avoid shielding of a signal.

I understand your frustration, but you've chosen not the easiest or common way to begin with. Most people on this forum use standalone serial or ethernet gateway and don't bother connecting radio directly to Raspberry, that's why you had such a bad time finding correct answers to your problems. Good thing is that you seem to be close to a working result and have gain much experience and can make valuable contribution to this community.

Radio networks are flawed by design in my opinion, I've had hard time with commercial Zwave products, and community support wasn't great either, as that is proprietary protocol and you basically rely solely on manufacturer once you've bought a node for 50$ you almost on your own.

[mhkid]

@monte this is exactly the point. There is no mention or way to know you're choosing the hardest possible combo of hardware until you've spent hours fiddling with it. You'll find more posts about counterfeit radios rather than what a good radio is. There is saying that they don't teach treasury agents what every counterfeit bill looks like because there are too many and it's always changing. They teach them what a real bill looks like so they will know a fake when they see it. Same thing here. Just post the place to buy a good radio. Don't post a guide on rpi gw if it's the hardest and least used option, leave that to the forum.

I would rather spend my time building and using the sensors rather than troubleshooting and figuring out problems, which it seems like that's what I do a lot of. It could very well be the choices of hardware I'm making I'll completely admit that but I don't know because there are so many paths you can take. OTA updates are something I'd like to incorporate but there is no clear path to success there for me. I feel like that will lead to frustration based on what I've read in the forum. Again because there are too many options without the clear path laid out.

I keep coming back because I really see the potential and there really is no other platform like MySensors. If I had that strong reliable baseline to build from I'd have some pretty cool ideas and sensors to contribute.
Instead I'm building sensors with Wi-Fi connections all over the place when a single gw connecting to the internet or a controller would be a much better solution.

[mhkid]

@pptacek said in Started with MySensors and about to give up (some feedback):

jModule2

Do you have details on these sticks? What is jmodule2?

[monte]

@mhkid Well, I think tutorials must be updated and graded in terms of difficulty, that will ensure that newcomers will first build serial node based on arduino nano and only after having working network will try harder solutions.
That said, I believe that troubleshooting is the best way to learn new topic. Especially if you have helping community to come for an answer when you feel like you've come to a dead end.

[alex28]

@monte mhkid summed it up nicely. My entire point here was that there was no way to know that my choice of hardware was uncommon. The guide made it sound like this would be a perfectly valid and well supported choice. Yes, I have a functioning Pi based GW now, but the radio link is far too unreliable to make it useful. After considering all pros and cons, I decided to not proceed with MySensors at this point. The lockdown is over and I now have much less time to put into this. Instead I will try the Z-Uno, which seems more in line with my specific use case.

Still, I'm glad my feedback has spawned some discussion around this. Maybe it will help to improve MySensors in that respect. Maybe I'll come back to MySensors at some point too, because it is an interesting platform after all.

@monte said in Started with MySensors and about to give up (some feedback):

That said, I believe that troubleshooting is the best way to learn new topic. Especially if you have helping community to come for an answer when you feel like you've come to a dead end.

Well that depends. Stupid analogy : If you're a photographer and get a new camera, a nice customizable one that gives you lots of manual options to play around with, you would still expect it to work out of the box for simply taking some basic photos. If you have to take it apart entirely and start swapping out its internal components using trial and error, just to be able to use it for the one thing you ultimately got it for - taking photos - then you'd probably not be very happy.

[kiesel]

I hope it is OK to post this here, I am not sure how to add this to the documentation, but my biggest pain point when starting out was that you have to use mysensors >2.2 when running on a raspberry pi 4, but the rfm gateway code has a bug in versions bigger than 2.2. I spent more than a week debugging that until I found the github issue and settled for a serial gateway, which I am very happy with, thank you mysensors!

If a warning could be put on the gateway page that would be great!

[scalz]

Hello,
@mhkid @alex28
That's too bad that you never asked for help (except mhkid, one time) while you were so many times in a dead end.. some waste of time, I completely agree with you. Don't be shy :)
Actually I'm a dev too, high or low or blackbelt level doesn't matter, but what I know very well is that I like to fix problems by myself, I rarely ask for help too. But the truth is when someone can save me some time, my precious spare time, I'm always very glad to find him.

@alex28 that's good to hear that perhaps you found another framework which can fit your need. I really hope and wish you success, especially if it costs 50€ per devboard.

This is quite interesting actually, now I wonder about something..

To people:
if we would sell boards for 50€, eg on mainsite "Store" page, with MySensors howtos and high success rate, would you really buy it?
or would you still try to use what you have left in your drawers?

If so, it's maybe possible to make your dream become true.
Of course, high rate of success would only apply to these kits

Note: to any newcomer reading this topic, regarding the "don't know where to buy genuine nrf24", you can find links on mainsite, "Store page", for some recommended modules. And don't forget those decoupling capacitors ;)

[alex28]

@scalz said in Started with MySensors and about to give up (some feedback):

That's too bad that you never asked for help (except mhkid, one time) while you were so many times in a dead end.. some waste of time, I completely agree with you. Don't be shy :)

That's a personality thing. Documentation is very important to me. When I encounter a technical problem, on personal projects or on work projects alike, I will do extensive research in order to assess, understand the cause and eventually solve the problem. I will very rarely post a technical question on a forum. That's just the way I work. Then again, I did mention that this was purely a personal experience report :slightly_smiling_face: Different people have different approaches and will have different experiences.

@scalz said in Started with MySensors and about to give up (some feedback):

@alex28 that's good to hear that perhaps you found another framework which can fit your need. I really hope and wish you success, especially if it costs 50€ per devboard.

Thanks. You pay mostly the proprietary ASIC on it, the ZWave license/brand, but also compliance with pretty strict performance specifications on the HF/radio side. Let's see how it goes.

In fact, the cheaper component price was not the reason that made me consider MySensors in the first place. The open nature of the platform was. I'm perfectly fine with paying more for solid and reliable hardware. I think that is also an important point to keep in mind. As far as I see it, this open and community driven aspect is the greatest strength of MySensors. While the cheap component aspect is certainly important for many people, it can also lead to a lot of problems due to counterfeit low quality HW.

[mhkid]

@scalz not sure how to take the comment. I've posted and contributed (even developing a library) so it's not like I haven't participated. I bought parts from the links and still had issues with radios. Whether help was asked for or not that doesn't make the constructive feedback invalid. Feedback is a gift and in this case it should be taken that way. These are suggestions to make MySenors accessible to more people.

Yeah there is a good chance I would buy it. When I started my journey into home iot devices I was buying $45 light switches so if it was a good product I would.
I've looked at the MySensors gw but several of the comments mentioned issues so l steered clear. It's not the cost. What's being suggested here is not a huge criticism it's just asking to have a clear path to a high probability of success. Which is mainly cleaning up the documentation. If I had a solid working setup I'd help do it but I don't. I love the platform and the idea is really great, it's just too scattered and needs to be more focused. Don't try to be all things to all people.

[scalz]

@alex28 @mhkid
I just meant what I thought.
I like to find solution by myself too. Like you it's super rare that I ask for help. But, I'm also an impatient guy, very impatient, I hate being stuck during a rush etc. I always feel getting stuck more than one day is like wasting time and money, don't you think. Still it happened more than once to me, of course. That's funny I'm saying that, as I actually I'm working in research field :sweat_smile:

This is why I said you shouldn't have hesitated to ask community for help, lot of kind people here.

@alex28 said in Started with MySensors and about to give up (some feedback):

In fact, the cheaper component price was not the reason that made me consider MySensors in the first place. The open nature of the platform was. I'm perfectly fine with paying more for solid and reliable hardware. I think that is also an important point to keep in mind. As far as I see it, this open and community driven aspect is the greatest strength of MySensors. While the cheap component aspect is certainly important for many people, it can also lead to a lot of problems due to counterfeit low quality HW.

I completely agree, that's why I trusted in MySensors years ago.
And supporting low quality, and too many platform, is pita, especially when you just have very little spare time, all of that for free of course. That's why we trust in community to help us.

About my own xp, I got success the first time I got started with MySensors+rf24Serial gw.
But it was just a quick test, as my first goal was to use rfm69. No one told me rfm69 was superior to rf24, to me it just made sense.

I'm in favor of serial gw actually because I think it's :

• more "simple"
• secure. Just a serial connection. no tcp connection etc
• reliable. doesn't rely on an OS to run the network, and stronger against noise in radio which could be introduced by rpi for example, when power supply is not polished, filtered, decoupled etc. No one said that in the docs, I know, it's certainly missing infos. It's just, imo, from personal xp.

That said, we have heard your wish about improving docs, and providing hw with trusted howtos (we have some good hw designs, for gw or nodes, with variable bom cost).
Thank you for your feedbacks

[Psilin]

What an interesting thread this became. Over some time/years ago I tried to start with MySensors. Unfortunately I failed miserably because I tried it to combine with starting with Openhab, and connect mysensors by mqtt.

The Openhab route combined with Mysensors is still the route I want to go: combine the open architecture of Openhab to connect different vendor approaches added with the flexible way of MySensors.

To be complete: My background is a bachelor in Physics. So the basic programming and electronics may not be the real problem, although I am certainly not claiming to know everything.

To end up this post: in the next couple of weeks/months I have spare time to invest in the project. When I combine this with the things the topic starter ave addressed I have the following to offer to the Mysensors community:

“I am offering to write and give feedback of the guides and information found in the forum, to document at what point I am stuck with the given information. I hope the community can and will do something with this feedback. So long as I am not going to be completely stuck, I can and will be writing the feedback.”

Off coarse I need to know if the community is waiting for such an approach before I start to write.

My thoughts of devices to use in the network:

• mqtt gateway: wired based on cloned arduine with hat ethernet
• mqtt gateway based on wifi based esp8266 nodemcu board
• radio network: base the nrf24 version
• radio network: based on rfm69 for secure transmission (opening doors with nrf24 does not feel safe)
• repeaters as needed or extra gateways (3 story concrete house)
• sensors and actuators: starting with wallpowered, later adding battery powered

So, how does this approach sound to you all?

[wergeld]

@pptacek Had not seen those before. Order some and a few nanos to go with it. I had been using the slim node v2 AA battery boards using just the 328p bare chip. Issue I have with that board is the layout of the pins for my needs. It is a great board and I use the nodes I have a lot - no disrespect meant to the developer of it.

I agree that the entry ramp to get mysensors working is a bit complex. I started with the RPi as the gateway because well, it should work. Had a hell of a time making it function. Ended up getting the RPi hat that has the capacitor and NRF slot on it that plugs into the GPIO headers. But, issues persisted with the library for the RPi not working on the RPi 2 until some very helpful folks pointed me in the right direction. A lot of this is pure tinkering and not yet a prime time off the shelf system. I knew that going in and accepted. It has been a great learning experience. The frustration I initially had was all because I expected the guides/docs to be up to date. They are not. As bad as that is, once you accept it and decided to a) add to the docs yourself or b) carry on screaming it becomes rewarding. As an example, getting MySensors to work in Hom Assistant is easy using an MQTT gateway....until you realize that HA tells the nodes what measurment units to use and the nodes then flip (from Celsius to Fahrenheit) when you think it should do the conversion on the front end of HA. I still, for the life of me cannot get the nodes to flip back to Celsius from an MQTT command (had to reflash the nodes with the code).

At the end of all this I am still having fun and learning new things. As the OP said this is not my first programming rodeo (currently a senior SQL developer working with epic sized data with a massive helping of javascript, R, python, and .NET). The micro controller code paradigm was (is still?) new to me even after almost 6 years of messing around with various projects including MySensors.

If we can help any new people stay on this journey it is time well spent. I offer my services for any help that may be needed for docs or guides.

[monte]

@Psilin I suggest you look into Home Assistant instead of OpenHAB. It is much more flexible and easy to set up and I feel like it has better perspective in general.