Best posts made by alex28

Hi everybody. I'd like to give some feedback on my experience as a new MySensors user here. As you can guess from the title, it's not going to be very positive. I hope this kind of post is not unwelcome. It's also very long. Please don't take this badly. It's just my own experience with MySensors I'd like to share.

Short info on me. I'm a software developer mainly working on GPU low level code. Although mostly a software guy, I have some experience with the hardware side of embedded systems and I know my way around a soldering iron and a scope. I was looking for a way to improve my home automation setup, mostly made of ZWave and cheap 433Mhz devices, with something open that would give me more control. Basically build my own nodes. MySensors seemed to fit this perfectly.

So came the lockdown and hey, perfect opportunity to get started ! Got myself a couple of Arduino Nanos, a few NRF24L01+ and lots of motivation to get my first GW and test node running. Unfortunately it ended up being an extremely frustrating and time consuming experience. More than once I had to resist the urge to throw everything in the nearest trash bin.

Problem 1. The gateway.
I had an ENC28J60 lying around for some time. The MySensors getting started guide clearly mentions that this is a supported option. So I connected it, got the UIPEthernet libs linked in the guide. Cloned the ntruchsess repo as mentioned in the readme. Hit compile and - incorrect library version. Great start. Cloned the MySensorsArduinoExamples entirely, copied the lib from there. Incorrect library version. OK. Started to hunt around Google, found the original UIPEthernet repo. Cloned that and it compiles. Good.

Started the node, doesn't reply to pings. Searched the MySensors forums, found a topic titled 'Can't ping Ethernet Gateway ENC28J60 + Arduino Nano Clone' from 2017. In that topic some MySensor devs clearly state the ENC28J60 support is broken and that, I quote, 'Not sure anyone has bothered checking with the latest version of the library...'. Uh what ? If you know that this chip won't work since 2017, why is it still a recommended option on the building a GW guide ? Why not make it clear that only the w5x00 chips are supported ?

Problem 2. The Raspberry Pi gateway.
So ordering a w5100 for my GW is not an option right now. It would take a month to arrive due to lockdown. So I decide to use a Pi Zero as a GW. At least the ENC28J60 works with that one. I visit the 'Building a Raspberry Pi gateway' page. It talks about some radio communication bug in the 'latest Jessie'. Raspbian Jessie has been shelved in 2017. So uh, is that bug still an issue in current Raspbian Buster ? Do I still need to get the dev branch or not ? OK, I end up pulling the master branch. The Pi has two independent hardware SPI ports. I connect the ENC to spi0. Works. I connect the NRF24 to spi1. Oh wait, the pinout image in the wiring diagram section of the 'Building a Raspberry Pi gateway' is wrong. SPI1 CS0 is definitely not on pin 36 by default (but on pin 11). Why leave a clearly incorrect pinout diagram on your guide page ?

So I try to compile the GW for using spi1. Doesn't work, radio connection failed. Searching the forums. Found a post titled 'Double SPI Radio Raspberry Pi'. Great, sounds like that guy has figured it out and a MySensor mod commented on it. Sounds like that should work. Followed his quick guide. Doesn't work. Search more. Oh wait, it actually can't possibly work - his cs0_pin assignment in the config is wrong (this setting expects BCM pin numbering, not Pi pin numbering as he used). So that forum topic - which probably pops up for everyone searching how to connect the NRF24 to a Pi spi1 port - won't work because it contains a fundamental error.

Problem 3. NRF24 range is bad. Really bad.
So I finally get the GW and a test node to communicate. Yay ! Smooth sailing from now on. But wait. If I position the node across the room, it starts dropping packets. Searched the forums. Filtering caps on both NRF modules - check. Clean power - check (battery pack for the node, lab power supply for the GW Pi). Range is still bad. Some forum posts seem to indicate that the Nano 3.3V regulator is not good enough to power the radio and you need a separate supply. OK. But why not mention this in your build guides ? All your guides clearly show that the NRF gets directly connected to the 3.3v rail of the Nano. Well I don't have a voltage regulator. Searching more. Lots of conflicting information. Maybe the NRF is counterfeit. Yep it probably is. Order and try another one ? Order and try an external regulator ? Lockdown, week long waits for any oversea deliveries, so nope, not an option. Found lots of topics with people having range issues on that radio. Common solution seems to be switching to the RFM69 on a different frequency band. Would have been nice to know that before. Why not mention this in the guides if it's such common knowledge apparently ?

Well that's where I stand right now. What was supposed to be a fun and satisfying experience to build some home made IoT devices ended up being a long and frustrating one leaving me with a feeling of having wasted my time. I wanted to build some fun nodes but after all that time spent trying and searching and ripping my hair out, I still don't even have a stable network running.

And yes, some of this is probably due to bad counterfeit HW and also my own overexpectations. But a lot of the frustration came from the obsolete, outdated, conflicting and sometimes blatantly false information presented both on the official MySensors guides, as well as in the forums. I wasted so much time searching and trial'n'erroring solutions to problems that wouldn't even exist if they were clearly outlined in some up-to-date guides.

At this point I'm unsure about how to proceed. I like the idea behind MySensors and support in Home Assistant (which I use for HA) seems good. I could order an NRF24 PA LNA. Or some RFM69's. And try both. And have a more reliable network - maybe.

But I don't really feel like spending weeks on this just tinkering around and trying to make the basic network behave in a reliable way. Maybe I misinterpreted what the MySensors platform is about. My impression of MySensors was that this is a solid and well tested platform where the basics would just work out of the box without much tinkering around. And where I could spend my time on what actually matters to me - building my custom nodes. Trying out dozens of different radios from different manufacturers, different power supplies, different DIY dipole antennas and whatnot is just not something I'm interested in. Maybe I'll just get a commercial ZWave interface with digital inputs and connect it to the Arduino instead.

Thanks for reading.

Hi, thanks a lot for all your replies. I really appreciate that my post wasn't just taken as a rant from a disgruntled new user, but as constructive criticism from a different perspective.

I'd like to address some points made. Please keep in mind that all this is from my own personal experience. I can't claim that it's universal, but just some subjective observations from one guy who tried MySensors. That said -

@BearWithBeard I have to respectfully disagree about the guides being generally fine. Yes, they are nicely written and well presented. They're very inviting for new users. That's important and sets MySensors apart from many other open projects which often look very unwelcoming to new users with very steep learning curves. Having been involved in a few FOSS projects, god knows I have been guilty of that myself. The problem with the guides is the information contained in them. It's outdated and misleading.

From what I could gather on the forums and also from replies here (as the one from bjacobse), is that there are a number of best practices to follow with a MySensors project. And you have to follow them closely. If you don't, you will inevitably run into problems. Don't use the ENC28J60 network chip, it won't work. Don't use the NRF24L01+ if you want a reliable connection. Use external voltage regulators. OK that's all fair. But while these practices seem to be very well known by the community, even taken for granted, they are not documented for new users. In fact, the very guides written to draw new users to the platform will actively encourage bad practices - by presenting them as perfectly valid options.

Yes the information is out there somewhere. But it's often hard to find, spread out and riddled with confusing and conflicting (and sometimes plain wrong) info, sidetracking you.

I'm also not that sure if my choice of hardware is so unusual. Maybe people selecting such HW will simply never show up on your radar, because they give up early on due to all the problems they run into.

For the network chip, yeah, it's not widely used on Arduino. Yet the ENC28J60 is not uncommon for giving RPi Zeros an ethernet port. Like most of these cheap Chinese modules they're usually sold in packs, so probably more than one newcomer to MySensors will have such a module lying around in a drawer somewhere. Especially if he comes from the RPi community. Your starting guides make it sound like this module is a perfectly fine choice. You just have to install these libs that MySensors specifically modified for it (but they were never updated and don't work with the new AVR stuff anymore), and you're good to go ! Well no. And then later on, after hours of frustration, you find a forum thread where mods openly say that the ENC is unstable, untested and should be avoided, then reading the guide sounds almost cynical to me. If something is known not to work for years, then remove it as an option from the newbie guides. Having only one single network chip class supported (the w5x00 line) is perfectly fine. Just don't mislead people into using unsupported HW.

The NRF24 situation is similar. I understand that historically this was the only supported radio. I also understand that it is again unwritten community knowledge that mostly due to the market being submerged by barely functional Chinese clones, there's a good chance the performance with this module will be abysmal. And also that the RFM69(HW) is a much better and more stable choice for many use cases. Yet, this is not mentioned anywhere in the newbie guides. The NRF24 is still presented as the primary and default choice of a radio. With an image of a cheapo Chinese clone module right next to it. I think your idea of adding the pros and cons of each radio would be tremendously helpful when starting out. Had I known the issues with the NRF clones, I would have either ordered a better nRF based module (like the one from Sparkfun) or something RFM69 based, before even starting. And this would have saved me from a big disappointment later on.

Also, please don't underestimate the number of users coming to MySensors from the RPi communities. A lot of DYI home automation people use controllers like Domoticz, Home Assistant or similar running on a Pi. I'm sure that many of your own established community members also do. It only makes sense for many to install the MySensors gateway directly on their Pi. And that's where people will inevitably run into issues like I mentioned in my OP. If you browse the forums from these communities, when people ask about creating their own IoT devices, they will more often than not be pointed to the ESP8266 / NodeCMU stuff running on Wifi. With the result that all their DIY IoT devices are directly connected to a public IP network, with all the fun and not so fun consequences this can have. MySensors would be a perfect and safe alternative here. But I can't stress this enough from my personal experience, having the newbie guides be up to date, working and promoting best practices is absolutely primordial here. They're the first contact people will have with MySensors. And not everybody will have the persistence to keep going through the rough to try to make it work. Many will probably just give up and turn to things like the ESP.

@bjacobse, I appreciate your comments and advise. I have a running ethernet GW at this point, albeit with very unreliable radio link. A serial GW was not an option due to the way my current HA infrastructure is laid out, which is an entirely Ethernet based internal IoT subnet. Your comments about best practices with the ENC and the NRF are spot on and should be made more clear to newcomers.

Again, please don't see this as a direct attack on the MySensors project. I think it's a great idea and a lot of awesome work has been put into it. But communication is really important for new users. And please also think about the type of user who is 'only' looking for a working and stable base platform he can build sensors on. For these type of users a known stable setup, both HW and SW wise, should be recommended in my opinion. Along with best practices and pros and cons for HW with valid options (like the radios).

@monte mhkid summed it up nicely. My entire point here was that there was no way to know that my choice of hardware was uncommon. The guide made it sound like this would be a perfectly valid and well supported choice. Yes, I have a functioning Pi based GW now, but the radio link is far too unreliable to make it useful. After considering all pros and cons, I decided to not proceed with MySensors at this point. The lockdown is over and I now have much less time to put into this. Instead I will try the Z-Uno, which seems more in line with my specific use case.

Still, I'm glad my feedback has spawned some discussion around this. Maybe it will help to improve MySensors in that respect. Maybe I'll come back to MySensors at some point too, because it is an interesting platform after all.

@monte said in Started with MySensors and about to give up (some feedback):

That said, I believe that troubleshooting is the best way to learn new topic. Especially if you have helping community to come for an answer when you feel like you've come to a dead end.

Well that depends. Stupid analogy : If you're a photographer and get a new camera, a nice customizable one that gives you lots of manual options to play around with, you would still expect it to work out of the box for simply taking some basic photos. If you have to take it apart entirely and start swapping out its internal components using trial and error, just to be able to use it for the one thing you ultimately got it for - taking photos - then you'd probably not be very happy.

@tbowmo @scalz
Hmm, I'm not so sure about the certification requirements. I'm not a lawyer (and it might be best to consult with one in case of doubts if you want to market your own RF product), but as far as I understand the legalese, if you use an RF module with "Full Modular Certification", you are not required to apply for a RF compliance certification anymore. You are highly encouraged to do self-hosted RF testing. And other compliance testing / certification might still apply (mains voltage use, spurious digital emissions etc). This is for the FCC. EU regulations are probably similar / less restrictive.

Here's an interesting document from SiLabs. Look at page 15, chapter 3.5.1.1:

"3.5.1.1 How Do Customers use FMA?
• No separate certification with the end product; just label the product and get instant access to the markets.
• The host must be labelled that it includes a certified module.
• An end product using a radio with full modular approval will not need radio certification testing if the restrictions mentioned in themodule grant are met."

They do mention that the module must adhere to certain characteristics to qualify, like a fully shielded RF part. So the RFM I linked to above will probably not work.

Here's a document from the FCC outlining the recommended self-testing procedures, filing and product labeling requirements. Note chapter 3.1:

"Testing of the host product with all the transmitters installed is recommended,to verify that the host product meets all the applicable FCC rules. The radio spectrum is to be investigated with all the transmitters in the final host product functioning to determine that no emissions exceed the highest limit permitted for any one individual transmitter as required by Section 2.947(f). A formal application for certification submission containing the results of this investigation is not required. The host manufacturer is responsible to ensure that when their product operates as intended it does not have any emissions present that are out of compliance that were not present when the transmitters were tested individually."

So if you use pre approved modules, you do not need to re-certify. But you're legally responsible to make sure you're within the regulatory limits. You still need to file with them (and pay them) and label your product appropriatlty.

At least that's what I could gather from quickly trying to parse the legalese

That said, the general certification processes, registration and filing fees, etc will still cost you a few thousand Dollars / Euros. But that's nothing compared to RF compliance testing when not using a pre-approved module, which can easily go into the 20k-30k range.

@Avamander But then aren't we back to the old 'get Arduino board with headers, get a random nRF module with headers, plug them together with jumper wires and - nothing works' scenario ? What I really liked about the original idea was that this would be an all in one, no headaches, everything just works module. Where you don't need to worry about fake components, decoupling caps, bad voltage regulators, etc.

@scalz said in Pre-assembled sensor modules:

@alex28
yes, but when you design a board it's very rare to follow application notes to the letter, because you want a different size of board etc. But I agree about the pre certified module which help a lot for selling hw. I'm just saying there are nuances, it's not a big YES or NO. This depends what components are on the final board, most of the time it may just affect range, or worse introduce noise through power supply, signals lines etc. To check that you can use rf spectrum analyzer, scope..

Correct. And according to the FCC docs, you're supposed to check for all this on your own. With regulations, it kinda is a black and white YES or NO. Either you're compliant or you're not. When using a pre-certified RF module, it's upon yourself to check for this. If you're in a almost YES but not quite situation and you sell it regardless, well, then you take a legal risk.

But that's the price you have to pay if you want to go commercial I guess. None of this really is a problem for simple DIY stuff you use at home (exceptions apply, see the stories you linked lol).