RFM69 Range issues

[bjornhallberg]

So I've been a little bit inactive for the last couple of years. I recently updated my nodes and gateway to 2.3.2 to see if it would fix some of the range issues I've been having and because it seemed convenient to be able to see RSSI and TX power. I also built the signal scanner (https://forum.mysensors.org/topic/7822/portable-rfm69-signal-scanner/6) so I'd be able to walk around and see if communications were working in certain spots.

Gateway (rfm69hw) initialization:

#define MY_IS_RFM69HW
#define MY_RFM69_FREQUENCY RFM69_868MHZ
#define MY_RFM69_NEW_DRIVER

Example node (rfm69cw) initialization:

#define MY_NODE_ID 0xAC
#define MY_PARENT_NODE_IS_STATIC
#define MY_PARENT_NODE_ID 0
#define MY_RADIO_RFM69
#define MY_RFM69_FREQUENCY   RFM69_868MHZ
#define MY_RFM69_NEW_DRIVER

I have a couple of nodes in my garage that are just damn near impossible to get to work. It's odd because I have deployed nrf24 modules there in the past and while transmission has been spotty, 868MHz should be superior in this regard, at least not perform worse. The nodes are all the same, but are using three different antennas. I originally had #define MY_TRANSPORT_UPLINK_CHECK_DISABLED and #define MY_DEBUG with those nodes but commented it out this afternoon on two of them. Still only one is coming through and that is the one with the commercial thick coil antenna that you can find on ebay. I recall reading somewhere on the forum that people had good results with those (https://forum.mysensors.org/topic/7845/rfm69-antennas-comparison).

Another node that I have out in the mailbox and that had been performing flawlessly on 2.2.0 I think it was and the old driver is now completely out of reach. I did the same commenting out as above but it still wont work.

I guess my first question is regarding ATC and how it works on 2.3.2. In particular in this case where nodes are either initialized within range of the gateway and brought outside or initialized out of range and installed. I understood the system as starting at the lowest dbm and working itself up until it reached an acceptable RSSI? How does it proceed if it cant reach the gateway?

Also I was wondering about disabling ATC and setting another modem config? Would it be a better troubleshooting strategy?

#define MY_RFM69_ATC_MODE_DISABLED
#define MY_RFM69_MAX_POWER_LEVEL_DBM (13) // 13dbm/20mw max on rfm69cw?
#define MY_RFM69_MODEM_CONFIGURATION RFM69_FSK_BR9_6_FD19_2

Anything else I can tweak in the settings?

I should add that the "signal scanner" (with a rfm69w) works really well with the code in the link. Transmit % is about ~0% next to the gateway. About 50-60% in the garage. And 70-80% at the mailbox. Perhaps a little bit lower once it has settled in those spots. Nothing special about the hardware either. Built it exactly like described.

[evb]

Recently, I had a conversation about the ATM with @mfalkvidd.
As result he added following section to https://www.mysensors.org/build/select_gateway

RFM69/RFM95 Special considerations
For gateways using the radio transport RFM69 or RFM95, when MY_GATEWAY_FEATURE is defined, the ATC mode of the gateway is disabled which makes perfectly sense since every node will report a different RSSI and there is no need to change the tx level at every transmission.
However, the default tx power will then be used by the gateway (MY_RFM69_TX_POWER_DBM which is 5 and MY_RFM95_TX_POWER_DBM which is 13) and whereas ATC will increase that value if needed, this is not happening on the gateway which will stay at 5 for RFM69 (or 13 for RFM95) forever despite not running on battery and so without any need to save power.
So if you are having range problems with your gateway, set the define MY_RFMxx_TX_POWER_DBM to a higher level. Be aware that setting this power setting to a higher level, will cause your node or gateway to 'screaming louder'. For nodes very close to the gateway, things could go even worse. So test with little steps.
Finally, pay attention to the hardware version of your radio. For example for RFM69, even though 20dBm may be specified, RFM69 is limited to +13dBm, only RFM69HW can reach real +20dBm. The opposite is also true, only RFM69 can use power values lower than -2dBm, RFM69HW is capped lower than that.

I don't find the topic in question for the moment
Like I did understand it, the ATM starts at the level defined by MY_RFM69_TX_POWER_DBM which is by default 5.
It will then adjust the power to the configured ATC value.

Try first to set the MY_RFM69_TX_POWER_DBM on the gateway to a higher value. Use little steps.

But what I don't understand, your node build as signal scanner is working well in the garage and the mailbox with the current setup?
And the other nodes not?
If their hardware is the same, that is very strange...

[mfalkvidd]

@evb the reason you can't find the topic is that there was no topic. The conversation was in an github issue https://github.com/mysensors/MySensors/issues/1157

[bjornhallberg]

No neither the temp sensors nor the mailbox sensor are working on 2.3.0 with the new rfm69 driver at any serious range at least. The hardware is this (https://github.com/EasySensors/ButtonSizeNode) and this (https://github.com/EasySensors/SwitchNode). Have tried different types of antenna.

The hardware difference would be that the scanner is running on AA batteries with no regulator and the sensors above use 3V button cells and some sort of regulator I think. Plus they use RFM69CW while the scanner uses RFM69W.

Like I say above, I've really only disabled debug to save some memory because I read in some other thread that it in fact helped. I have not tried setting TX power manually on either the gateway nor the sensors.

Still 5bdm on the gateway doesn't sound too bad though. What was the gateway TX dbm on 2.2.0 and old regular driver?

[evb]

The RFM69W and RFM69CW have both 13dBm power output, so that can't be the reason.

• You could try to power the buttonsizenode, not by the button battery, but by an external power/battery supply. This to test if it is not a battery button problem not providing enough juice for the radio.
• If same result, try to add an extra 47µF elco or bigger on the power inputs of the radio.
• Do a test with the dBm output set to 13dBm.

[bjornhallberg]

@evb Thanks I'll try that.

I was thinking about setting up a second gateway also (I have another sensebender gateway I'm not using) and running a different frequency for the network. The new driver allows you to set basically any frequency that the module supports right down to the Hz?

And the 868MHz modules are able to go as high as 915MHz (even if it isn't optimal) and vice versa? I do recall something from Lowpowerlabs from years ago that said the 915MHz ones could at least be run at 868MHz? Felix and other users on their forum even confirm it. So that would rule out the possibility that I have radio modules of the wrong frequency,

I am starting to question some of these radio modules I bought off of AliExpress 3-4 years ago. I know for a fact that I have RFM69W modules that are 915, don't remember why I even have those. The signal scanner used a good and known *W 868 module. All the *CW modules are marked as 868. But I am nevertheless starting to wonder if there could be some foul play here. Either knockoffs, or real duds. It would explain a lot of the problems I've had with HopeRF modules over the years, with them often being outperformed by knockoff nrf24s.

[Yveaux]

@bjornhallberg if you have a cheap sdr lying around you could use that to verify transmission frequencies and signal levels to some extent.
This can rule out bad modules easily.

[bjornhallberg]

@Yveaux I'm embarrassed to admit I did buy a RTL-SDR last year on account of the problems I had with 433MHz sensors on the RFLink gateway and wanting to decode wireless m-bus. But I never really learned how to interpret the data from SDR#. I booted it back up just now and am flailing around in the interface

[canique]

@bjornhallberg it's improbable that there's something wrong with the RFM69
you have a frequency marking at the back, and I've never had a module with a wrong marking.

If an RFM69 is being outperformed by an nrf24, than this is kilometers away from normal. 868 MHz will -if used right- always outperform 2.4 GHz in terms of range. Always.
I tested nrf24 years ago and the first thing I did was: dismiss them immediately. It was crap.

Possible causes for low range can be: wrong settings (like coding for a HW but using a W version) or oscillations @ the gateway or @ the node (noise in the supply/gnd voltage, or nearby the device).

[evb]

@Yveaux, Do you have specific weblinks how to use the rtl-sdr module to debug radio range issues?
I'm searching and reading about it on the net, but till now not yet found a usable debug workflow.

@bjornhallberg are you progressing?

I've also still range issues for my nodes 25m away from my gateway. There are isolated brick walls, sunproof glass, concrete floors, etc between these nodes and the gateway, but when I read about ranges of more then 400m, free sight I suppose, I'm wondering of there are external factors playing a role here...
The nodes are communicating with the gateway, but I loose regularly messages and for door nodes that's a little unfortunate

[canique]

@evb when doing range tests I also tested range through multiple buildings, so there was no clear line of sight. Distance was approximately 200 meters @ 13 dBm (max transmission power restricted due to regulations in the EU) with an external 5dbi antenna. That was the edge where TX was working.

For door nodes, ACKs and retransmissions are a must. I even use that in my temperature nodes.

[evb]

@canique The door nodes sketches have code to treat the acks and retransmission, but still sometimes, the messages are lost...

I thought first that it could be because the pro mini's are running on 1MHz and it was reported that MySensors lib didn't work well at this frequency.
But my door node at 3m away from the gateway, didn't miss yet one single message and the nodes are not freezing, the next time the message is getting through.
So maybe there is 'noise' around who is disturbing the radio communication, but how to detect that without professional expensive equipment?
Maybe via this RTL-SDR thing?

[canique]

@evb If it's intermittent, chances are high that it is noise. But the noise could also stem from the gateway itself. Just a theoretical example to illustrate what I mean: access to an SD card on the gateway => noise. Then it would appear that you randomly lose messages, although it coincides with an access to the SD card.

It's time intensive to spot stuff like this. You could setup a gateway that just scans the same frequency (in promiscous mode) and samples RSSI. You could log each detection of a transmission (when RSSI values go from say -100dBm to -70 dBm) and then check if something is being transmitted from your nodes at the time in question.

Or you could setup a second gateway right next to your first one and see if that one is receiving the message. If one of them is receiving the message, it's probably a gateway issue. If neither is receiving the message there could be another transmission ongoing or it's a node issue.

[boanjo]

I agree with @Yveaux that it's a good idea to fire up the RTL-SDR dongle if you have one. I've been using it quite extensively in the past when working with more plain OOK devices and it's a real eye-opener when you see the spectrum. A lot of noise and garbage where i live at least in the 433Mhz spectrum. In the 868MHz you "shouldn't" have that much (mainly your own devices) perhaps. I used mainly SDR# (SDRSHARP) which (IMHO) is the best option if you run windows. GQRX (linux) does the job too but is a bit harder i think. Move the RTL dongle around (close to GW etc) to see if the actual GW is radiating or if you pick up some other sources.

[evb]

Besides the 'how to' question, what are the models used?
I see the Airspy R2 at 219 euros or the Airspy Mini (dongle) at 138 euros, still quite expensive...
And I see a RTL-SDR kit with dipole antenna for 34.95 dollars or 40 to 50 euros in Europe.

[boanjo]

@evb Ah, sorry i ment to write which models i used too... No i have only tried the really cheap ones

Here are the ones i've used:

The mini is just crap if you compare it with the two bigger sized dongles so i didn't link to it. The two bigger ones work well so it's more up to you what kind of antenna connector you want (and price). See the links for more details
RTL-SDR.COM is ~28$
https://www.ebay.com/itm/272411458376

The middle DVB-T TV kit is ~11$
https://www.ebay.ca/itm/USB2-0-Digital-DVB-T-SDR-DAB-FM-TV-Tuner-Receiver-Stick-RTL2832U-FC0012-Z-/124383617220?oid=182403360614

The how part i actually not much harder than to hook it upp connect to the device, turn the frequency to the wanted range and view the energy mass on the center frequency (it's often good to set the decay a bit longer so the eye can capture the peaks). But if there are questions feel free to ask!

[evb]

Ok, the RTL-SDR.COM kit is ordered. So, now waiting for it

[Tmaster]

The best antena that i tried on rfm69 is the dipole. Direct solder to the rfm69 board. One wire 0.8mm with 8.2cm long on Ant pin,and other in ground in oposite Direction,same size.thos for 868mhz.
I have the gate sensor at 50m outdoor and never seen a miss comunication.

I tried spring antenna and same dipole with an coax cable and didn't work at that range. And a mono-pole without ground plane(wire) have more lost packets( i just try communicate directly,dont have any signal scanner).

[evb]

Did receive yesterday my RTL-SDR.COM USB stick

A first measure of the spectrum around the 868MHz band from my fix desktop computer with the Airspy SDR# software recommended on the RTL-SDR.COM website.

You see clearly the lines in the waterfall blue section when a node sends something.
With my limited knowledge about radio signals and use of this Airspy SDR# Studio software, I think that the spectrum around my house is relatively quiet?

Yet another printscreen with the SDR Console version software

Will install the software now on my laptop to be able to go near the nodes and gateway.
To be continued.

Already a question, my RFM69HW are working in the 868Mhz band, yet I see movement at 867.970 MHz and 868.020 MHz. At the center 868 MHz, I see no movement.
Someone with more knowledge about this, can explain this? I would expect a signal at 868 MHz...

[mfalkvidd]

@evb that's how frequency shift keying works. 867.970 is a "0" and 868.020 is a "1" (in the default MySensors configuration, which uses 50kHz deviation).

[skywatch]

A better SDR would be the airspy R2. It puts the others to shame. (even the mini).

They are cheaper from itead in China than a local supplier (as with some other things).

It showed me that quite a few neighbours use 868MHz for garage door openers.

[mfalkvidd]

@skywatch why would the rtlsdr be insufficient for this task?

[evb]

@skywatch Yes, it should be better for his price of 219 euros
I bought now the rtlsdr for 29 euros directly from the RTLSDR site from their Chinese store.
My purpose was to see if there was much noise or not in my environment.

[skywatch]

@mfalkvidd It all depends on range and signal to noise ratio. The longer the range the better the receiver SNR nedds to be for the same signal to be received - This thread is about 'Range Issues' so therefore it matters.

You might run into problems if your SDR is less sensitive than your chosen tx/rx pair.

[skywatch]

@evb From itead it is 139.84 euros. see here https://www.itead.cc/airspy.html

If you can wait until December they usually have a sale on with about 30% discount. That is how I got them for less than £90 GBP....

[mfalkvidd]

@skywatch that's incorrect. The reason sdr is discussed in this thread is because yveaux said

if you have a cheap sdr lying around you could use that to verify transmission frequencies and signal levels to some extent

[evb]

@skywatch Based on your answers, I see that you seemingly have knowledge in this matter.
I have range issues and just wanted to see if this is due to external environmental factors (neighbors using the same frequency, etc)
And to spend more than 200 euros for something that I will use maybe one time, that was way too much.
(Purchase price manufacturer 140 euros + 10 euros shipping + import taxes because more than 22 euros + customs clearance = 219 euros). I live in Europe where they specialize in taxes

I know you usually get what you paid for, so I don't expect extraordinary performance either...

[boanjo]

@evb Yes, i agree your 868MHz doesn't look that busy and shouldn't be the root cause. Have you been able to establish the specific sensor that you have range issues with (i.e. how it look in the spectrum analyzer)? This is a bit easier when working with "dumb" sensors that sends repeatedly every minute or so. Might be a good idea to add a frequent send (just temporarily while troubleshooting this).

[skywatch]

@evb said in RFM69 Range issues:

@skywatch Based on your answers, I see that you seemingly have knowledge in this matter.

Yup! - 40+ years as a licenced radio amateur along with years working on aircraft radar receivers, air-to-air missile seeker heads, underwater systems and satcomms. We had some great equipment at work!

My point is that if the sdr has a noise floor of say -70dBm and the RFM has a noise floor of -80dBm then the RFM can still 'see' interfereing signals that the sdr doesn't, You can see the signal to noise ration in sdr# by looking at the blue bar to the right of the main spectrum display. To get your sdr set to the optimum level you need to adjust the gain to maximise this figure, then you are operating in the optimal domain for SNR and hence what you can see'hear.

When you say 'range issues' what sort of range are we talking about and in what conditions? Have you tried small movements of the radio module as often that can make a difference.

[evb]

@skywatch said in RFM69 Range issues:

My point is that if the sdr has a noise floor of say -70dBm and the RFM has a noise floor of -80dBm then the RFM can still 'see' interfereing signals that the sdr doesn't, You can see the signal to noise ration in sdr# by looking at the blue bar to the right of the main spectrum display. To get your sdr set to the optimum level you need to adjust the gain to maximise this figure, then you are operating in the optimal domain for SNR and hence what you can see'hear.

Like said my knowledge is limited in this matter.
Can you give more information about how to do it? Or point me to some How To's?
I use the software SDR Console version software and I think you mean this:

But how to adapt the gain? I played with this, you can move the little white line in this vertical bar up and down, but not knowing what I'm doing is not getting me further

I think that from the picture I can conclude that my environment spectrum is quite, today at least.
My problem is that some messages from this node are not making it to the gateway.
Sometimes it works great for days and then suddenly the node does not send anything.
For this I added already a repeater node between, thinking it would get better, but nada.

My idea is now to measure with the SDR at these moments:

• Is the node trying to send a message or not? If I see nothing, then it is a hardware/battery problem of the node itself. If I see the same signal figure, then it is not my node who has a problem
• Is my environment suddenly filled with other signals? blocking my signal?

[skywatch]

@evbI have not used SDR console yet, I was talking about SDR# (a.k.a. SDR sharp) - It is free to download from the airspy web site and will work with your rtlsdr. I can help you with that if you like.

I had similar problems with nodes suddenly not sending. I don't think it is RF signals that are the problem based on my experience. If a repeater didn't help then in my mind that just confirms it.

I suggest to take a step back and look at things again. Power supply, cabling, solder joints, dupont connectors (only useful for testing really).

Really look at solder joints with a good magnifing glass. If in doubt re-solder. Try and solder as many connections as you can. Dupont connectors have a habit of moving due to thermal expansion and contraction as well as through environmental vibrations (think busy road nearby). If you have to use them then take a pair of pliers and squish the connectors so that they really grip! (remembering of course to take the plastic shield off first!)

Can you post a photo of the node that is causing trouble? And again, what distance are you expecting and in what environment? 10M in an apartment block or 100M in a rural setting?

[evb]

@skywatch Thanks for wanting to help me.
I have also the SDR# software installed, but was getting quicker a visual result with the other software, that's why I used this software.
So I'm curious how to set my sdr to the optimum level by adjusting the gain.

For photo's you can see them here : https://forum.mysensors.org/topic/11499/checking-mechanical-locked-doors-by-a-battery-based-windows-door-sensor-node
At the end, you have photos of the problem node.

Like said, today, the node was working.
I've reprogrammed also the repeater node, because I was doing a new temperature repeater node and remarked that my sketch code of the first repeater node was missing my encryption key???
I don't know what did happen here. The repeater node was recognized by the gateway and the controller, so the used encryption key is normally correct. Or is a presentation message not encrypted???
Maybe I did a mistake in saving my github sketch where I delete my keys of course
To be sure, I reprogrammed the repeater node with the correct encryption key.

If it happens again, I will check the spectrum and in case it isn't sending something, I will check the node on soldering errors and move also the node upwards in the doorframe.

I will check also the MySensors docs to see how we can see if a repeater node is working and how we can see that a sensor node is using the repeater and not trying to send directly to the gateway.

[canique]

@evb I'm doing RFM69W based tests rights now. It would be good to compare figures about packet loss in a more scientific way.
Right now I am sending from 5 custom nodes (max 10 meters from custom gateway) every 30 seconds. That's 600 messages per hour.
I have a second gateway only listening on that frequency and counting messages. This 2nd gateway has missed 11 messages in 1 hour.
That's a packet miss rate of 1.8% (while RSSI is always stronger than -80 dBm).
A quick research in the internet shows ~1% should be more or less normal.

One thing that caught my eye was data whitening. If a message contains multiple zeros in succession, this can lead to the receiver not waiting till the end of the message.
Solutions: Using dating whitening or encryption.

[skywatch]

@evb To get the SNR to the ballpark you'll need to do the following.....

1. Tune to a frequency close to the one you want that has no signal.
2. Click the cog or three horizontal lines top left (if using the latter then click 'radio'.
3. Select 'sensitive' and move the gain slider all the way to the left.
4. To the right of the spectrum display adjust the 'offset' so that you can see the noise at the bottom of the spectrum display.
5. Move the gain slider to the right until the noise floor jumps up about 5dB using the scale on the left of the display.
6. Tune to a constant signal and contimue to move the gain slider to the right. Use your eyes and ears to detect changes in the sound of the signal. You want it as clear as possible. The verticla blue graph on the right of the spectrum display will show signal to noise ratio. It takes a little while to catch up with you so work slowly. Eventually you will get good signal to noise ratio with a high signal to noise ration.

Often as the gain goes higher the noise floor increases and this will degrade the signal, hence the need to find the 'sweet spot' where the signal benefits most from the gain amp whilst the noise is still at a low level.

Hope that makes sense. Any touble and I will try and ecplain it differently.

You are amining for the biggest gap between signal peak and noise.

[Grubstake]

@bjornhallberg said in RFM69 Range issues:

Gateway (rfm69hw) initialization:
#define MY_IS_RFM69HW
#define MY_RFM69_FREQUENCY RFM69_868MHZ
#define MY_RFM69_NEW_DRIVER

Maybe I missed someone pointing this out, but the original poster's gateway initialization looks like it's missing something. These defines are not at all obvious, especially with the HW version. Strange things can happen, including unexpectedly low power. Check the API docs and also another thread.

With the HW version of the RFM69 on the gateway, and the RFM69 new driver, in addition to what you list, you need to also include:
#define MY_RADIO_RFM69
#define MY_RFM69HW true

https://www.mysensors.org/apidocs/group__RFM69SettingGrpPub.html#gaf1455cd3427c9dc4c4564542c3dafc16

https://forum.mysensors.org/topic/11316/rfm69hw-atc-not-working