nRF5 action!

Charall

Where to start?

nagelc

@Charall For MySensors NRF5, start with these:

https://forum.mysensors.org/topic/6705/mysensors-nrf5-platform
https://forum.mysensors.org/topic/9266/guide-nrf5-nrf51-nrf52-for-beginners

ncollins

Just started testing my new NRF52805 breakout board. I added a new generic variant to sandeepmistry/arduino-nRF5, added a new board to mysensors nRF5, now I'm testing compatibility.

This new NRF52805 chip is interesting. Way stripped down compared to the rest of the NRF5 line. No PWM, no LPCOMP, only 10 GPIO (maybe 8 usable).

But, it's cheap, ~$2.50 for the EBYTE BT104-BT5005 module, compact, and appropriate for most low-power use cases.

Interestingly, the chip does not support the current MySensors default, but deprecated, data rate of 250Kbps. Today I also learned that modules can only communicate on the same data rate. So I created a ESP8266 + NRF24L01+ gateway at MY_RF24_DATARATE RF24_1MBPS, and it worked!

Still need to test a few things:

I2C
SPI
Power consumption

If all goes well, I'll submit the merge requests and publish this board schematic on openhardware sometime in the next week.

ncollins

@NeverDie I spent the last week really deep diving into NRF BLE & NRF networking capabilities.

Some things I learned:

BLE is a great way for low power communication with a dedicated, physically close parent node
BLE alone does not establish a network, just a link to a central/gateway node. Giving the limited range, that means having multiple gateways, and dedicated links (painful, inflexible)
That led to the creation of BLE Mesh. I found this article from Integra Sources to be really helpful as they talk through the real world limitations they experienced trying to go to production with it. In summary, BLE Mesh is a "managed flood" which quickly leads to network storms if not optimized
Then I found OpenThread
After messing around for few days, I found this guide https://github.com/kyberpunk/openthread which really simplifies getting started. Two docker containers: OpenThread Border Router (sensor network gateway) and MQTT-SN (UDP6 MQTT bridge to MQTT Broker). I was able to setup a network with two NRF52840 dongles + NRF52840 DK, and post messages to my MQTT instance within a couple of hours.

OpenThread is supported on the the NRF52840, NRF52811, NRF52833, as well as 10-15 chips from other vendors.

I really like the ability to route via IP and the fact that all of the nodes communicate with common, well known protocols. I like the secure-by-default approach, and the built in tools for administering networks (commissioning new nodes).

I'm going to keep exploring this route, seems promising....

ncollins

Update on my NRF52805 Breakout:

I2C is now working, just a few small changes to select the right interface
Power consumption looks good, around 1.8uA. The Si7021 module is reading 2.5uA in isolation.

Just need to test SPI and figure out a clever name for the board, and should be good to go.

NeverDie

@ncollins said in nRF5 action!:

Just started testing my new NRF52805 breakout board. I added a new generic variant to sandeepmistry/arduino-nRF5, added a new board to mysensors nRF5, now I'm testing compatibility.

This new NRF52805 chip is interesting. Way stripped down compared to the rest of the NRF5 line. No PWM, no LPCOMP, only 10 GPIO (maybe 8 usable).

But, it's cheap, ~$2.50 for the EBYTE BT104-BT5005 module, compact, and appropriate for most low-power use cases.

Interestingly, the chip does not support the current MySensors default, but deprecated, data rate of 250Kbps. Today I also learned that modules can only communicate on the same data rate. So I created a ESP8266 + NRF24L01+ gateway at MY_RF24_DATARATE RF24_1MBPS, and it worked!

Still need to test a few things:

I2C

SPI

Power consumption

If all goes well, I'll submit the merge requests and publish this board schematic on openhardware sometime in the next week.

Great work! For the gateway you might want to look into Ebyte E01-2G4M27D (https://www.aliexpress.com/item/4000536940861.html?spm=a2g0s.9042311.0.0.60da4c4dGywFPS), which should have better receive sensitivity than the usual nrf24L01 because it includes an LNA.

NeverDie

@ncollins said in nRF5 action!:

@NeverDie I spent the last week really deep diving into NRF BLE & NRF networking capabilities.

Some things I learned:

BLE is a great way for low power communication with a dedicated, physically close parent node

BLE alone does not establish a network, just a link to a central/gateway node. Giving the limited range, that means having multiple gateways, and dedicated links (painful, inflexible)

That led to the creation of BLE Mesh. I found this article from Integra Sources to be really helpful as they talk through the real world limitations they experienced trying to go to production with it. In summary, BLE Mesh is a "managed flood" which quickly leads to network storms if not optimized

Then I found OpenThread

After messing around for few days, I found this guide https://github.com/kyberpunk/openthread which really simplifies getting started. Two docker containers: OpenThread Border Router (sensor network gateway) and MQTT-SN (UDP6 MQTT bridge to MQTT Broker). I was able to setup a network with two NRF52840 dongles + NRF52840 DK, and post messages to my MQTT instance within a couple of hours.

OpenThread is supported on the the NRF52840, NRF52811, NRF52833, as well as 10-15 chips from other vendors.

I really like the ability to route via IP and the fact that all of the nodes communicate with common, well known protocols. I like the secure-by-default approach, and the built in tools for administering networks (commissioning new nodes).

I'm going to keep exploring this route, seems promising....

Very inspiring! I've had high hopes for Thread, and from your report it sounds like maybe now is a good time to jump in.

ncollins

@NeverDie Thanks, and good call on the Ebyte NRF24 with LNA. I have a few laying around that I've been meaning to incorporate into a generic multi-sensor gateway/repeater node. If they're always connected, might as well collect environment data and maybe motion too.

Avamander

@ncollins

There's this nice HAT for the Raspberry Pi you might find interesting :) : https://gitlab.com/electronutlabs-public/indrium-pi-hat

On another topic, I would honestly like to see MySensors using OpenThread or BLE Mesh transport...

ncollins

@Avamander That would make a great gateway. I believe I read a getting started guide by the creator of that Pi Hat, Playing with Thread and MQTT-SN on Nordic nRF52840. The tutorial is a little outdated now, but it's a good overview of the process of setting up a Thread network. There is a lot of good content on that blog.

I agree, but it is still not clear to me what the appropriate relationship between MySensors and OT / BLE Mesh is. It seems there is a lot of overlap in functionality. While I prefer to focus on the NRF5 series, a majority of the community is using ATMEGA328 and NRF24, or LoRa modules, which wouldn't support OT / BLE.

At the very least, I'd like to see MySensors tap into some more of the advanced NRF5 capabilities: OTA, NFC.

There are also a few examples of NRF5 modules running in dual mode: BLE + OT. If it's possible to run 2.4GHz + BLE in parallel, then there is potential to create a MySensors bridge device.

Or maybe leveraging BLE to configure MySensor nodes at runtime: modifying sleep duration, send frequency.

Given the abundance of storage and memory on the NRF5 devices, it is also feasible to create something like Tasmota / ESPeasy for MySensors. Use the same firmware on every node, have the MySensors gateway/controller manage and push configuration templates. Given the great work on MySensors NodeManager and the SmartSleep functionality, it seems like the foundation is already in place.

monte

I wanted to write that Softdevice and ESB are not compatible for use at the same time, but then decided to fact check myself. Seems like now you can use Softdevice and ESB simultaneously. You can either disable softdevice in program, or use Timeslot API to manage access to radio of different protocols.
https://infocenter.nordicsemi.com/index.jsp?topic=%2Fsds_s140%2FSDS%2Fs1xx%2Fconcurrent_multiprotocol_tsl_api%2Ftsl_usage_examples.html&cp=4_5_3_0_8_2
https://devzone.nordicsemi.com/f/nordic-q-a/55847/priority-level-overlap-between-softdevice-and-esb
https://jimmywongiot.com/2020/06/22/radio-timeslot-api-for-the-multiple-protocols/
If this is doable I can't think of a reason, why mysensors shouldn't use softdevice.

acb

(Been away for a while, but saw this and thought I should chime in...)

@monte - I can confirm that this is true. I have custom nRF52832 boards with bootloaders and SoftDevices (S132) that are all currently happily running MySensors code too - I needed the bigger memory footprint AND FOTA updates, so...

As you suggest, you have to disable the SoftDevice before MySensors takes ownership of the radio otherwise there are problems. And depending on how/where you stack things, make sure that @d00616's awesome arduino-NVM layer (pseudo-EEPROM implementation) isn't going to write over anything important. I used Adafruit's FreeRTOS implementation as a starting point, but I'm not sure I fully understand what I've done in the scheduler.

I've not used the Timeslot API myself, but instead do a sort of hard-switch between the two for configuration, BLE OTA updates, etc. So for example, I have a switch in HA to put any individual node into "DFU OTA via BLE" mode - just like in the Nordic examples - then I can use their app to upload new firmware, update the SoftDevice, etc. and then reboot.

Hope that made sense.

Also @NeverDie, on the threads and meshes - I've had good success with Apache MyNewt. Their BLE Mesh implementation (using NimBLE) was relatively straightforward to work with. I made a 5x node test mesh with Publishers and Subscribers, everything provisioned using secure Network and App keys using Nordic's nRF Mesh app. Press a button on one node, all the others do something, etc. At that point, I'm not sure how useful it is to implement the MySensors protocol on top of that, but rather just choose a Publisher to act as a Gateway to your Home Automation software of choice.

monte

@acb so you just compile mysensors code with sofdevice chosen in options, and everything works? That's great news!
Do you mean you use FreeRTOS with mysensors also? So I guess you are using adafruit's bootloader then?

acb

@monte - No, not quite. I'll have to dig back into my local codebase. I use PlatformIO, but I remember having to modify one or more arduino-NVM files to make them compile for Adafruit's nRF52 BSP, something else with how the digital pins were defined (probably minor) elsewhere and of course update MyMainNRF5.cpp to work with their FreeRTOS scheduler. I remember that last part being a bit painful to debug - even with a JLink - but the nodes are pretty solid now.

I can do a diff later this evening if you're interested...?

Yes, Adafruit went with using an RTOS to manage their SoftDevice interfacing. I suppose there are pros/cons to that, but their support was definitely a pro! The useful part is being able to disable it cleanly (!) and then being able to jump back into it when it's safe to do so. FOTA updates with Nordic's app are always solid.

And yes, I've got a custom version of their bootloader running on my boards. Nothing fancy - different pins, buttons, LEDs, etc. I might have modified one or two other bits here and there, but their base should be a good enough starting point.

What boards are you using? And how are you programming them?

monte

@acb said in nRF5 action!:

I can do a diff later this evening if you're interested...?

Yes please, I think I'm not the only one who will appreciate your code :)
I have a bunch of 51822's but wanted to get into 52840. I have paused my research of NRF5 for some time, while I am finishing my other work, but I would like to have more complete experience with NRF5 than what is now provided by mysensors.
I guess FreeRTOS is a way to go, when you have far mo complex MCU then atmega, and far more capable.
I hope that mysensors core team will consider moving forward to more complex, but more powerful architecture.

acb

@monte: Okay, sorry for the delay in getting back to you about this (- FWIW: UK went into another national lockdown, schools closed, so it’s been pretty crazy here the last few days...!)

I’ve dug back into my original code, done some experimenting and have tried to get down to a minimal set of steps necessary to get MySensors code to play "nicely" with a SoftDevice. Hopefully it’ll be helpful as a starting point. But note that minimal is not optimal - or even the best - see more on that below.

Caveats aside...

As far as prepping the nRF52832 goes, this HEX file will load the chip with the following MySensors-compatible regions either “in-mind” or in-place:

0x00000 to 0x00FFF -   4K MBR Region, currently up to 0xAFF.
0x01000 to 0x25FFF - 148K SoftDevice Region, currently with S132 up to 0x2514F.
0x26000 to 0x6FFFF - 272K Application Region, bootloader intelligently fills.
0x70000 to 0x70FFF -   4K VirtualPage? managed by arduino-NVM.
0x74000 to 0x7EFFF -  44K Bootloader Region, currently custom up to 0x78FCB.
0x7F000 to 0x7FFFF -   4K “EEPROM” Area? managed by arduino-NVM.

It’s best to get the bootloader to load the application code (via an FTDI) since it knows where to put it, otherwise you’re doing it manually with a JLink, etc.

The linked bootloader also expects:

UART TX on P0.11
UART RX on P0.12
USR BTN on P0.15
BLE DFU on P0.16
MCU RST on P0.21

Other optional, but useful pins are:

LED1 on P0.13
LED2 on P0.08

Once the above-linked HEX has been flashed (using JLink, BMP, etc.), you can boot the nRF52832 into a state where it is expecting a DFU over UART by grounding USR BTN (P0.15) and resetting (ground P0.21 briefly). LED1 will fade up and down slowly to confirm the new state.

Or you could do this over BLE by grounding both USR BTN and BLE DFU (P0.16) and resetting. LED1 and LED2 will fade up and down more quickly to confirm.

I’ve found the custom tools in Adafruit’s nRF52 BSP for Arduino to be pretty solid for FTDI uploading, but YMMV. Otherwise, it’s Nordic’s nRF Toolbox app.

(Note that if you do go with Adafruit’s BSP, you can call built-in functions in your Sketch code like enterSerialDfu() and enterOTADfu() to perform the same button operations as above.)

As far as prepping MySensors goes, these are (I think) the minimum files and modifications needed to make things work. In summary:

Either comment out the define check for NRF5 right at the bottom of MySensors/drivers/NVM/Flash.h (or define it somewhere) so that the flash driver gets compiled. This is the only file that uses that define, AFAIK.
If you’re using Adafruit’s nRF52 BSP (recommended), you have to make sure MySensors uses the correct pin assignment array. Quick hack is to just replace line 37 of nrf5_wiring_digital.c (MySensors/hal/architecture/NRF5/drivers) with ulPin = g_ADigitalPinMap[ulPin];
Finally, you will probably want to play around with the implementation of MySensors and FreeRTOS in this MyMainNRF5.cpp file (replacing the one in MySensors/hal/architecture/NRF5).

This last file is where FreeRTOS is being used to create a sort of Arduino-like structure with a loop_task() being added to the scheduler that contains both the usual setup() and loop() functions you’ll probably be familiar with from Arduino IDE land - setup() is run by MySensors inside the _begin() function.

Note that this is NOT the best or lowest power way to do this! But it is (possibly) the fastest way to get your existing MySensors sketches up and running.

Remember that this is now an RTOS running on the MCU. So ideally, you probably want to completely dispense with system ticks and move to a setup that only uses interrupts, timers, callbacks, etc. to get the lowest possible power, depending on what you’re trying to do in your Sketch. If you want a quick-and-dirty low power hack, just lower the configTICK_RATE_HZ define in FreeRTOSConfig.h (Adafruit nRF52 BSP) to something like 4 (Hz) or so - but that not the “right” way to do it...

However, that’s outside the scope of this (now, not-so) little post. If you want to dig deeper, I’d suggest familiarising yourself with FreeRTOS and reading through this low power Github issue with Adafruit’s implementation and the associated links mentioned in it - this page from FreeRTOS on Low Power Tickless Idle Support is good too. Here’s an example of an Arduino Sketch using tasks, timers and interrupts - but using Bluetooth - that illustrates a loop-less implementation.

I’ve tinkered down to the 2 or 3 µA region with MySensors using the RTC, portsense interrupts, etc. and you can, of course, go sub-µA using a custom SYSTEM OFF sleep implementation and an external nA region RTC.

I can try to put together a better MySensors integration, if folk are interested. It’ll take time though, especially now that we’re locked down for the next month…

Anyway, hope that was helpful. Any questions, I’ll do my best to respond with what I know.

(P.S. Oh, and you'll need the latest MySensors development branch to get the most stable nRF52832 radio implementation.)

Have fun! :)

monte

@acb thank you for such complete answer!
One more question for now, is this standard, or modified bootloader?
And how many additional steps do I need to take to make this work on nrf52840?

acb

@monte - Glad you found it helpful!

The bootloader I linked to is a custom (modified) one to fit the board I'm using. I remember tinkering with things like buffer sizes to make large BLE DFUs more reliable - I'd have to dig back into my notes to be sure...

But I believe Adafruit's standard nRF52 Feather should keep approximately the same memory regions - which is the "main" hurdle. I think the only other potential issue for the nRF52832 was their use of a BOOTLOADER_SETTINGS region at 0x7F000, so you'll have to check that it's not in the way of the arduino-NVM sections. On the Adafruit Bootloader side, it's all linker-related, nrf52.ld is the file I think.

Or you can, of course, modify arduino-NVM to pick it's sections somewhere else, say, underneath the bootloader, for example. Just be sure to limit how much it erases from there.

I remember nRF52840 being trickier because Adafruit had (and used) both the BOOTLOADER_SETTINGS region (this time at 0xFF000) and a BOOTLOADER_CONFIG region (at 0xFE000 - 2K, or something like that...) which they used for USB VID+PID, UF2 family, etc.

I've got a similar (custom) Adafruit-based bootloader I made for Nordic's nRF52840 Dongle that works with MySensors, if you're interested?

I couldn't get Nordic's default Dongle bootloader to play nicely with MySensors (at the time). They sign their bootloader so you can't replace it through nRF Connect without a JLink/BMP/etc. - you have to wipe the chip completely - which creates another problem. The bootloader also needs to set REGOUT0 in the UICR to 3v so you can see your LEDs and debug the thing without level-shifting (the default is 1.8v). Anyway, it does all work (eventually!) - I've currently got a Dongle serial gateway and Dongle nodes running in various places. Oh, you'll need to up the data rate too, since the 840 doesn't support the MySensors (deprecated) default of 250kbps.

Does that help at all?

What nRF52840 board(s) do you have?

monte

@acb I have a couple of Ebyte's E73 modules. I also have BMP, so it shouldn't be a problem.
If you could share your bootloader for 52840 it would be a great start for my tinkering.

@acb said in nRF5 action!:

I couldn't get Nordic's default Dongle bootloader to play nicely with MySensors (at the time).

So have you gotten any success eventually?
Once again, thank you for such extensive information :) I hope others will appreciate it too.

NeverDie

@monte said in nRF5 action!:

I hope others will appreciate it too.

Yes! Any examples which shed light on how to write a custom bootloader for the nRF52 would be greatly appreciated.

nRF5 action!

16

11.7k

11.2k

113.0k