I am an old electronic engineer. I started programming of micro-controllers (better to say micro processors) in the 1970/80s with simple hex-terminals and two line displays at an Intel 8080 and 8085 or Zilog Z80. My big jump was an Apple 2e with Z80 Softcard and CPM (from a rather small US company called Microsoft). My first big task was to integrate the (very modern !) TEAC drives with 640 kBytes into CPM, instead of the Apple drives with 146 kBytes. Microsoft had sent the sources of CPM to help. My "office software" was WordStar. I programmed new printer drivers to make the changing of different printwheels possible when printing my dissertation. I defined colon-commands for WordStar to synchronize printing and printer handling. There was only space of 256 bytes for custom drivers. Of course, they had to be developed with a macro assembler. Often used branches to the same destination I replaced by preloading the stack (once) and jumping via return (several times), winning two bytes for every jump. And the technical equipment was expensive. I remember the cost of 10.000 DM (2 DM = 1 €) for a 1 MB memory extension of a VAX 750 (our mainframe computer at the institute I worked with).

Times have changed a lot. Sometimes I imagine to have a time machine, taking a modern USB-stick with 128 GB and reading speed of 400 MB/s, and jumping to the past to discuss technology development with my old friends. At that time, the technical features of today were not thinkable. 128 GB in a thimble and 400 MB/s transfer speed would have been so believable like interstellar spaceship jumps to the next galaxy.

In the last decades I experienced a lot of trends and hypes, based on convincing technical details and great promises of the industry. For example the arise of MAP (Manufacturing Automation Protocol) introducing Token Passing Bus for real-time communication and announcing the end of Ethernet. Or the international fieldbus (which led to a standardization of 8 different fieldbus systems with thousands of pages), that promised to make the world of automation cheaper and to save a lot of copper cables. And enabling the plug-and-play of different supplier's devices at the same network (ha, ha, ha).

But Ethernet survived and the problems of bottle necks by bus collision simply vanished because nobody uses Ethernet with a bus. But with tree structures of routers and switches on full duplex lines. No collisions, extremely high speed, and no discussion about the waste of copper and expensive cables.
And WLAN seems to go a similar way, increasing speed, increasing reliability and decreasing prices.

OK, a long introduction for a short view on home automation. I think, the main problem is, that (especially big) companies do not want to be simple device suppliers. They want to create dependencies via networking. There is no interest to present universal usable equipment inter-operating with the equipment of their competitors. And of course there is no interest, that users look behind the curtain of the theater and modify the devices to their local needs or even repair defect devices.

From that point of view, motions like Arduino (and possibly MySensors) are game changers. Meanwhile even big companies support Arduino and open the chance for users (as programmers) to test their own ideas and build an automation world fitting to their local needs.
But the big companies will try to create dependencies wherever it is possible. They promise to make your life more easy, e.g. by presenting pre-compiled (fantastic) libraries, or to make the communication secure with their easy to handle key management. The most simple and effective way for a dependency is, to make the devices (or software and/or software development) depending on a cloud connection.
I can only hope, that users/programmers see the traps and endeavor for their independence.

What we need for home automation is/are

1. a communication background, that is simple to handle and provides tools for different applications and new ideas. Like TCP and UDP with IP provided via sockets without any restrictions. (I am using broadcast communication like the CAN bus in cars and I will not use a network, which does not support broadcasting.)
2. device suppliers with the interest to provide reliable and reasonable devices for home automation without creating dependencies via clouds or hidden protocols and features.
3. developers with new ideas to create a real "smart" home, not only "the big remote control" for television, coffee maker, light modification and so on.

I think, it is a good time for learning and testing, building own devices and improve technologies (and software). Big companies still put their pants on one leg at a time as we do and we should not hesitate to present our ideas and improve our capabilities in fruitful discussions.

Thanks for reading ... and (hopefully) for your comments.

Thanks to the thread Everything nRF52840 from NeverDie I found the courage to start a project for developing my own Beacons based on the Bluetooth Core Specification 5.2, Volume 6. The idea is also, to use broadcasts for providing sensor data to any interested listener in my home network. For this I defined a PDU with 12 measurement values of 16 bit and some additional management information. The result is a BLE advertising packet with the maximum payload of 31 bytes.

My development environment is Eclipse/Sloeber (Eclipse C++ with an Arduino Plugin) and an Arduino Nano BLE 33. I do not use the mbed-OS nor the BLE library of the Nano board, but started to program the nRF52840 directly.

The current hurdle is to receive data with the smartphone. I programmed a BLE-Scanner on Android and tested it with some bought Beacons by displaying the address, the rssi-value and the payload bytes (I hope, I can trust the getBytes-function of the accordingly subclass in ScanResult, because the data of the bought Beacons seem to be encrypted).
But my Ardunino board is not seen.

Does someone know, whether the BLE-Scanner of Android detects simple advertizing packets (not scannable and not connectable)?

My next step will be to program a scanner with the Arduino board to prove, that my advertiser is running.

I am new here at MySensors. But I have my own ideas and do not want to adapt to a given middleware or networking. My reason to be here is the know-how presented by the many contributors of the forum.

I like the idea of Arduino very much. Having your basic routines running on nearly any hardware is really a nice thing. And after a first look into MySensors software, I also like it. Though I will not use it, because I have other ideas on microcontroller communication.

Do we really need to go with the "mainstream"? I think, the mainstream of software development has become a kind of "gaming" in the last years. It seems not to be necessary to know anything about programming, just be able to move your mouse and make some drags and drops her and there. Being an expert based on the knowledge of other people (companies).

As long as here are real experts talking about details of microcontrollers and their application, I will stay here.

Hello @mfalkvidd ,
thank you very much for the valuable link to the Android app, which I followed further to the sources on Github. It seems, that the author uses other vocabulary for the features of BLE than those described in the BLE specification (v 5.2), which makes it difficult for me to understand his program at once. The libraries I used are part of the IDE (Android Studio) and offer similar resources.

I just started to develop a scanner based on my Beacon software for the Arduino Nano BLE 33. As soon as the radio data exchange between two Arduino boards is running, I will care again for the Android app.

Update

Meanwhile I made progress with my project. Together with some colleagues I want to install a real-time measurement system with more than 30 channels (i.e. 30 Nano BLE 33 measuring accel. and gyro.) and one evaluation unit (another Nano BLE 33 combined with other computers). The 6 values (2*x,y,z) of 16 Bit each shall be transmitted with 10 measurements per second.

Instead of spontaneous sending beacons with new measurement values from the 30 devices (to many collisions, real-time simply not possible) I use a master-slave polling, done by the evaluation unit. And I continue using non-connectable/non-scannable BLE-Beacons, for the request messages and for the response messages.

Meanwhile, my beacons can be detected by typical BLE scanner APPs on my smartphone (Android 10). I do not know, why that did not work at the beginning. May be I corrected some mistakes incidentally in the last months.

I am working also on a channel hopping (in case of disturbances by other devices with BT and WLAN) and using more channels for higher measurement speed. But the implementation of that will start later.

8 hours later

Now I found a reason for the behavior of the Scan-APPs: My request beacons (without data, only user address of 6 bytes) are detected, but not the response beacons with data.
I do not need the smartphones now (the speed of about 500 beacons per second is to fast for them), so I am happy with my solution.

Hello @monte and @TheoL
I wanted to join your discussion here, but that may go off topic for this thread. So I started a new thread here, hoping to meet your theme.

I am an old electronic engineer. I started programming of micro-controllers (better to say micro processors) in the 1970/80s with simple hex-terminals and two line displays at an Intel 8080 and 8085 or Zilog Z80. My big jump was an Apple 2e with Z80 Softcard and CPM (from a rather small US company called Microsoft). My first big task was to integrate the (very modern !) TEAC drives with 640 kBytes into CPM, instead of the Apple drives with 146 kBytes. Microsoft had sent the sources of CPM to help. My "office software" was WordStar. I programmed new printer drivers to make the changing of different printwheels possible when printing my dissertation. I defined colon-commands for WordStar to synchronize printing and printer handling. There was only space of 256 bytes for custom drivers. Of course, they had to be developed with a macro assembler. Often used branches to the same destination I replaced by preloading the stack (once) and jumping via return (several times), winning two bytes for every jump. And the technical equipment was expensive. I remember the cost of 10.000 DM (2 DM = 1 €) for a 1 MB memory extension of a VAX 750 (our mainframe computer at the institute I worked with).

Times have changed a lot. Sometimes I imagine to have a time machine, taking a modern USB-stick with 128 GB and reading speed of 400 MB/s, and jumping to the past to discuss technology development with my old friends. At that time, the technical features of today were not thinkable. 128 GB in a thimble and 400 MB/s transfer speed would have been so believable like interstellar spaceship jumps to the next galaxy.

In the last decades I experienced a lot of trends and hypes, based on convincing technical details and great promises of the industry. For example the arise of MAP (Manufacturing Automation Protocol) introducing Token Passing Bus for real-time communication and announcing the end of Ethernet. Or the international fieldbus (which led to a standardization of 8 different fieldbus systems with thousands of pages), that promised to make the world of automation cheaper and to save a lot of copper cables. And enabling the plug-and-play of different supplier's devices at the same network (ha, ha, ha).

But Ethernet survived and the problems of bottle necks by bus collision simply vanished because nobody uses Ethernet with a bus. But with tree structures of routers and switches on full duplex lines. No collisions, extremely high speed, and no discussion about the waste of copper and expensive cables.
And WLAN seems to go a similar way, increasing speed, increasing reliability and decreasing prices.

OK, a long introduction for a short view on home automation. I think, the main problem is, that (especially big) companies do not want to be simple device suppliers. They want to create dependencies via networking. There is no interest to present universal usable equipment inter-operating with the equipment of their competitors. And of course there is no interest, that users look behind the curtain of the theater and modify the devices to their local needs or even repair defect devices.

From that point of view, motions like Arduino (and possibly MySensors) are game changers. Meanwhile even big companies support Arduino and open the chance for users (as programmers) to test their own ideas and build an automation world fitting to their local needs.
But the big companies will try to create dependencies wherever it is possible. They promise to make your life more easy, e.g. by presenting pre-compiled (fantastic) libraries, or to make the communication secure with their easy to handle key management. The most simple and effective way for a dependency is, to make the devices (or software and/or software development) depending on a cloud connection.
I can only hope, that users/programmers see the traps and endeavor for their independence.

What we need for home automation is/are

1. a communication background, that is simple to handle and provides tools for different applications and new ideas. Like TCP and UDP with IP provided via sockets without any restrictions. (I am using broadcast communication like the CAN bus in cars and I will not use a network, which does not support broadcasting.)
2. device suppliers with the interest to provide reliable and reasonable devices for home automation without creating dependencies via clouds or hidden protocols and features.
3. developers with new ideas to create a real "smart" home, not only "the big remote control" for television, coffee maker, light modification and so on.

I think, it is a good time for learning and testing, building own devices and improve technologies (and software). Big companies still put their pants on one leg at a time as we do and we should not hesitate to present our ideas and improve our capabilities in fruitful discussions.

Thanks for reading ... and (hopefully) for your comments.

I'm happy, finally I got it working. Not all, what I wanted, but at least I detected the limits.

So for those, who are interested in direct programming the nRF52840 on the Arduino Nano BLE 33 (ask me if you have questions about that):

My Scanner (on the Nano) runs perfect and detects all Advertisers around (a Beacon from EM Microelectronic, the default Advertiser of the Nordic nRF52840 DK and my own private Beacon on the second Nano).

But ... my Android App does not detect my private Beacon. And now I can see the difference. The EM Beacon does scannable and and connectable advertising (pdu type ADV_IND) and the Nordic does scannable (pdu type SCAN_RSP). My own Beacon does non-connectable and non-scannable advertising (pdu type ADV_NONCONN_IND), which is obviously ignored by the Android BLE library (and I do not know, how to make that working or if it is possible at all).
I tested several scanner Apps on Android and they behave the same way as my App. So it seems to be a fundamental problem and I will have to wait for an Android expert.

I am new here at MySensors. But I have my own ideas and do not want to adapt to a given middleware or networking. My reason to be here is the know-how presented by the many contributors of the forum.

I like the idea of Arduino very much. Having your basic routines running on nearly any hardware is really a nice thing. And after a first look into MySensors software, I also like it. Though I will not use it, because I have other ideas on microcontroller communication.

Do we really need to go with the "mainstream"? I think, the mainstream of software development has become a kind of "gaming" in the last years. It seems not to be necessary to know anything about programming, just be able to move your mouse and make some drags and drops her and there. Being an expert based on the knowledge of other people (companies).

As long as here are real experts talking about details of microcontrollers and their application, I will stay here.

Hello @mfalkvidd ,
thank you very much for the valuable link to the Android app, which I followed further to the sources on Github. It seems, that the author uses other vocabulary for the features of BLE than those described in the BLE specification (v 5.2), which makes it difficult for me to understand his program at once. The libraries I used are part of the IDE (Android Studio) and offer similar resources.

I just started to develop a scanner based on my Beacon software for the Arduino Nano BLE 33. As soon as the radio data exchange between two Arduino boards is running, I will care again for the Android app.

Thanks to the thread Everything nRF52840 from NeverDie I found the courage to start a project for developing my own Beacons based on the Bluetooth Core Specification 5.2, Volume 6. The idea is also, to use broadcasts for providing sensor data to any interested listener in my home network. For this I defined a PDU with 12 measurement values of 16 bit and some additional management information. The result is a BLE advertising packet with the maximum payload of 31 bytes.

My development environment is Eclipse/Sloeber (Eclipse C++ with an Arduino Plugin) and an Arduino Nano BLE 33. I do not use the mbed-OS nor the BLE library of the Nano board, but started to program the nRF52840 directly.

The current hurdle is to receive data with the smartphone. I programmed a BLE-Scanner on Android and tested it with some bought Beacons by displaying the address, the rssi-value and the payload bytes (I hope, I can trust the getBytes-function of the accordingly subclass in ScanResult, because the data of the bought Beacons seem to be encrypted).
But my Ardunino board is not seen.

Does someone know, whether the BLE-Scanner of Android detects simple advertizing packets (not scannable and not connectable)?

My next step will be to program a scanner with the Arduino board to prove, that my advertiser is running.

Hello @mmarks3141,

I am just learning about nRF52840. My IDE is Eclipse/Sloeber and Arduino NANO BLE 33. And I do not want to use the mbed-OS, because it destroys the timing of my self developed libraries running on any Arduino-Board.
Your question is about using a serial interface, which depends on the firmware of your board. If it is mbed-OS, please look to the documentation of mbed: https://os.mbed.com/handbook/Serial.
If it is not mbed-OS, look for the resources of Your boards firmware/libraries.
For me (Arduino), it would be the initialization with Serial.begin(bitrate) in setup() and using Serial.print() for the output somewhere in loop().
By the way, thanks to @NeverDie for the examples.