ESP8266 WiFi gateway port for MySensors

[Yveaux]

UPDATE: The official build guide for the ESP8266 gateway can now be found here.
The description in this post is for reference only!

---

Please find an early release of ESP8266 gateway support in my MySensors 1.5 branch:
~~https://github.com/Yveaux/Arduino~~

It has not been tested heavily, but seems to run stable at least with one sensor.
Great feature of this setup is you can run a WiFi gateway with just an ESP8266 and nRF24L01+ radio, no further Arduino (ATMega328) is required!

I'm releasing this now, hoping you can start experimenting with it and help development to get to a stable, official release asap.

Setting things up:

• Install Arduino IDE 1.6.5
• Add support for ESP8266 to Arduino, see https://github.com/esp8266/Arduino (Installing with Boards Manager)
• Get the code form my fork https://github.com/Yveaux/Arduino/archive/master.zip and extract the zip
• Edit libraries/MySensors/MyConfig.h when you use e.g. a different datarate, channel or base radio ID.
• Point your Arduino configuration to the extracted dir (File -> Preferences -> Sketchbook location)
• Restart the Arduino IDE

Compiling the gateway:

• Open the WiFi gateway in the Arduino IDE (File -> Sketchbook -> Libraries -> MySensors -> Esp8266Gateway
• Save it, to allow editing
• Enter your SSID and WiFi password in the 'ssid' and 'pass' variables
• Select the ESP8266 board you're targeting in Tools -> Board. I use an ESP12 module, which is a 'Generic ESP8266 Module' board target
• Verify your sketch. It should compile without errors.

Connecting the radio:

• Make sure to use an ESP8266 module which breaks out pins GPIO 4, 12, 13, 14, 15 (e.g. ESP12 module)
• Connect the radio as described in the header of the gateway code.
• Connect the other signals as described in the header of the gateway code.
• I mounted two switches (for reset & bootload) which your ESP8266 may already be equipped with (e.g. the NodeMCU boards have them most of the time)
• I use an FTDI USB->Serial converter to connect the ESP8266 to my PC, which outputs levels at 3.3V. This is very important as the ESP8266 cannot officially handle 5V signals, and cheap USB->Serial converters often output at 5V levels. When in doubt, please check before connecting!
• Power both the ESP8266 and the nRF24L01+ from a separate 3.3V power supply. The 3.3V supply from many USB->Serial converters is often too weak.
• Decouple the power supply with a large capacitor (e.g. 1000uF) and 100nF, to buffer the input and suppress any noise.

Downloading your sketch:

• Start wit a low upload speed (e.g. 9600baud). It will take ages to download, but I sometimes have issues downloading at high speeds.

• Press reset + bootload buttons.

• Keep bootload pressed while releasing reset

• After a few seconds release bootload.

• The ESP8266 is now in bootload mode, and ready to download your code.

• Choose Upload in the Arduino IDE

• After some time (a lot of dots) upload should finish and your code starts. The serial monitor (115200 baud) shows the progress:

    ESP8266 MySensors Gateway
    Connecting to SSID
    ...........Connected!
    IP: 192.168.1.119
    0;0;3;0;9;gateway started, id=0, parent=0, distance=0
    0;0;3;0;9;read: 123-123-0 s=1,c=1,t=24,pt=5,l=4,sg=0:2810347
    123;1;1;0;24;2810347
  

• This text will be preceded by some garbage from the bootloader (it outputs at a lower baudrate)

• The IP address of the WiFI gateway will currently be obtained from a DHCP server.

Now connect a telnet session (e.g. putty) to the IP address mentioned in the serial output (192.168.1.119 in this case), port 5003 and send some serial commands!

[scalz]

@Yveaux: wow! great work :clap: I will try this when I will have time. thx for your share.

[nikil511]

@Yveaux Great work, well done! :+1:
I will give it a try as soon as possible, and post back.
Thanks!

[Mickey]

Great work!
Wanted to see it for a long time.
Now the way for making a mqtt with the esp8266 is shorter...

[Yveaux]

@Mickey In principle the MQTT gateway should also run on the ESP8266, if you make the same changes to it as I did to the Ethernet gateway example (diff of the files should help a lot: https://www.diffchecker.com/eeo5ahzn).
On the other hand, the pubsubclient (https://github.com/knolleary/pubsubclient) MQTT client implementation by knolleary is heavily tested and works fine on ESP8266 (I tested it myself).
The ESP8266 has a lot more Flash & RAM available than an ATMega, so implementing MQTT should not be a problem.

[DrJeff]

@Yveaux said:

diff of the files should help a lot: https://www.diffchecker.com/eeo5ahzn).

Thanks for the heads up on this service Diff checker I didn't know of this site I will be using it! Thanks

[Yveaux]

Just a quick update on the status of this port. Following has been fixed/implemented:

• LED status updates
• Inclusion mode
• Message signing (untested) (hardware SHA204 not supported, but ESP8266 should be fast enough to do software signing)
• RF69 support (untested -- RF69_IRQ_PIN & RF69_IRQ_NUM are hardcoded in rfm69.h. Make sure to adjust them first!)
• External flash is not supported. It is however currently not used by gateway implementation, and ESP8266 should have sufficient flash onboard if required for some application.

This means full MySensors functionality should now be supported (please let me know if I missed something).

Only problem so far: I need YOU to test, test, test, test... ;-)

[gloob]

Do you have a recommendation where and what to buy to build such a gateway?

[Yveaux]

@gloob I only have a separate ESP12 module on a breakout board (white carrier board in the picture above) to fit it in a breakout board. The carrier board is really too wide for the breakout board, so I had to bring all signals to the side (white/purple wires). Then some reset/download buttons and resistors are still required to use the board...
This setup is not ideal, IMO.

I ordered some NodeMcu boards which seem to have all the required pullup/pulldown resistors onboard, a USB to serial converter and some electronics to automatically download from the Arduino IDE (no need to push reset & download buttons). Only thing needed to make it a gateway would be a connection to the nRF24 radio.
As said, I just ordered the boards and they still have to prove their ease of use. They have to come from China, so it'll take a few weeks probably.

[scalz]

@Yveaux: I have two huzzah. but no time to play with for the moment. but your aliexpress link is very interesting. so cheap! I think I will order one to see. at this price...
Some time ago, I tried to design a very basic esp GW, based on huzzah pinout. But I didn't have time to finish it, as esp was not mysensors compatible and tbowmo is doing something great too. And I feel more secure with a usb GW...
Here : https://github.com/scalz/MySensors-HW/tree/development/MysensorsGW_huzzah
But it tempts me to improve design (not finished), and test it with your work...

[Yveaux]

@scalz Could be interesting to develop a MySensors WiFi gateway board in the future, based on an ESP module.
That should certainly lower the bar for everyone.
I understand this is a chicken-and-egg situation for the port to get stable, but hopefully using a board like the NodeMcu most MySensors users can get a working prototype.

A short inventory of all the ESP modules/breakouts everyone on the forum owns could help in writing a more elaborate how-to on the subject, so please post your modules (and a link to some details when available)

[scalz]

@Yveaux: yes why not. The huzzah I have, are homemade as I had some esp8266-07 (with ceramic antenna and ipex connector) in stock, and it was sold out at adafruit. their files are opensource.
https://www.adafruit.com/products/2471
http://www.electrodragon.com/product/esp8266-wifi-board-full-ios-smd/
but your aliexpress link is cheaper. I will look at the pinout if it is compatible or I will adapt and improve my old pcb project, for fun, and my learning curve :wink:

[Mickey]

I have up and running a mqtt gateway with esp8266 that is connected to ordinary serial gateway (with the arduino) I based it on 2 projects-
ESPHTTPD and tuanpmt mqtt for esp and all is working fine and deliver the serial gateway messages to cloud based mqtt broker and I also have the benefit of configuring everything by webpages (mqtt params ,wifi params)and use all benefits of http server like serving pictures(favicon)and jquery for all web based configuration like this,and now after I see this project I like to remove the arduino and connect the esp directly to nrf but the native sdk is c and I use eclipse.
Is it possible to pull all the relative mysensors code from this work (I see it's mainly in c++) and add it to the code in native esp mqtt (which is mainly in c)?

[Yveaux]

@Mickey I'm afraid it's not that simple... MySensors is purely written in C++, and although this code could be rewritten in C it is a lot of work and requires decent C/C++/MySensors knowledge.
Did you consider doing it the other way around and port your work to fit in this MySensors gateway?
Could be a lot less work (depending on how much code you wrote yourself) as C code compiles (almost) without problems in the Arduino IDE.

[Mickey]

@Yveaux said:

@Mickey I'm afraid it's not that simple... MySensors is purely written in C++, and although this code could be rewritten in C it is a lot of work and requires decent C/C++/MySensors knowledge.
Did you consider doing it the other way around and port your work to fit in this MySensors gateway?
Could be a lot less work (depending on how much code you wrote yourself) as C code compiles (almost) without problems in the Arduino IDE.

those esp projects I mention are very extensive to just port them.(mainly the httpd) but I once saw a guy here that made the mysensors code work on the native sdk ide but only wanted to sell the binaries and wouldn't share his code.

[Andy Pep]

i have a few esp8266mod demo bords a node mcu 9.5 a few esp-01, when i get home i will throw a gateway together :) and test all you like ..i do not mind doing alpha or beta testing if needed ..great work btw ...

also has anyone used this beta .. looks very cool aswell :) http://www.esp8266.com/viewtopic.php?f=29&t=4540

kind regards
andy

[Yveaux]

@Mickey Maybe I don't really understand you application then... The web based configuration link points to a post about runnning gitlab, a local github. I dont get ít, sorry.
Running a webserver next to the gateway code shouldn't be a problem. The ESP has a lot of spare flash and the Arduino port supports a complete filesystem to read/write to it (see http://arduino.esp8266.com/versions/1.6.5-1044-g170995a/doc/reference.html, section "File system object (SPIFFS)")
What's to configure apart from SSID & password? This is just a gateway. Once online you can connect to it and stream all data to/from it using any client supporting the Ethernet gateway.

[Yveaux]

@Andy-Pep said:

i have a few esp8266mod demo bords a node mcu 9.5 a few esp-01, when i get home i will throw a gateway together :) and test all you like ..i do not mind doing alpha or beta testing if needed ..great work btw ...

Thanks! Please join me in testing this code!

also has anyone used this beta .. looks very cool aswell :) http://www.esp8266.com/viewtopic.php?f=29&t=4540

Nice, but as I stressed in my previous post this seems like overkill for a gateway to me.
Of course we can cram a lot of extra code and functionality in the ESP. It has a lot of power and someone might even build a whole home automation suite directly in it (anybody? ;-) )
Let's get this gateway usable, stable and tested. From there you can use it as a base for other applications.

[Andy Pep]

ok i have used a esp8266mod dev board for my test hooked it up flashed the sketch and domoticz is seeing it ok and relaying my sensor data .. not a painfull process at all :) ..
will update tomorrow to see if it fails or goes fubar ..but so far so good :P)

[Yveaux]

@Andy-Pep Awesome! Good to hear its running without any troubles so far! :+1:
Maybe I frightened people by writing a too elaborate description how to set things up :dizzy_face: