ESP8266 gateway + WebServer = Awesome!

[Japio]

Hi,

I used the indications feature of MySensors 2.0 and a webserver to have the gateway provide status information through a web page:

This is how I did it:

In the global section I added this:

#include <ESP8266WiFi.h>
#include <ESP8266WebServer.h>
#include <WiFiClient.h>

ESP8266WebServer WebServer(80);
unsigned long startTime=millis();
unsigned long gwMsgSent = 0;
unsigned long gwMsgRec = 0;

String WebPage = "<h1>ESP8266 MQTT client for Mysensors</h1>";

void setupWebServer();
void showRootPage();
String readableTimestamp(unsigned long milliseconds);

In the setup() Added this:

  Serial.begin(9600);

  setupWebServer();

Defined the following functions:

void setupWebServer()
{
  WebServer.on("/", HTTP_GET, showRootPage);


  WebServer.begin();
  Serial.println("WebServer started...");

}

void indication( const indication_t ind )
{
  switch (ind)
  {
    case INDICATION_GW_TX:
      gwMsgSent++;
      break;

    case INDICATION_GW_RX:
      gwMsgRec++;
      break;

    default:
    break;
  };
}

void showRootPage()
{
  unsigned long runningTime = millis() - startTime;
  String page = WebPage;
  page+="<br>General information</br>";
  page+= "<style> table, th, td { border: 1px solid black;}</style>";
  page+="<table style=\"width:400\">";
  page+="<tr>"; page+=  "<th>Item</th>";        page+=  "<th>Value</th>";                                            page+="</tr>";
  page+="<tr>"; page+=  "<td>Running for</td>";               page+=  "<td>"; page += readableTimestamp(runningTime) ; page+= "</td>";                 page+="</tr>";
  page+="<tr>"; page+=  "<td>Gateway messages sent</td>";     page+=  "<td>"; page += gwMsgSent; page+= "</td>";                                page+="</tr>";
  page+="</table>";

  page+="<br>MySensors gateway information</br>";
  page+= "<style> table, th, td { border: 1px solid black;}</style>";
  page+="<table style=\"width:400\">";
  page+="<tr>"; page+=  "<th>Item</th>";        page+=  "<th>Value</th>";                                            page+="</tr>";
  page+="<tr>"; page+=  "<td>Gateway messages received</td>";     page+=  "<td>"; page += gwMsgRec; page+= "</td>";                                page+="</tr>";
  page+="<tr>"; page+=  "<td>Gateway messages sent</td>";     page+=  "<td>"; page += gwMsgSent; page+= "</td>";                                page+="</tr>";
  page+="</table>";

  Serial.println("WS: Send root webpage");
  WebServer.send(200, "text/html", page);
}

String readableTimestamp(unsigned long milliseconds)
{
  int days = milliseconds / 86400000;

  milliseconds=milliseconds % 86400000;
  int hours = milliseconds / 3600000;
  milliseconds = milliseconds %3600000;

   int minutes = milliseconds / 60000;
   milliseconds = milliseconds % 60000;

   int seconds = milliseconds / 1000;
   milliseconds = milliseconds % 1000;

    String timeStamp;
    timeStamp = days; timeStamp += " days, ";
    timeStamp += hours; timeStamp += ":";
    timeStamp += minutes ; timeStamp +=  ":";
    timeStamp +=seconds; timeStamp += ".";
    timeStamp +=milliseconds;
    Serial.println(timeStamp);
    return timeStamp;
}

And in loop() added the following:

  WebServer.handleClient();

If you have any more idaes on what to add to this page (I am thinking about maintaining a list of registered nodes and sensors) just post a reply!

[mfalkvidd]

Nice work, thanks for sharing!

[Yveaux]

@Japio Very nice indeed! Good to see you like the indication-feature

Btw. I slightly reformatted your post to include code-blocks, to improve readability (use the </> markup feature for code).

[Japio]

@Yveaux thanks for formatting the code.

Would it be possible to get the list of registered nodes and sensors for example?

[Yveaux]

@Japio it's not exported at the api as far as I know. The routing table is stored in eeprom, which you can read from your sketch (but you have to know the internal format then).
A better solution would be to extend the api.

[hek]

The routing table only holds the next jump for a destination. Might be better to implement receive method and build a new data structure (as the ESP has plenty of memory). Then you would be able to pick up both from-field and even presentation messages to show which type of sensors the nodes has attached.

[Japio]

@hek Indeed, the ESP has both plenty of processing power and memory compared to the arduino.

This implementing a receive method, is that a method I can implement in my sketch, with a specific name and signature, and it will be called automatically (like the indication handler)? or does it require changes in de library code? Sounds promising, I can just take out the data I need and store it in a separate structure until the webpage is requested.

Another thing I found "somewhere on the Internet" is an ESP8266 program that, when the wifi connection is not established, or the stored network SSID and password are empty, places the ESP in Access point mode. You can connect to this wifi network and configure the wifi settings. That way you don't need your SSID and password in the sketch.

Too many things to do, and too little time I am afraid.

[hek]

Yes. Receive is standard functionality.

[mfalkvidd]

@Japio perhaps you mean https://github.com/tzapu/WiFiManager ? I'm using that. It is pretty ok, but I would prefer something that had support for the async web server.

[Japio]

Hmm I implemented a receive function like this:

void receive(const MyMessage &message)
{
  Serial.println("@@@@@@@#####@@@@@Message");
  
}

And when I connect my smartmeter sensor I get the following output in the serial console of my MQTT gateway:

0;255;3;0;9;TSP:MSG:READ 20-20-0 s=6,c=1,t=17,pt=7,l=5,sg=0:350
0;255;3;0;9;Sending message on topic: mygateway1-out/20/6/1/0/17
0;255;3;0;9;TSP:MSG:READ 20-20-0 s=7,c=1,t=17,pt=7,l=5,sg=1: 0
0;255;3;0;9;Sending message on topic: mygateway1-out/20/7/1/0/17

So I assume the receive function is not called?

Still, as with the indication function it is fuzzy to me how it i possible that a function is called when it is implemented, but the compiler does not complain when it is not

Is this some kind of inheritance relation?

[Yveaux]

@Japio said:

Still, as with the indication function it is fuzzy to me how it i possible that a function is called when it is implemented, but the compiler does not complain when it is not

The function is defined as weak in the library.
This means the user can implement its own version and then the pointer to the function will be non-null, or you can leave it unimplemented and the pointer to the function becomes null. It's up to the caller (the MySensors library in this case) to test if the pointer is null before calling the function.

[Japio]

@Yveaux ok, thanks, I just have learned something

So I can assume the receive function is not called from the library then? So which function is called, on reception of data from a node?

[Yveaux]

@Japio Looks like you hit a bug in 2.0.0 (receive() was not called for gateways), which has been fixed in development: https://github.com/mysensors/MySensors/commit/24e3dfa36c711b4174ed615408994e8624a2b1f3#diff-9ff5690366e84c42d1e97f40dfe3b8a1R580

Please try with development branch (2.0.1 beta) https://github.com/mysensors/MySensors/tree/development and see if that fixes your issue.

[Japio]

Yes, I see the difference. I'll try the bèta version tomorrow.

[chrille]

@Japio Very nice idea - I just tested it on my ESP8266/RFM69 gateway. However the counters doesn't seem to increment (always shows 0) - is there anything special I need to make sure the indication() handler is called?

[Japio]

No, it worked right out of the box, something that surprised me as well.

You do need Mysensors 2.0 for this.

You can check if you have a myindication.h file in the source tree of the Mysensors library. That file also contains the indications sent.

[chrille]

@Japio I am using the latest beta from github - it must be something else

[chrille]

@chrille said:

I am using the latest beta from github - it must be something else

It works on my NRF24 gateway, but not on the RFM69 gateway... Odd...
I'm curious if someone else with a RFM69 based gateway could this this

• Jan

[chrille]

@Japio It's very easy to get information about available memory and wifi signal - just add

page+="<tr>"; page+=  "<td>Free memory:</td>";               page+=  "<td>"; page += ESP.getFreeHeap() ; page+= " bytes</td>";                 page+="</tr>";
  page+="<tr>"; page+=  "<td>WIFI signal:</td>";               page+=  "<td>"; page += WiFi.RSSI(); ; page+= " dBm</td>";                 page+="</tr>";

[Japio]

Great! This will be added.

I am going to continue with the node and sensor status tomorrow.

[Mark Swift]

Love this, subscribed. It's s such a shame I always has issues with my ESP gateway and had to revert to standard ethernet. The ESP always used to fail upon large amounts of traffic

I believe this was down to blocking code, is this resolved? (off topic).

BTW, is it possible to somehow record any failures?

[Japio]

Yes, this works!

0;255;3;0;9;TSF:MSG:READ,20-20-0,s=255,c=3,t=15,pt=2,l=2,sg=0:0
0;255;3;0;9;TSF:MSG:SEND,0-0-20-20,s=255,c=3,t=15,pt=6,l=2,sg=0,ft=0,st=OK:0100
0;255;3;0;9;TSF:MSG:READ,20-20-0,s=255,c=0,t=17,pt=0,l=5,sg=0:1.5.3
0;255;3;0;9;Sending message on topic: mygateway1-out/20/255/0/0/17
@@@@@@@#####@@@@@Message
0;255;3;0;9;TSF:MSG:READ,20-20-0,s=255,c=3,t=6,pt=1,l=1,sg=0:0
0;255;3;0;9;Sending message on topic: mygateway1-out/20/255/3/0/6
@@@@@@@#####@@@@@Message
0;255;3;0;9;TSF:MSG:READ,20-20-0,s=255,c=3,t=11,pt=0,l=10,sg=0:Smartmeter
0;255;3;0;9;Sending message on topic: mygateway1-out/20/255/3/0/11
@@@@@@@#####@@@@@Message
0;255;3;0;9;TSF:MSG:READ,20-20-0,s=255,c=3,t=12,pt=0,l=3,sg=0:0.1
0;255;3;0;9;Sending message on topic: mygateway1-out/20/255/3/0/12
@@@@@@@#####@@@@@Message
0;255;3;0;9;TSF:MSG:READ,20-20-0,s=1,c=0,t=13,pt=0,l=0,sg=0:
0;255;3;0;9;Sending message on topic: mygateway1-out/20/1/0/0/13
@@@@@@@#####@@@@@Message
0;255;3;0;9;TSF:MSG:READ,20-20-0,s=2,c=0,t=13,pt=0,l=0,sg=0:
0;255;3;0;9;Sending message on topic: mygateway1-out/20/2/0/0/13
@@@@@@@#####@@@@@Message
0;255;3;0;9;TSF:SANCHK:OK

The @@@@###@@@ Message is printed in my receive() function!

Awesome.

[korttoma]

Nice work @Japio

I'm working on a ESP8266 RFM69 Gateway myself and will definitely try this also.

[chrille]

@korttoma I'm looking forward to hear your results, as I couldn't get it to work with RFM69. However, there doesn't seem to be transport specific code for indication(), so I really don't understand why it works with NRF and not RFM

[mfalkvidd]

ESP8266+RFM69 discussion forked to https://forum.mysensors.org/topic/4889/esp8266-rfm69-gateway/

[chrille]

@chrille said:

It works on my NRF24 gateway, but not on the RFM69 gateway... Odd...
I'm curious if someone else with a RFM69 based gateway could this this

I created an issue for this, but I'm still interested in hearing from other RFM69 users

https://github.com/mysensors/MySensors/issues/584

[tekka]

@chrille Please see my comment on github

[korttoma]

Would it be possible to add information about if a controller is connected to the GW WebHMI?

[Mark Swift]

Is this still under development?

[Japio]

Yes, it actually is.

Current features:

• Stores wifi and mqtt broker configuration in flash. When no configuration is found, or the flash button of the ESP is pressed for 10 seconds, released, and then the reset button is pressed, it will restart in access point mode, allowing configuration to be done.
• Status page shows all nodes and sensors connected and the last sensors data and presentation time.
• Status indications on the box itself using a neopixel stick of 8 leds:
  • Gateway status (operation or access point mode)
  • Wifi strength
  • Mqtt connection status
  • Transmit error
  • Receive indication.

I designed a board to nicely fit in a standard Aliexpress housing.

Main problem still is the instability of the ESP, it resets often, and I tried to work around it by maikng sure it resets quickly.

I am planning to make a more elaborate topic, including board designs soon.

[r-nox]

@japio

Where did this all end up?

[pihome]

@mfalkvidd I know this is very old post but I stumble upon this while looking for esp gateway wifimanager git, can you share ur code if you don’t mind, save me time on re-inventing the wheel again...

[mfalkvidd]

@pihome I have a sketch at https://github.com/mfalkvidd/Arduino-Datormagazin-Pomodoro/blob/datormagazin/Arduino-Datormagazin-Pomodoro.ino but I think it is easier to just follow the instructions in the wifimanager readme.