ESP8266 WiFi gateway port for MySensors

robosensor

So another confirmation of what we knew already.

Yes. It seems that I now understand what is happening. WiFiClient::write() blocks thread until TCP ACK packet received or until timeout (5 seconds). Windows sending TCP ACKs after 200ms timeout. That's why write() delayed for ~200 milliseconds for windows-based controller. More information (and link to non-blocking library) is available here: https://github.com/esp8266/Arduino/issues/922

Do you know how many nRF messages are actually sent during this wifi write call?
The nRF has a rx buffer for 3 messages -- not sure if MySensors uses all 3 of them.

Nodes sending 3-7 (sometimes more) messages with 35 ms delay between messages, so in my case 3-packet hardware buffer in nrf24 is overflowed after 105-140 milliseconds of write() call.

Also disconnection of client during clients[i].write() call causes reboot by watchdog
I guess this is ESP core related. Is this fixed in code 2.0.0?

I did not tried to check this reboots in 2.0.0, but 2.0.0 is much more stable and I never seen any reboots/resets.

Yveaux

@robosensor said:

2.0.0 is much more stable and I never seen any reboots/resets.

That sounds promising! I'll have to give the 2.0.0 core a try myself then.

patrick schaerer

Thanks for your work!
I made my wifigateway today and added variables for a static IP Adress.
I also tried to add an DHT11 to the wifi gateway, but it didn't work. I cannot include those sensors because I need to startup while inclusion and this would restart the gateway.
Mabe someone has an Idea how to add a sensor to the wifi gateway. edit: I found this thread on the forum ... http://forum.mysensors.org/topic/1387/sensors-on-gateway

/**
 * The MySensors Arduino library handles the wireless radio link and protocol
 * between your home built sensors/actuators and HA controller of choice.
 * The sensors forms a self healing radio network with optional repeaters. Each
 * repeater and gateway builds a routing tables in EEPROM which keeps track of the
 * network topology allowing messages to be routed to nodes.
 *
 * Created by Henrik Ekblad <henrik.ekblad@mysensors.org>
 * Copyright (C) 2013-2015 Sensnology AB
 * Full contributor list: https://github.com/mysensors/Arduino/graphs/contributors
 *
 * Documentation: http://www.mysensors.org
 * Support Forum: http://forum.mysensors.org
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * version 2 as published by the Free Software Foundation.
 *
 *******************************
 *
 * REVISION HISTORY
 * Version 1.0 - Henrik EKblad
 * Contribution by a-lurker and Anticimex, 
 * Contribution by Norbert Truchsess <norbert.truchsess@t-online.de>
 * Contribution by Ivo Pullens (ESP8266 support)
 * 
 * DESCRIPTION
 * The EthernetGateway sends data received from sensors to the WiFi link. 
 * The gateway also accepts input on ethernet interface, which is then sent out to the radio network.
 *
 * VERA CONFIGURATION:
 * Enter "ip-number:port" in the ip-field of the Arduino GW device. This will temporarily override any serial configuration for the Vera plugin. 
 * E.g. If you want to use the defualt values in this sketch enter: 192.168.178.66:5003
 *
 * LED purposes:
 * - To use the feature, uncomment WITH_LEDS_BLINKING in MyConfig.h
 * - RX (green) - blink fast on radio message recieved. In inclusion mode will blink fast only on presentation recieved
 * - TX (yellow) - blink fast on radio message transmitted. In inclusion mode will blink slowly
 * - ERR (red) - fast blink on error during transmission error or recieve crc error  
 * 
 * See http://www.mysensors.org/build/ethernet_gateway for wiring instructions.
 * The ESP8266 however requires different wiring:
 * nRF24L01+  ESP8266
 * VCC        VCC
 * CE         GPIO4          
 * CSN/CS     GPIO15
 * SCK        GPIO14
 * MISO       GPIO12
 * MOSI       GPIO13
 *            
 * Not all ESP8266 modules have all pins available on their external interface.
 * This code has been tested on an ESP-12 module.
 * The ESP8266 requires a certain pin configuration to download code, and another one to run code:
 * - Connect REST (reset) via 10K pullup resistor to VCC, and via switch to GND ('reset switch')
 * - Connect GPIO15 via 10K pulldown resistor to GND
 * - Connect CH_PD via 10K resistor to VCC
 * - Connect GPIO2 via 10K resistor to VCC
 * - Connect GPIO0 via 10K resistor to VCC, and via switch to GND ('bootload switch')
 * 
  * Inclusion mode button:
 * - Connect GPIO5 via switch to GND ('inclusion switch')
 * 
 * Hardware SHA204 signing is currently not supported!
 *
 * Make sure to fill in your ssid and WiFi password below for ssid & pass.
 */
#define NO_PORTB_PINCHANGES 

#include <SPI.h>  

#include <MySigningNone.h> 
#include <MySigningAtsha204Soft.h>
#include <MyTransportNRF24.h>
#include <MyTransportRFM69.h>
#include <EEPROM.h>
#include <MyHwESP8266.h>
#include <ESP8266WiFi.h>

#include <MyParserSerial.h>  
#include <MySensor.h>  
#include <stdarg.h>
#include "GatewayUtil.h"


const char *ssid =  "yourSSID";    // cannot be longer than 32 characters!
const char *pass =  "yourPassword"; //
IPAddress local_ip(192,168,0,6);
IPAddress dns_address(192,168,0,1);
IPAddress gateway_ip(192,168,0,1);
IPAddress subnet(255,255,255,0);

//Code for DHT Temp and Hum
/*#include <DHT.h>  

#define CHILD_ID_HUM 9
#define CHILD_ID_TEMP 10
#define HUMIDITY_SENSOR_DIGITAL_PIN 1
long intervall = 300000; //read Temp and Hum every 5 minutes
long last_millis = -300000;
DHT dht;
float lastTemp;
float lastHum;
boolean metric = true; 
MyMessage msgHum(CHILD_ID_HUM, V_HUM);
MyMessage msgTemp(CHILD_ID_TEMP, V_TEMP);
//End of Temp and Hum */

#define INCLUSION_MODE_TIME 1 // Number of minutes inclusion mode is enabled
#define INCLUSION_MODE_PIN  5 // Digital pin used for inclusion mode button

#define RADIO_CE_PIN        4   // radio chip enable
#define RADIO_SPI_SS_PIN    15  // radio SPI serial select

#ifdef WITH_LEDS_BLINKING
#define RADIO_ERROR_LED_PIN 7  // Error led pin
#define RADIO_RX_LED_PIN    8  // Receive led pin
#define RADIO_TX_LED_PIN    9  // the PCB, on board LED
#endif


// NRFRF24L01 radio driver (set low transmit power by default) 
MyTransportNRF24 transport(RADIO_CE_PIN, RADIO_SPI_SS_PIN, RF24_PA_LEVEL_GW);
//MyTransportRFM69 transport;


// Message signing driver (signer needed if MY_SIGNING_FEATURE is turned on in MyConfig.h)
#ifdef MY_SIGNING_FEATURE
MySigningNone signer;
//MySigningAtsha204Soft signer;
#endif

// Hardware profile 
MyHwESP8266 hw;

// Construct MySensors library (signer needed if MY_SIGNING_FEATURE is turned on in MyConfig.h)
// To use LEDs blinking, uncomment WITH_LEDS_BLINKING in MyConfig.h
MySensor gw(transport, hw
#ifdef MY_SIGNING_FEATURE
    , signer
#endif
#ifdef WITH_LEDS_BLINKING
  , RADIO_RX_LED_PIN, RADIO_TX_LED_PIN, RADIO_ERROR_LED_PIN
#endif
  );
  

#define IP_PORT 5003         // The port you want to open 
#define MAX_SRV_CLIENTS 5    // how many clients should be able to telnet to this ESP8266

// a R/W server on the port
static WiFiServer server(IP_PORT);
static WiFiClient clients[MAX_SRV_CLIENTS];
static bool clientsConnected[MAX_SRV_CLIENTS];
static inputBuffer inputString[MAX_SRV_CLIENTS];

#define ARRAY_SIZE(x)  (sizeof(x)/sizeof(x[0]))


void output(const char *fmt, ... )
{
  char serialBuffer[MAX_SEND_LENGTH];
  va_list args;
  va_start (args, fmt );
  vsnprintf_P(serialBuffer, MAX_SEND_LENGTH, fmt, args);
  va_end (args);
  Serial.print(serialBuffer);
  for (uint8_t i = 0; i < ARRAY_SIZE(clients); i++)
  {
    if (clients[i] && clients[i].connected())
    {
//       Serial.print("Client "); Serial.print(i); Serial.println(" write");
       clients[i].write((uint8_t*)serialBuffer, strlen(serialBuffer));
    }
  }
}

void setup()  
{ 
  // Setup console
  hw_init();

  Serial.println(); Serial.println();
  Serial.println("ESP8266 MySensors Gateway");
  Serial.print("Connecting to "); Serial.println(ssid);

  (void)WiFi.begin(ssid, pass);
  while (WiFi.status() != WL_CONNECTED)
  {
    delay(500);
    Serial.print(".");
  }
  Serial.println("Connected!");
  Serial.print("IP: "); Serial.println(WiFi.localIP());
  Serial.flush();
  
  (void)WiFi.config(local_ip, dns_address, gateway_ip, subnet);
  while (WiFi.status() != WL_CONNECTED)
  {
    delay(500);
    Serial.print(".");
  }
  Serial.println("Connected!");
  Serial.print("IP: "); Serial.println(WiFi.localIP());
  Serial.flush();
  
  
  setupGateway(INCLUSION_MODE_PIN, INCLUSION_MODE_TIME, output);

  // Initialize gateway at maximum PA level, channel 70 and callback for write operations 
  gw.begin(incomingMessage, 0, true, 0);
  
  // start listening for clients
  server.begin();
  server.setNoDelay(true);  
  
 /* //Code for Temp and Hum
  dht.setup(HUMIDITY_SENSOR_DIGITAL_PIN); 

  // Register all sensors to gw (they will be created as child devices)
  gw.present(CHILD_ID_HUM, S_HUM);
  gw.present(CHILD_ID_TEMP, S_TEMP);
  
  metric = gw.getConfig().isMetric;
  //End of Temp and Hum */
}


void loop() {
  gw.process();  
  
  checkButtonTriggeredInclusion();
  checkInclusionFinished();

  // Go over list of clients and stop any that are no longer connected.
  // If the server has a new client connection it will be assigned to a free slot.
  bool allSlotsOccupied = true;
  for (uint8_t i = 0; i < ARRAY_SIZE(clients); i++)
  {
    if (!clients[i].connected())
    {
      if (clientsConnected[i])
      {
        Serial.print("Client "); Serial.print(i); Serial.println(" disconnected");
        clients[i].stop();
      }
      //check if there are any new clients
      if (server.hasClient())
      {
        clients[i] = server.available();
        inputString[i].idx = 0;
        Serial.print("Client "); Serial.print(i); Serial.println(" connected"); 
        output(PSTR("0;0;%d;0;%d;Gateway startup complete.\n"),  C_INTERNAL, I_GATEWAY_READY);
      }
    }
    bool connected = clients[i].connected();
    clientsConnected[i] = connected;
    allSlotsOccupied &= connected;
  }
  if (allSlotsOccupied && server.hasClient())
  {
    //no free/disconnected spot so reject
    Serial.println("No free slot available");
    WiFiClient c = server.available();
    c.stop();
  }
  
  // Loop over clients connect and read available data
  for (uint8_t i = 0; i < ARRAY_SIZE(clients); i++)
  {
    while(clients[i].connected() && clients[i].available())
    {
      char inChar = clients[i].read();
      if ( inputString[i].idx < MAX_RECEIVE_LENGTH - 1 )
      { 
        // if newline then command is complete
        if (inChar == '\n')
        {  
          // a command was issued by the client
          // we will now try to send it to the actuator
          inputString[i].string[inputString[i].idx] = 0;
    
          // echo the string to the serial port
          Serial.print("Client "); Serial.print(i); Serial.print(": "); Serial.println(inputString[i].string);
    
          parseAndSend(gw, inputString[i].string);
    
          // clear the string:
          inputString[i].idx = 0;
          // Finished with this client's message. Next loop() we'll see if there's more to read.
          break;
        } else {  
         // add it to the inputString:
         inputString[i].string[inputString[i].idx++] = inChar;
        }
      } else {
        // Incoming message too long. Throw away 
        Serial.print("Client "); Serial.print(i); Serial.println(": Message too long");
        inputString[i].idx = 0;
        // Finished with this client's message. Next loop() we'll see if there's more to read.
        break;
      }
    }
  }
 /* //Code for Temp and Hum
  if (millis() >= last_millis + intervall) {
    float temperature = dht.getTemperature();
    if (isnan(temperature)) {
      Serial.println("Failed reading temperature from DHT");
    } else if (temperature != lastTemp) {
      lastTemp = temperature;
    if (!metric) {
      temperature = dht.toFahrenheit(temperature);
    }
    gw.send(msgTemp.set(temperature, 1));
    Serial.print("T: ");
    Serial.println(temperature);
    }
  
  float humidity = dht.getHumidity();
  if (isnan(humidity)) {
      Serial.println("Failed reading humidity from DHT");
  } else if (humidity != lastHum) {
      lastHum = humidity;
      gw.send(msgHum.set(humidity, 1));
      Serial.print("H: ");
      Serial.println(humidity);
  }
  last_millis = millis();
 }
  
 //End of Temp and Hum */
  
  
  
}

Alex B Goode

Any chance to achieve some success using ESP-01 or I really have to buy ESP-07/12?

I just have some ESP-01 modules already and I wonder if I can use them

Yveaux

@Alex-B-Goode the Spi bus is not available on the esp01 header, so you won't be able to connect the radio then...

hek

But I guess you could use it as a sensor node (mqtt/ethernet).

sdndba

@Yveaux said:

@krajcl Does it only crash when sending this particular message?

It's late (early morning) I did not want to read over all the last three months posts so my question... is there a solution for this problem? I just downloaded the latest library and found the NodeMCU will reboot when a node initializes. Data below. Just point me to the solution if you would please. Thanks very much.

0;0;3;0;9;read: 3-3-0 s=1,c=1,t=0,pt=7,l=5,sg=0:21.54
3;1;1;0;0;21.54
0;0;3;0;9;read: 3-3-0 s=2,c=1,t=38,pt=7,l=5,sg=0:3.44
3;2;1;0;38;3.44
0;0;3;0;9;read: 3-3-0 s=1,c=1,t=0,pt=7,l=5,sg=0:21.76
3;1;1;0;0;21.76
0;0;3;0;9;read: 3-3-0 s=2,c=1,t=38,pt=7,l=5,sg=0:3.44
3;2;1;0;38;3.44
0;0;3;0;9;read: 1-1-0 s=1,c=1,t=0,pt=7,l=5,sg=0:21.11
1;1;1;0;0;21.11
0;0;3;0;9;read: 1-1-0 s=2,c=1,t=38,pt=7,l=5,sg=0:3.44
1;2;1;0;38;3.44
0;0;3;0;9;read: 3-3-0 s=1,c=1,t=0,pt=7,l=5,sg=0:21.76
3;1;1;0;0;21.76
0;0;3;0;9;read: 3-3-0 s=2,c=1,t=38,pt=7,l=5,sg=0:3.44
3;2;1;0;38;3.44
0;0;3;0;9;read: 0-0-0 s=255,c=0,t=17,pt=0,l=5,sg=0:1.5.1
0;255;0;0;17;1.5.1
0;0;3;0;9;read: 0-0-0 s=255,c=3,t=6,pt=1,l=1,sg=0:0

Exception (28):
epc1=0x40203911 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000000 depc=0x00000000

ctx: cont
sp: 3ffefe50 end: 3fff00e0 offset: 01a0

stack>>>
3ffefff0: 3ffeef14 3ffeee8b 3ffeee84 402037d5
3fff0000: 000000ff 00000003 00000006 00000001
3fff0010: 00000001 00000000 3ffeeed8 00000030
3fff0020: 00000000 3fffdc20 3ffef0ac 00000030
3fff0030: 00000003 00000001 00000001 00000000
3fff0040: 00000000 00000000 000000ff 00000023
3fff0050: 00000006 40209ee1 3ffeee64 3ffeee64
3fff0060: 00000000 00000000 3ffeee84 3ffeee84
3fff0070: 3fffdc20 00000000 3ffef0a4 40202582
3fff0080: 402020cc 3ffeee64 3fff1720 402051e6
3fff0090: 00000000 00ffffff 0100000a 3ffef0ac
3fff00a0: 00000000 00000000 00000016 40101941
3fff00b0: 40205499 1100000a 00000000 3ffef0ac
3fff00c0: 3fffdc20 00000000 3ffef0a4 402054c1
3fff00d0: 00000000 00000000 3ffef0c0 40100114
<<<stack<<<

ets Jan 8 2013,rst cause:4, boot mode:(3,6)

wdt reset
load 0x4010f000, len 1264, room 16
tail 0
chksum 0x42
csum 0x42
~ld

ESP8266 MySensors Gateway
Connecting to TESTNETWORK
....Connected!
IP: 10.0.0.17
0;0;3;0;9;gateway started, id=0, parent=0, distance=0
0;0;3;0;9;read: 0-0-0 s=6,c=0,t=13,pt=0,l=0,sg=0:
0;6;0;0;13;
0;0;3;0;9;read: 0-0-0 s=1,c=1,t=16,pt=2,l=2,sg=0:0
0;1;1;0;16;0
0;0;3;0;9;read: 0-0-0 s=4,c=1,t=2,pt=2,l=2,sg=0:0
0;4;1;0;2;0
0;0;3;0;9;read: 0-0-0 s=5,c=1,t=2,pt=2,l=2,sg=0:0
0;5;1;0;2;0
0;0;3;0;9;read: 3-3-0 s=1,c=1,t=0,pt=7,l=5,sg=0:21.76
3;1;1;0;0;21.76
0;0;3;0;9;read: 3-3-0 s=2,c=1,t=38,pt=7,l=5,sg=0:3.44
3;2;1;0;38;3.44
0;0;3;0;9;read: 3-3-0 s=1,c=1,t=0,pt=7,l=5,sg=0:21.97
3;1;1;0;0;21.97
0;0;3;0;9;read: 3-3-0 s=2,c=1,t=38,pt=7,l=5,sg=0:3.44
3;2;1;0;38;3.44

emc2

@Yveaux
Is there any specific reason on why CE is plugged on D2?
I'm asking as I want to add a BME280 sensor directly on my gateway and by default the I2C bus is on D2/D1. It can be changed to any pin with Wire.begin(SDA,SCL) so it is not a major problem, but for user friendliness it would be nicer id CE was plugged on D9 or eventually D4 or D3.

If there is no specific reason can I change MyConfig.h line 329 to #define MY_RF24_CE_PIN 3 safely?

Thanks again for this gateway!

Yveaux

@emc2 said:

Is there any specific reason on why CE is plugged on D2?

Not that I know of.

emc2

Switching it to D9 and adding #define MY_RF24_CE_PIN 3 directly to the program worked indeed.

Now everything seems to work either by moving the I2C bus or the CE. It may be nice to officially free up the I2C port on the next versions of the library by default? It could be easier than to modify each I2C sensor or even display libraries individually.

I will let it run a little while and if I don't see any major problem I will try to put the code in a new topic. Thanks for your fast answer @Yveaux !

rapzak

Hi,

I have added som support, where the gateway can upload data to emoncms, transparant to other functions (not fully testet):

in gatewayutil add this in top of file, after the ARDUINO check:

#define EMONCMS

#ifdef EMONCMS
static WiFiClient emon_client;
const char* emon_host = "emoncms.org";  
const int emon_httpPort = 80; 
const int emon_addr_offset = 10;
#endif

Change this function to:

void incomingMessage(const MyMessage &message) {
//  if (mGetCommand(message) == C_PRESENTATION && inclusionMode) {
//	gw.rxBlink(3);
//   } else {
//	gw.rxBlink(1);
//   }
   // Pass along the message from sensors to serial line
   serial(PSTR("%d;%d;%d;%d;%d;%s\n"),message.sender, message.sensor, mGetCommand(message), mGetAck(message), message.type, message.getString(convBuf));
#ifdef EMONCMS
     if (!emon_client.connect(emon_host, emon_httpPort)) {
      Serial.println("emoncms connection failed");
      }

      emon_client.print(String("GET /emoncms/input/post.json?&node="+ String(message.sender+emon_addr_offset)+"&json={"+String(message.sensor)+":"+message.getString(convBuf)+"}&apikey=XXXXXXXXXXXXXXXXXXXXXXXXXXXX\r\n"));
      Serial.println(String("GET /emoncms/input/post.json?&node="+ String(message.sender+emon_addr_offset)+"&json={"+String(message.sensor)+":"+message.getString(convBuf)+"}&apikey=XXXXXXXXXXXXXXXXXXXXXXXXXXXX\r\n"));
#endif
}

It will upload data with NODE ID and Sensor ID from the network, with an offset of emon_addr_offset...

/Rapzak

scalz

hi.

Has someone already thought about using websockets with wifiGW ??? before I start to reinvent the wheel!
I have already made some tests with websockets but it was not with mysensors. it was not big tests too but I was able to have an html5 client with js. it was a simple test like blink led+serial monitor, receive/sending serial things. very simple!
I did this because I would like to have sort of display on my phone even if there are no controller. and I don't want to use spiffs and webserver. I prefer to have the stuff on client side and esp8266 acting as a simple websocket server. More lightweight I think, to handle something more complex if needed...For the client, then it could be html+boostrap or something more hybrid. I am not expert at this but I should be able to make some poc..but I don't want to waste my time if it already exists! I have lot of other projects so I don't know how long it could take, and maybe someone is faster on this, so it depends..but it's a feature I would like to have in future and I will make it when needed :)

So what do you think ??? dumb idea?

hek

Yeah, thought about it the other day and found this.

https://github.com/Links2004/arduinoWebSockets

Would have been fun to create something cloud.mysensors.org where people could share sensor data (and of course make it embeddable on the forum). Creating a websocket gateway-variant would probably be my first choice if I decide going down that path.

It also seems to support wss which is kind of nice.

scalz

@hek : cool so you think it could be useful too.
yes I use this lib, nice features and simple to use.
so, even if someone has already done something, maybe I will dig a little for fun, time to time...

hek

I haven't done anything yet. The old head is just constantly spinning ideas you know ;)

So please go ahead and see if you can create a MyGatewayTransportWebsocket.cpp

ErrK

@scalz @hek you can look at socket.io if you are thinking of using websockets.

hek

@ErrK said:

socket.io

Yep, use it on my dayjob. Allows downgrade to polling which (I guess) the Arduino library doesn't support. Might be better to use raw websocket on the server side in this case.

scalz

@ErrK : yes I saw it too. but here a discussion about socket.io with esp websockets https://github.com/Links2004/arduinoWebSockets/issues/42
very interesting ;)

Yveaux

@scalz @hek a local server on the ESP serving gateway status overview (connected clients, routing table, sensors presented by clients, data etc) would be a very nice addition to any gateway IMHO.
The ESP has enough power/flash to implement this.

ErrK

@hek: Yes, it has fallbacks to use other protocols. I think there is a setting if you don't want to use all the default protocols.
@scalz: interesting discussion. It looks like the best way shuld be to only use the websockets :)

ESP8266 WiFi gateway port for MySensors

25

11.7k

11.2k

113.1k