Other Controller get disconnected when sending any command to the Eth. Gateway (v2.0)

[soif]

Short :

I'm trying to send mysensors message to the ethernet gateway. I've also another Controller (domoticz) connected at the same time. Whatever message I type (even a simple CR) cause Domoticz to be immediatly disconnected.

Long :

As I knew that my 1.51 version is not supposed to support mutiple connection to the Eth gateway, I've downloaded the latest dev version from github (# 1f27c3d ) and compiled the GatewayW5100.ino example and just added at top of the file (around line 108, just before the first include):

#define MY_GATEWAY_MAX_CLIENTS 3

I suppose this makes the gateway able to handle multiple connections at the same time.

It seems to work, as I can see messages printed on the telnet, while Domoticz is also connected, sending hearbeat messages.

But sending any message, or just sending a CarriageReturn) from telnet, just make Domoticz to be immediately disconnected, with a message in its log file :

MySensors: Connection reset!
TCP: Reconnecting in 30 seconds...

obviously letting him away for 30 seconds is not acceptable in production.

According to Domoticz MySensors source code, the error may be caused by :

(error == boost::asio::error::eof) ||
(error == boost::asio::error::connection_reset)

Questions:

• Am I doing something wrong (my C++ understanding being very limited?
• Is #define MY_GATEWAY_MAX_CLIENTS 3 supposed to work at this time?
• Is this a bug from MySensors? From Domoticz?
• Is it definitively not possible to send messages when another controller is also connected?

Any help would be greatly appreciated !

[hek]

It could very well be a bug on the MySensors side. Multi connected controllers hasn't been tested (by me at least).

Appreciate your effort trying to identify the problem. Suggest you start adding debug prints in the critical section where connection are being handled and thrown away.

Probably somewhere here:
https://github.com/mysensors/Arduino/blob/7e3f5d2d5410117367dbb52b5c1e578f40b2e590/libraries/MySensors/core/MyGatewayTransportEthernet.cpp#L273-L298

[soif]

Thank you for poiting me to this part of the code. It immediately showed something interesting :

Your highlighted code is for ESP8266 but i'm using W5100 (the recommended one)...
If I read correctly the code just following for the W5100, I see that :

// W5100/ENC module does not have hasClient-method. We can only serve one client at the time.

• Would it mean there is absolutely no way to get multiple clients connected with the W5100?
• Not codable?
• only workaround it to buy a ESP8266 Wifi (wifi sucks) module ?

Sorry my C++ knowledge is very limited...

[hek]

Hmm.. good that I write comments sometimes...

Well... Obviously I didn't find any good way to do it. If you can find a way, please pursue it.

[soif]

Yep, comments are often useful when we are not that lazy to include them

BTW I've found a topic telling about modifying the Eth library to get the Source IP, and being then able to support multiple clients. I don't know if it really helps.

Also found this where the guy says that W5100 supports 4 sockets (so max 4 clients) but the Eth lib can NOT distinguish them if they are on the same port (might be a a workaround, to open 4 port instead of one, ie 5003,5004,5005,5006). He's finally submitted a patch against the Eth lib to allow 4 different sockets on the same port.... But the more interesting seems to be a comment by "Gene" who has posted complete sketch showing how to connect multiple telnet clients . Here is his code:

/* exampleMultiTelnetServer.ino */
/* 
 *  Created: May 21, 2015
 *
 *  Created by Gene Reeves to demonstrate how the W5100 eithenet handles multiple connections
 *  using a single server object instance.
 * 
 *  once the sketch is loaded on your arduino and you see the start up banner display in the
 *  Serial Monitor, fire up your favorite telnet client and connect a few instances.
 *  
 *  from the serial monitor you can send text to one or all clients. Use '*" for the ID to send
 *  to all, otherwise use the ID you wish to send to, using this format to send,
 *   "ID:TEDXT_TO_SEND".  That should make everything as clear as mud  :-)
 *
 *  Dependencies:
 *
 *  uses LinkedList class from here https://github.com/ivanseidel/LinkedList
 *  to store list of connected clients.
 *
 *  uses StringStream class from here https://gist.github.com/cmaglie/5883185
 *  Not sure why this was not a part of the StringObject to start with..
 *
 *  uses elapsedMillis class from here http://playground.arduino.cc/Code/ElapsedMillis
 *  much improved version (for Teensy brand mcu's) here https://www.pjrc.com/teensy/td_timing_elaspedMillis.html
 *  Unless I am mistaken, I beleive this code was originated by Paul Stoffregen of www.pjrc.com
 *  If you are not famialar with Paul or his Teensy microcontroller, you would do yourself a 
 *  solid to spend some time reviewing his code and you can't beat the Teensy 3.1 for 
 *  price/performance/support!!!   
 *  Teensy 3.1 - 32bit-ARM  mcu @ 72MHz (overclockable to 168MHz) w/ 256K flash / 64K ram / USB
 *  for under $20.  https://www.pjrc.com/teensy/index.html
 *
 *  BTW, I am in no way affilaited with Paul or PJRC, I just think he built an awesome mcu.
 *
 */
 
 
#include <SPI.h>
#include <LinkedList.h>
#include <elapsedMillis.h>
#include <StringStream.h>
#include <Arduino.h>
#include <EthernetUdp.h>
#include <EthernetServer.h>
#include <EthernetClient.h>
#include <Ethernet.h>
#include <Dns.h>
#include <Dhcp.h>
#include <IPAddress.h>
#include <Server.h>
#include <Client.h>
 
 
/*  DEFINES  */
                      
/*****************************************************/
/*  Change the following to suite your environment.  */
#define TELNET_SERVER_PORT 23
#define TELNET_SERVER_MAC 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED
#define TELNET_SERVER_IP 192, 168, 1, 177
#define TELNET_SERVER_GATEWAY 192, 168, 1, 1
#define TELNET_SERVER_NETMASK 255, 255, 255, 0
#define SERIAL_BAUD 115200
#define WELCOME_BANNER_STRING "Welcome to MultiTelnetServer.\r\nType 'exit' to disconnect\r\n"
#define GOODBYE_BANNER_STRING "Come Back Again,   Goodbye.."
 
#define APP_BANNER_STRING "\r\n\r\n\tMultiTelnetServer Example\r\n  This example application strives to\r\ndemonstrate using the Wiznet W5x00\r\nEthernet Shield's Arduino library usage\r\nwhen serving mulitple concurrent\r\nconnects using a single 'server object'\r\ninstance."
#define SERVER_LISTENING_STRING "\tServer Listening on"
 
/******************************************************/
 
 
/*
 *  This is required to add RemoteIP, RemotePort support to the EthernetClient class
 *  I stole it from here http://forum.arduino.cc/index.php?topic=210857.0
 */
//namespace ethernet_fix
//{
  #include <utility/w5100.h>
 
  template<typename Tag, typename Tag::type M>
  struct AccessMember{
    friend typename Tag::type get(Tag){ return M; }
  };
 
  struct EthernetClient_Socket{
    typedef uint8_t EthernetClient::*type;
    friend type get(EthernetClient_Socket);
  };
 
  template struct AccessMember < EthernetClient_Socket, &EthernetClient::_sock > ;
 
  IPAddress RemoteIP(EthernetClient &c){
    byte remoteIP[4];
    W5100.readSnDIPR(c.*get(EthernetClient_Socket()), remoteIP);
    return (remoteIP);
  }
 
  uint16_t RemotePort(EthernetClient &c){
    return W5100.readSnDPORT(c.*get(EthernetClient_Socket()));
  }
//} // namespace "ethernet_fix"
/*
 *  EthernetClientEx - Because there is still no RemoteIP or RemotePort
 *  properties built in to th current EthernetClient!!
 *
 */
 
class EthernetClientEx :
  public EthernetClient
{
protected:
  uint8_t _sock; // hack to get access to socket #
public:
  EthernetClientEx(const EthernetClient &orig) : EthernetClient(orig) {}
  IPAddress remoteIP() { return RemoteIP((EthernetClient &)(*this)); }
  int16_t remotePort() { return RemotePort((EthernetClient &)(*this)); }
 
  bool isSameSock(const EthernetClientEx &c) { return (_sock == c._sock); }
};
 
 
/*
 *  ClientItem is class to wrap EthernetClient for storage on linked list
 *  we will use it to add a ts (millis at connection time) and an index.
 *
 */
class ClientItem
{
public:
  unsigned long timestamp;
  int index;
  size_t recv_cnt;
  String recv_buffer;
  EthernetClientEx *client;
 
  ClientItem() { timestamp = millis(); index = -1; client = (EthernetClientEx *)false; }
  virtual ~ClientItem() { delete client; /* release memory */}
   
  unsigned long elapsed(void) { unsigned long ts = millis(); return (ts - timestamp); }
};
 
/*
 *  ClientList is linkedlist for storing connected clients. 
 *  For now, we'll just use it to store our clients, but should
 *  expand this close to include testing all clients for disconnects,
 *  testing all clients for pending recv data and sending data
 *  to all connected clients.
 */
 
class LinkedClientList :
  public LinkedList<ClientItem*> //ClientList;
{
public:
  //EthernetServer *server;
 
  ClientItem *getClientItem(int idx)
  {
    for (ListNode<ClientItem *> *n = root;n;n=n->next)
    { 
      if (n->data->index == idx)
        return n->data;
    }
    return (ClientItem *)0;
  }
   
  bool exists(const EthernetClientEx &ece)
  {
    for (ListNode<ClientItem *> *n = root; n; n = n->next)
      if (n->data->client->isSameSock(ece))
        return true;
    return false;
  }
 
  void drop(int idx)
  {
    ClientItem *cli;
    int real_idx = -1;
    int ci = 0;
    for (ListNode<ClientItem *> *n = root; n; n = n->next)
      if (n->data->index != idx)
        ci++;
      else
      {
        cli = n->data;
        real_idx = ci;
        break;
      }
 
    if (real_idx != -1)
    {
      if (cli->client->connected())
        cli->client->stop();
      cli->index = -1;
      remove(real_idx);
    }
  }
 
  size_t send(const String &s, int idx)
  {
    size_t ret = 0;
    ClientItem * cli = getClientItem(idx);
    if (cli)
    {
      ret = cli->client->print(s);
      cli->client->flush();
    }
    return ret;
  }
 
  size_t send2All(const String &s)
  {
    size_t ret = 0;
    for (ListNode<ClientItem *> *n = root; n; n = n->next)
    {
      ret += n->data->client->print(s);
      n->data->client->flush();
    }
    return ret;
  }
};
 
LinkedClientList ClientList;
 
// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network.
// gateway and subnet are optional:
byte mac[] = { TELNET_SERVER_MAC };
IPAddress ip(TELNET_SERVER_IP);
IPAddress gateway(TELNET_SERVER_GATEWAY);
IPAddress subnet(TELNET_SERVER_NETMASK);
 
// buffer to hold input from console
StringStream con_buffer("");
 
// telnet defaults to port 23
EthernetServer server(TELNET_SERVER_PORT);
 
/* forward's */
void server_check_new_connections(void);
void server_check_client_recv(void);
int server_accept(EthernetClientEx *enetcli);
bool clientExists(const EthernetClientEx &c);
bool test_for_cmd(const String &itm, const String &src);
size_t console_recv(void);
void proc_line(StringStream &ss);
 
size_t client_recv(ClientItem *cli);
void client_disconnecting(ClientItem *cli);
void client_disconnected(ClientItem *cli);
 
size_t send_to(int idx, const String &str);
size_t send_to_all(const String &str);
 
/*********************************************************/
void setup()
{
 
  /* add setup code here */
  // reserve some space for input buffer
  con_buffer.reserve(256);
 
  // Open serial communications and wait for port to open:
  Serial.begin(SERIAL_BAUD);
  delay(3000);  // 3 second delay for Serial to connect.
 
  pinMode(10, OUTPUT);
  digitalWrite(10, HIGH);
  pinMode(4, OUTPUT);
  digitalWrite(4, HIGH);
 
  // initialize the ethernet device
  Ethernet.begin(mac, ip, gateway, subnet);
  // start listening for clients
  server.begin();
 
  // Print "App Banner" and "Server Listening" notice.
  Serial.println(F(APP_BANNER_STRING));
  Serial.println(F(SERVER_LISTENING_STRING));
  Serial.print(F("\t\t"));
  Serial.print(ip);
  Serial.print(':');
  Serial.println(TELNET_SERVER_PORT);
  Serial.println(F("\r\n\r\n"));
 
}
 
void loop()
{
 
  /* add main program code here */
 
  // check for any new clients
  server_check_new_connections();
 
  // give up a timeslice
  yield();
 
  // check to see if any of the clients 
  // have recv data pending
  if (ClientList.size() > 0)
    server_check_client_recv();
 
  // give up a timeslice
  yield();
 
  // check to see if console has any 
  // pending recv data.
  if (Serial.available())
    console_recv();
 
  // give up a timeslice
  yield();
}
 
/*********************************************************/
 
size_t console_recv(void)
{
  size_t ret = 0;
  int c = 0;
  while ((c = Serial.read()) >= 0)
  {
    if (c == 10)
    {
      proc_line(con_buffer);
      con_buffer = String("");
      con_buffer.begin();
      continue;
    }
    if (c != 13)
    {
      con_buffer += (char)c;
      ret++;
    }
  }
  return ret;
}
 
void proc_line(StringStream &ss)
{
 
  //  Console input should be in the for of
  //  "ID:string_literal"
  //  where: 
  //    ID is client index to send to or "*" (all)
  //    string_literal is string to send
  //
  // check to ensure 2nd char is a ":"
  if ((ss.length() < 3) || (ss.charAt(1) != ':'))
  {
    Serial.print(F("Invalid Console Input \""));
    Serial.print(ss);
    Serial.print(F("\"\r\nPlease use the following format:\r\n\t"));
    Serial.println(F("\"ID:string_literal\"\r\n"));
    Serial.flush();
    return;
  }
  int idx = -1;
  Serial.print(F(" first char is '"));
  Serial.print(ss.charAt(0));
  Serial.print(F("',  "));
  if (ss.charAt(0) != '*')
  {
    idx = ss.parseInt();
  }
  Serial.print(F("idx="));
  Serial.print(idx);
  Serial.print(F(", indexOf(':')="));
  Serial.println(ss.indexOf(':'));
  ss = ss.substring(1 + ss.indexOf(':'));
  ss += F("\r\n");
  if (idx < 0)
    ClientList.send2All(ss);
    //send_to_all(ss);
  else
    ClientList.send(ss, idx);
}
 
size_t console_print_connected_count(void)
{
  size_t ret =
    Serial.print(F("\t\t")) +
    Serial.print(ClientList.size()) +
    Serial.println(F(" connected client(s).\r\n"));
  Serial.flush();
  return ret;
}
 
bool test_for_cmd(const String &itm, const String &src)
{
  String tst(src);
  tst.toLowerCase();
  return tst.startsWith(itm);
}
 
void server_check_new_connections(void)
{
  EthernetClient obj = server.available();  
  if (obj)
  {
    // convert to EthernetClientEx
    EthernetClientEx new_client(obj);
    // is it a new connection?
    if (!ClientList.exists(new_client))
    {
      // accept new connection
      server_accept((EthernetClientEx *)&new_client);
 
      // send welcome banner
      new_client.println(F(WELCOME_BANNER_STRING));
      new_client.flush();
    }
  }
}
 
int server_accept(EthernetClientEx *enetcli)
{
  // add itm to list
  ClientItem *itm = new ClientItem();
  itm->timestamp = millis();
  itm->index = ClientList.size();
  itm->client = new EthernetClientEx((*enetcli));
  ClientList.add(itm);
 
  // print notice on console
  Serial.print(F("Accepted new connection ("));
  Serial.print(itm->index);
  Serial.print(F(") from "));
  Serial.print(enetcli->remoteIP());
  Serial.print(':');
  Serial.println(enetcli->remotePort());
  Serial.println();
  Serial.flush();
 
  // print connected count
  console_print_connected_count();
 
  // return items index
  return itm->index;
}
 
void server_check_client_recv(void)
{
  if (ClientList.size() == 0) return;
  ClientItem *itm;
  for (int idx = ClientList.size() - 1; idx >= 0; idx--)
  {
    itm = ClientList.get(idx);
    if (itm->client->connected()==0)
    {
      client_disconnected(itm);
      continue;
    }
    if (itm->client->available())
    {
      size_t rc = client_recv(itm);
      if (rc > 0)
      {
        // echo to Serial port
        Serial.print(F("Client #"));
        Serial.print(itm->index);
        Serial.print(F(" sent \""));
        Serial.print(itm->recv_buffer);
        Serial.println('\"');
        Serial.flush();
      }
      // rc will == 0 if nothing received
      // rc will == -1 if client entered "exit"
    }
    yield();
  }
}
 
size_t client_recv(ClientItem *cli)
{
  int c;
  EthernetClientEx *cIn = (EthernetClientEx *)cli->client;
  cli->recv_buffer = "\0";
  cli->recv_cnt = 0;
  while ((c = cIn->read()) >= 0)
  {
    cli->recv_buffer += (char)c;
    cli->recv_cnt++;
  }
  // check to see if they typed "exit"
  if (test_for_cmd(String(F("exit")), cli->recv_buffer))
  {
    client_disconnecting(cli);
    delay(1); // give up timeslice
    cli->client->stop();
    Serial.print(F("called client->stop() for #")); Serial.println(cli->index); Serial.flush();
    return -1;
  }
  return cli->recv_cnt;
}
 
void client_disconnecting(ClientItem *cli)
{
  Serial.print(F("Client #"));
  Serial.print(cli->index);
  Serial.println(F(" is disconnecting."));
  // send Goodbye Banner
  send_to(cli->index, String(F(GOODBYE_BANNER_STRING)));
  Serial.flush();
  delay(1);
  cli->client->stop();
}
 
void client_disconnected(ClientItem *cli)
{
  // print notice on console
  Serial.print(F("Client #"));
  Serial.print(cli->index);
  Serial.print(F(" was connected for "));
  Serial.print(cli->elapsed() / 1000);
  Serial.print(F(" seconds, and has disconnected."));
  Serial.flush();
 
  // remove from list of clients
  ClientList.drop(cli->index);
  (*cli) = (*(ClientItem *)(unsigned long)0);
  // print connected count
  console_print_connected_count();
 
  // this causes seg fault ??
  //delete cli;
}
 
size_t send_to(int idx, const String &str)
{
  return ClientList.send(str, idx);
}
 
size_t send_to_all(const String &str)
{
  return ClientList.send2All(str);
}

Bingo ? Sounds good?

Again, sorry my C++ knowledge is very limited...

HTH

[hek]

Looks good, but unfortunately I don't have the time to pursue this one at the moment.

[soif]

Happy if i could have helped you to save you some time to find a way for implementing a solution.

Let us know when you'd made some progress

[soif]

BTW would you mind if I opened an issue at GH about this, linking to this topic, to help you and others developpers to have a reminder at the right place.

And maybe another C++ hero would see it, and start working on it!

[hek]

Sure, do that.

[soif]

Issue #398 opened at GitHub.

Now seeking for a C++ hero !

HTH

[petewill]

@soif I'm no good at C++ so I can't help there

I was wondering if you have had your Ethernet gateway crash at all while connecting two controllers? I have been experimenting with Domoticz (my primary controller is Vera) and my Ethernet gateway crashes about every other day. I usually have to power cycle the gateway to get it to come back (sometimes the reset button works). I had previously run the MYSController for many days without issue (but that was when I was on 1.4.1). Just curious if you are getting the same results since you have a similar setup to me. What is the other controller you are trying to connect?

My setup is a little different from yours as I'm using Node-red on a Pi to connect to the Ethernet gateway (Domoticz thinks it's looking at a serial connection so I haven't noticed the disconnect messages like you).

[soif]

My GW don't crash while the second controller get a connection, it just throw away the previous connection.
I've tested this with MYS v1.51, 1.54 and 2.0beta.

My two controllers are :

• Domoticz (primary)
• a simple telenet connection from the terminal

You can check this by having one controller connected, then from a terminal console type :

  telnet IP_OF_THE_GW 5003

It will launch a telnet session an you will see the GW messages scrolling, but when you send any characters (ie just hit return), boom, the first connected controller is immediately disconnected.

[petewill]

@soif said:

My GW don't crash while the second controller get a connection, it just throw away the previous connection.

Interesting. My Vera and Node-red must constantly be disconnecting each other. This could be a great clue as to what's going on with my gateway crashing. The strange thing is the myscontroller worked so well before...