Arduino Mega + W5500 + BME280 = Connection refused all the time

[NeoX]

Hi All!

I'm quite new MySensors user, and i really enjoy this framework so far
I've tested it with "serial gateway" with BME280 and it worked just great so far. I don't use radio, only local gateway sensors.
Now things goes more complicated when i tried to use Ethernet Gateway. I've connected my W5500 module, and wrote this simple sketch (A little bit modified BME280 example):

#undef MY_RADIO_RF24
#undef MY_OTA_USE_I2C_EEPROM
//#define MY_GATEWAY_SERIAL
#define MY_GATEWAY_W5100
#define MY_IP_ADDRESS 192,168,8,35
#define MY_PORT 5003
//#define MY_MAC_ADDRESS 0x00, 0xAA, 0xBB, 0xCC, 0xDE, 0x02
#define MY_W5100_SPI_EN 10
#define MY_DEBUG_VERBOSE_GATEWAY

#include <Ethernet.h>


// LIBRARIES
#include <SPI.h>                                  // A communication backbone, the Serial Peripheral Interface.
#include <MySensors.h>                            // The MySensors library. Hurray!
#include <Wire.h>                                 // Enables the Wire communication protocol.
//#include <Adafruit_BME280.h>                    // alternative library you could try (DIY; no code for this is in here yet).
//#include <SparkFunBME280.h>                     // alternative library you could try (DIY; no code for this is in here yet).
#include <BME280_MOD-1022.h>                      // Bosch BME280 Embedded Adventures MOD-1022 weather multi-sensor Arduino code, written originally by Embedded Adventures. https://github.com/embeddedadventures/BME280


// VARIABLES YOU CAN CHANGE
const float ALTITUDE = 14;                        // Change this value to your location's altitude (in m). Use your smartphone GPS to get an accurate value, or use an online map.
unsigned long BME280measurementInterval = 60000;  // Sleep time between reads for the BME sensor (in ms). Keep this value at 60000 if you have enabled the forecast feature, as the forecast algorithm needs a sample every minute.
#define COMPARE_TEMP 1                            // Send temperature only if it changed? 1 = Yes 0 = No. Can save battery.
float tempThreshold = 0.1;                        // How big a temperature difference has to minimally  be before an update is sent. Makes the sensor less precise, but also less jittery, and can save battery.
#define COMPARE_HUM 1                             // Send humidity only if changed? 1 = Yes 0 = No. Can save battery.
float humThreshold = 0.1;                         // How big a humidity difference has to minimally be before an update is sent. Makes the sensor less precise, but also less jittery, and can save battery.
#define COMPARE_BARO 1                            // Send barometric pressure only if changed? 1 = Yes 0 = No. Can save battery.
float presThreshold = 0.1;                        // How big a barometric difference has to minimally be before an update is sent. Makes the sensor less precise, but also less jittery, and can save battery.
            

// VARIABLES YOU PROBABLY SHOULDN'T CHANGE
#define TEMP_CHILD_ID 0        // for MySensors. Within this node each sensortype should have its own ID number.
#define HUM_CHILD_ID 1        // for MySensors. Within this node each sensortype should have its own ID number.
#define BARO_CHILD_ID 2       // for MySensors. Within this node each sensortype should have its own ID number.
float lastTemperature = -1;     // Stores the previous measurement, so it can be compared with a new measurement.
float lastHumidity = -1;      // Stores the previous measurement, so it can be compared with a new measurement.
float lastPressure = -1;      // Stores the previous measurement, so it can be compared with a new measurement.
unsigned long BME280measurementSleepTime = 0;   // variable to store the calculated Sleep time if the node is battery powered.
bool receivedConfig = false;      // The MySensors gateway will tell the node if it should output in metric or not.

#define CONVERSION_FACTOR (1.0/10.0)    // used by forecast algorithm to convert from Pa to kPa, by dividing hPa by 10.

// MYSENSORS COMMUNICATION VARIABLES
MyMessage temperatureMsg(TEMP_CHILD_ID, V_TEMP);
MyMessage humidityMsg(HUM_CHILD_ID, V_HUM);
MyMessage pressureMsg(BARO_CHILD_ID, V_PRESSURE);


void setup() {
  Wire.begin(); // Wire.begin(sda, scl) // starts the wire communication protocol, used to chat with the BME280 sensor.
  Serial.begin(115200); // for serial debugging over USB.
  Serial.println("Hello world, I am a sensor node.");

}


void presentation()  {
  // Send the sketch version information to the gateway and Controller
  sendSketchInfo("BME280 Sensor", "1.1");

  // Tell the MySensors gateway what kind of sensors this node has, and what their ID's on the node are, as defined in the code above.
  present(BARO_CHILD_ID, S_BARO);
  present(TEMP_CHILD_ID, S_TEMP);
  present(HUM_CHILD_ID, S_HUM);
}


void loop() {

  // You should not change these variables:
  static unsigned long previousBME280Millis = 0;  // Used to remember the time that the BME280 sensor was asked for a measurement.
  unsigned long currentMillis = millis();         // The time since the sensor started, counted in milliseconds. This script tries to avoid using the Sleep function, so that it could at the same time be a MySensors repeater.
  static boolean BME280shouldAsk = true;          // This is true when the time is right for a new measurement to be made.
  static boolean BME280justAsked = false;         // This indicates whether we have just asked the sensor module for a measurement, so the receiving part of the code (part 2) should be primed. This two-part construction helps to bridge the time where the BME280 module is busy, without blocking the entire node from doing anything else (like being a repeater, or working with other connected sensor modules).


  // PART 1. If enough time has passed, a new measurement should be taken:
  if (BME280shouldAsk == true && currentMillis - previousBME280Millis >= BME280measurementInterval) {
    previousBME280Millis = currentMillis; // store the current time as the previous measurement start time.
    BME280shouldAsk = false;
    Serial.println("");
    Serial.println("BME280 - Requesting new data from sensor module.");
    BME280.readCompensationParams();    // Need to read the NVM compensation parameters.

    // Normal mode for regular automatic samples
    BME280.writeStandbyTime(tsb_0p5ms);         // tsb = 0.5ms
    BME280.writeFilterCoefficient(fc_16);       // IIR Filter coefficient 16
    BME280.writeOversamplingPressure(os16x);    // pressure x16
    BME280.writeOversamplingTemperature(os8x);  // temperature x8
    BME280.writeOversamplingHumidity(os8x);     // humidity x8
    BME280.writeMode(smNormal);
  
    // As we exit part 1, in theory BME280.isMeasuring() should now be true.
    BME280justAsked = true;   
  }


  // Part 2. This will trigger if the sensor has just been asked for a measurement, and is also just done figuring out those measurements.
  if(BME280justAsked == true && BME280.isMeasuring() == false) { // 
    BME280justAsked = false; // makes sure we don't do this part again in the next pass through the main loop.
    Serial.println("BME280 - Sensor module has some new values ready:");
      
    // Read out the data - must do this before calling the getxxxxx routines
    BME280.readMeasurements();
  
    float temperature = BME280.getTemperatureMostAccurate();                    // Must get the temperature first.
    float humidity = BME280.getHumidityMostAccurate();        // Get the humidity.
    float pressure_local = BME280.getPressureMostAccurate();                    // Get pressure at current location
    float pressure = pressure_local/pow((1.0 - ( ALTITUDE / 44330.0 )), 5.255); // Adjust to sea level pressure using user altitude

    // Useful for debugging
    Serial.print("BME280 - Temperature = ");
    Serial.print(temperature);
    Serial.println(" °C");
    Serial.print("BME280 - Humidity = ");
    Serial.print(humidity);
    Serial.println(" %");
    Serial.print("BME280 - Pressure = ");
    Serial.print(pressure);
    Serial.println(" hPa");

    // Now, let's send the measurements to the gateway.

    // Send temperature
    if (COMPARE_TEMP == 1 && abs(temperature - lastTemperature) < tempThreshold) { // is the temperature difference bigger than the threshold?
      Serial.print(temperature - lastTemperature);
      Serial.print("- BME280 - Temperature difference too small, so not sending the new measurement to the gateway.\n");
    } else {
      Serial.print("BME280 - Sending the new temperature to the gateway.\n");
      send(temperatureMsg.set(temperature, 1));
      lastTemperature = temperature; // Save new temperatures to be able to compare in the next round.
    } 

    // Send humidity
    if (COMPARE_HUM == 1 && abs(humidity - lastHumidity) < humThreshold) { // is the humidity difference bigger than the threshold?
      Serial.print(humidity - lastHumidity);
      Serial.println("- BME280 - Humidity difference too small, so not sending the new measurement to the gateway.");
    } else {
      Serial.println("BME280 - Sending the new humidity to the gateway.");
      send(humidityMsg.set(humidity, 1));
      lastHumidity = humidity; // Save new humidity to be able to compare in the next round.
    }

    // Send pressure
    if (COMPARE_BARO == 1 && abs(pressure - lastPressure) < presThreshold) { // is the pressure difference bigger than the threshold?
      Serial.print(pressure - lastPressure);
      Serial.println("- BME280 - Pressure difference too small, so not sending the new measurement to the gateway.");
    } else {
      Serial.println("BME280 - Sending the new pressure to the gateway.");
      send(pressureMsg.set(pressure, 1));
      lastPressure = pressure; // Save new pressure to be able to compare in the next round.
    }

    Serial.println("BME280 - Measurement complete. Going to wait until next measurement.");
    BME280shouldAsk = true; // Ready for the new round.
  }

} // end of main loop.

Now device goes IP address, and i can ping it, but i cant open port 5003 for communication:

0 MCO:BGN:INIT GW,CP=R-NGA---,FQ=16,REL=255,VER=2.3.2
565 GWT:TIN:IP=192.168.8.35
1567 MCO:BGN:STP
Hello world, I am a sensor node.
1569 MCO:REG:NOT NEEDED
1574 MCO:BGN:INIT OK,TSP=NA

BME280 - Requesting new data from sensor module.
BME280 - Sensor module has some new values ready:
BME280 - Temperature = 25.90 °C
BME280 - Humidity = 63.51 %
BME280 - Pressure = 996.91 hPa
BME280 - Sending the new temperature to the gateway.
BME280 - Sending the new humidity to the gateway.
BME280 - Sending the new pressure to the gateway.
BME280 - Measurement complete. Going to wait until next measurement.

And board doesn't get frozen because i get measurements every 10 seconds.

I'm using latest libs (Ethernet, MySensors, BME) from latest Arduino. My board (arduino and w5500) is powered from external power supply, with a lot power. Moreover, i 've uploaded a simple arduino http example sketch, and it worked great, so i don't think it's a hardware problem.

But what's interesting, Home Assistant sometimes can reach this board. Very rarely, just after board reset.

Do you know what's going on here?

Thanks,
Michał

[mfalkvidd]

Welcome to the community @NeoX Great to hear the library is useful for you.

Could it be that you have another device with same IP or MAC address on the network?

[NeoX]

@mfalkvidd Unfortunately, no i'm 100% sure that IP and MAC are unique.
I've also added MY_GATEWAY_ADDRESS and MY_IP_SUBNET_ADDRESS, no luck

[TRS-80]

Well, I am only part way into my first cup of coffee, so maybe my brain just isn't working yet. But so far I am failing to understand what all this shouldAsk and justAsked business is all about? Why not just do a simple check on millis() and if enough time has passed, enter the measurement function? Something like:

void loop() {
  if ( millis() > previousBME280Millis + BME280measurementInterval ) {
    // take measurements, etc...
  }

The way it is presented in OP is, IMO, much more complicated than necessary. Or maybe someone smarter can point out what I am missing?

Your BME280measurementInterval = 60000 (60 seconds), so if you:

@NeoX said in Arduino Mega + W5500 + BME280 = Connection refused all the time:

get measurements every 10 seconds

... then it makes me suspect that something is wrong with the main loop there. And if the loop is hogging the CPU, maybe the Ethernet interface can't get a word in edgewise?

But I couldn't seem to figure out the loop, with everything you've got going on there. Maybe I try again in a little while when I have had some coffee and am all the way "on."

[NeoX]

@TRS-80 I've pasted old version, i;ve modified BME280measurementInterval = 10000
But it doesnt matter, to be honest.
Now ive completely commented out loop() body, and it's still same problem

Maybe it's some kind of problem with library version?
I found that only compatibile Ethernet library for MySensors is 2.0.0. so i cant downgrade it, but maybe MySensors has some bug here?

[TRS-80]

@NeoX said in Arduino Mega + W5500 + BME280 = Connection refused all the time:

Maybe it's some kind of problem with library version?
I found that only compatibile Ethernet library for MySensors is 2.0.0. so i cant downgrade it, but maybe MySensors has some bug here?

I just put together Ethernet based gateway (with MQTT, but that shouldn't matter) based on W5500 and Arduino Nano not too long ago. I used platform.io, which (I just double checked my local files) apparently pulled in same v2.0.0 of Ethernet library. And mine works absolutely flawless for a few weeks now.

How about we back up a little and go step by step. Try just the plain (MySensors) Ethernet gateway code and see how that works?

[NeoX]

@TRS-80 Well what can i say, it works. I can ping, just after a reboot i can connect to 5003 via telnet.
When i close connection (by ctrl+] and close), then i cannot connect to it again

[TRS-80]

@NeoX said in Arduino Mega + W5500 + BME280 = Connection refused all the time:

Well what can i say, it works. I can ping, just after a reboot i can connect to 5003 via telnet.
When i close connection (by ctrl+] and close), then i cannot connect to it again

Was this with the plain (MySensors) Ethernet code? Or your code from OP with the loop commented out?

[NeoX]

@TRS-80 No, with plain code unfortunately.

[TRS-80]

OK, so it appears the Ethernet part works? I am still not sure what this is about though?:

@NeoX said in Arduino Mega + W5500 + BME280 = Connection refused all the time:

When i close connection (by ctrl+] and close), then i cannot connect to it again

Other than this (potential?) problem, if Ethernet works, brings me back to what I speculated above about the main loop running too much...

[NeoX]

@TRS-80 But it's not possible, or some bug in Mysensors.
Because in plain example the loop in sketch is empty.
Do you use lib version 2.3.3 of MySensors lib?

[TRS-80]

I just checked my local files for my gateway, and looks like MySensors is v2.3.2.

I am inclined to think the problem is somewhere else, before assuming a bug in MySensors, however it is possible (although unlikely IMO) that potentially could be the problem. One easy way to find out, anyway...

[Yveaux]

@NeoX mysensors has an empty main loop() by default. The stack runs in the 'background', after each loop iteration, so your loop() should not block!

[NeoX]

@TRS-80 Do you have any idea whats more i can check?
Because i have no idea what could be wrong when i use plain sketch

[TRS-80]

Didn't you say plain sketch was working? Other than only being able to connect only once via telnet?

To be honest I am not familiar with using telnet for testing this sort of thing, so I installed it here on my Debian development machine to try and replicate what you are doing, but so far no luck. Probably because first time with tool and simply haven't figured it out yet. But this is what I get so far (probably doing something wrong):

┌─[ 2020-06-26 09:33 trs80@host:~ ]
└─▶ $ telnet 192.168.x.y 1883

Trying 192.168.x.y...
telnet: Unable to connect to remote host: Connection refused

┌─[ 2020-06-26 09:33 trs80@host:~ ]
└─▶ $ telnet 192.168.x.y:1883

telnet: could not resolve 192.168.x.y:1883/telnet: Name or service not known

Note I am running an MQTT gateway, therefore I set port 1883 in my sketch, furthermore I am running in MY_GATEWAY_MQTT_CLIENT mode (I think I read server mode is default in plain (non-MQTT) Ethernet sketch? not sure). So that could be part of my problem here, too.

[NeoX]

@TRS-80 The plain sketch is working in Server mode, so it is possible to connect to port 5003.
Since i don't know any tools to test MySensors ethernet, i've used telnet

My plain Ethernet example sketch is working, but only for one time after reset, which is weird. Just like some resources hasn't been released or W5500 didnt realised that connection was closed after ctrl+] ?
No idea...
I will take a look at MQTT, since it is working in Client mode...

[Yveaux]

@NeoX what bme library are you using?

[NeoX]

@Yveaux Adafruit BME280 in version 2.0.0, as it said in Arduino IDE

[NeoX]

Well, i tested MQTT and it worked just fine for a few minutes. Any problems.
I belive that there is some kind of bug in TCP server mode or maybe W5500 driver?
I can also say that when i close telnet connection via ctrl+] i can see that my PC is still waiting for close ack ( connection state in netstat is WAIT_FIN_2).

Don't know, but i switched to MQTT and it works fine, and i think MQTT client is more important for me.

[Yveaux]

@NeoX Maybe not relevant, but that doesn't appear to be the Adafruit BME280, as you're including <bme280_mod-1022.h>. This headerfile appears to be part of this library: https://github.com/embeddedadventures/BME280 which is very different from the Adafruit implementation.
If that library blocks too long in one of its calls it might cause troubles.

[NeoX]

@Yveaux Hmm, interesting, since only one library i do find in Arduino IDE library manager as installed was this Adafruit.
But's should be fine

[TRS-80]

@NeoX said in Arduino Mega + W5500 + BME280 = Connection refused all the time:

only one library i do find in Arduino IDE library manager

I started using PlatformIO recently, and I much prefer their way of searching libraries. When you do pio run it will even take the library.h file name that you are missing and give you a link to look it up in their library search. You can also search there for library name, or file names. Very handy!

Anyway, a bit offtopic for this thread, so recently I started another one where we can discuss PlatformIO (if you are interested).

@NeoX said in Arduino Mega + W5500 + BME280 = Connection refused all the time:

i switched to MQTT and it works fine, and i think MQTT client is more important for me

I rebuilt my gateway recently (few weeks ago) and this time I decided to go MQTT way. I actually have MQTT broker running on separate machine from my HA controller. Sort of distributed architecture, if you will. So far I am really liking this approach. Soon I will add some additional monitoring and metrics based directly off the MQTT, these will also be totally independent of the controller.