[SOLVED] ESP8266 MQTT Gateway: Can't Subscribe from Domoticz with Mosquitto running on Pi 3

[Neo Mod]

SOLUTION:

For anyone reading this topic, the solution is to use the MOSQUITTO (MQQT Broker) IP ADDRESS also for the MySensors Gateway.

I have read almost any post regarding this topic, but I'm afraid I wasn't able to find a solution.

I have an ESP8666 (NodeMCU) running MySensors MQTT Gateway with just one test sensor attached: everthig is working fine, I can monitor Sensor Data and MQTT messages in Serial Monitor.

Below the sketch I'm using:

#define MY_GATEWAY_MQTT_CLIENT
#define MY_GATEWAY_ESP8266

// Set this node's subscribe and publish topic prefix
#define MY_MQTT_PUBLISH_TOPIC_PREFIX "domoticz/in/mymqtt"
#define MY_MQTT_SUBSCRIBE_TOPIC_PREFIX "domoticz/out/mymqtt"

// Set MQTT client id
#define MY_MQTT_CLIENT_ID "prova1"

// Enable these if your MQTT broker requires username/password
//#define MY_MQTT_USER "username"
//#define MY_MQTT_PASSWORD "password"

// Set WIFI SSID and password
#define MY_WIFI_SSID "MYSSID"
#define MY_WIFI_PASSWORD "MYPASS"

// Set the hostname for the WiFi Client. This is the hostname
// it will pass to the DHCP server if not static.
 #define MY_HOSTNAME "mqtt-sensor-gateway"

// Enable MY_IP_ADDRESS here if you want a static ip address (no DHCP)
//#define MY_IP_ADDRESS 192,168,1,46

// If using static ip you can define Gateway and Subnet address as well
//#define MY_IP_GATEWAY_ADDRESS 192,168,178,1
//#define MY_IP_SUBNET_ADDRESS 255,255,255,0

// MQTT broker ip address.
#define MY_CONTROLLER_IP_ADDRESS 192, 168, 1, 39

//MQTT broker if using URL instead of ip address.
// #define MY_CONTROLLER_URL_ADDRESS "test.mosquitto.org"

// The MQTT broker port to to open
#define MY_PORT 1883

// Enable inclusion mode
//#define MY_INCLUSION_MODE_FEATURE
// Enable Inclusion mode button on gateway
//#define MY_INCLUSION_BUTTON_FEATURE
// Set inclusion mode duration (in seconds)
//#define MY_INCLUSION_MODE_DURATION 60
// Digital pin used for inclusion mode button
//#define MY_INCLUSION_MODE_BUTTON_PIN D1

// Set blinking period
//#define MY_DEFAULT_LED_BLINK_PERIOD 300

// Flash leds on rx/tx/err
//#define MY_DEFAULT_ERR_LED_PIN 16  // Error led pin
//#define MY_DEFAULT_RX_LED_PIN  16  // Receive led pin
//#define MY_DEFAULT_TX_LED_PIN  16  // the PCB, on board LED


#include <ESP8266WiFi.h>
#include <MySensors.h>

#include <SPI.h>
#include <Wire.h>

// BME280 libraries and variables
// Bosch BME280 Embedded Adventures MOD-1022 weather multi-sensor Arduino code
// Written originally by Embedded Adventures
// https://github.com/embeddedadventures/BME280
#include <BME280_MOD-1022.h>

#define BARO_CHILD 0
#define TEMP_CHILD 1
#define HUM_CHILD 2

const float ALTITUDE = 124; // <-- adapt this value to your location's altitude (in m). Use your smartphone GPS to get an accurate value!

// Sleep time between reads (in ms). Do not change this value as the forecast algorithm needs a sample every minute.
const unsigned long SLEEP_TIME = 60000; 

const char *weather[] = { "stable", "sunny", "cloudy", "unstable", "thunderstorm", "unknown" };
enum FORECAST
{
  STABLE = 0,     // "Stable Weather Pattern"
  SUNNY = 1,      // "Slowly rising Good Weather", "Clear/Sunny "
  CLOUDY = 2,     // "Slowly falling L-Pressure ", "Cloudy/Rain "
  UNSTABLE = 3,   // "Quickly rising H-Press",     "Not Stable"
  THUNDERSTORM = 4, // "Quickly falling L-Press",    "Thunderstorm"
  UNKNOWN = 5     // "Unknown (More Time needed)
};

float lastPressure = -1;
float lastTemp = -1;
float lastHum = -1;
int lastForecast = -1;

const int LAST_SAMPLES_COUNT = 5;
float lastPressureSamples[LAST_SAMPLES_COUNT];


// this CONVERSION_FACTOR is used to convert from Pa to kPa in the forecast algorithm
// get kPa/h by dividing hPa by 10 
#define CONVERSION_FACTOR (1.0/10.0)

int minuteCount = 0;
bool firstRound = true;
// average value is used in forecast algorithm.
float pressureAvg;
// average after 2 hours is used as reference value for the next iteration.
float pressureAvg2;

float dP_dt;
boolean metric;
MyMessage tempMsg(TEMP_CHILD, V_TEMP);
MyMessage humMsg(HUM_CHILD, V_HUM);
MyMessage pressureMsg(BARO_CHILD, V_PRESSURE);
MyMessage forecastMsg(BARO_CHILD, V_FORECAST);


float getLastPressureSamplesAverage()
{
  float lastPressureSamplesAverage = 0;
  for (int i = 0; i < LAST_SAMPLES_COUNT; i++)
  {
    lastPressureSamplesAverage += lastPressureSamples[i];
  }
  lastPressureSamplesAverage /= LAST_SAMPLES_COUNT;

  return lastPressureSamplesAverage;
}


// Algorithm found here
// http://www.freescale.com/files/sensors/doc/app_note/AN3914.pdf
// Pressure in hPa -->  forecast done by calculating kPa/h
int sample(float pressure)
{
  // Calculate the average of the last n minutes.
  int index = minuteCount % LAST_SAMPLES_COUNT;
  lastPressureSamples[index] = pressure;

  minuteCount++;
  if (minuteCount > 185)
  {
    minuteCount = 6;
  }

  if (minuteCount == 5)
  {
    pressureAvg = getLastPressureSamplesAverage();
  }
  else if (minuteCount == 35)
  {
    float lastPressureAvg = getLastPressureSamplesAverage();
    float change = (lastPressureAvg - pressureAvg) * CONVERSION_FACTOR;
    if (firstRound) // first time initial 3 hour
    {
      dP_dt = change * 2; // note this is for t = 0.5hour
    }
    else
    {
      dP_dt = change / 1.5; // divide by 1.5 as this is the difference in time from 0 value.
    }
  }
  else if (minuteCount == 65)
  {
    float lastPressureAvg = getLastPressureSamplesAverage();
    float change = (lastPressureAvg - pressureAvg) * CONVERSION_FACTOR;
    if (firstRound) //first time initial 3 hour
    {
      dP_dt = change; //note this is for t = 1 hour
    }
    else
    {
      dP_dt = change / 2; //divide by 2 as this is the difference in time from 0 value
    }
  }
  else if (minuteCount == 95)
  {
    float lastPressureAvg = getLastPressureSamplesAverage();
    float change = (lastPressureAvg - pressureAvg) * CONVERSION_FACTOR;
    if (firstRound) // first time initial 3 hour
    {
      dP_dt = change / 1.5; // note this is for t = 1.5 hour
    }
    else
    {
      dP_dt = change / 2.5; // divide by 2.5 as this is the difference in time from 0 value
    }
  }
  else if (minuteCount == 125)
  {
    float lastPressureAvg = getLastPressureSamplesAverage();
    pressureAvg2 = lastPressureAvg; // store for later use.
    float change = (lastPressureAvg - pressureAvg) * CONVERSION_FACTOR;
    if (firstRound) // first time initial 3 hour
    {
      dP_dt = change / 2; // note this is for t = 2 hour
    }
    else
    {
      dP_dt = change / 3; // divide by 3 as this is the difference in time from 0 value
    }
  }
  else if (minuteCount == 155)
  {
    float lastPressureAvg = getLastPressureSamplesAverage();
    float change = (lastPressureAvg - pressureAvg) * CONVERSION_FACTOR;
    if (firstRound) // first time initial 3 hour
    {
      dP_dt = change / 2.5; // note this is for t = 2.5 hour
    }
    else
    {
      dP_dt = change / 3.5; // divide by 3.5 as this is the difference in time from 0 value
    }
  }
  else if (minuteCount == 185)
  {
    float lastPressureAvg = getLastPressureSamplesAverage();
    float change = (lastPressureAvg - pressureAvg) * CONVERSION_FACTOR;
    if (firstRound) // first time initial 3 hour
    {
      dP_dt = change / 3; // note this is for t = 3 hour
    }
    else
    {
      dP_dt = change / 4; // divide by 4 as this is the difference in time from 0 value
    }
    pressureAvg = pressureAvg2; // Equating the pressure at 0 to the pressure at 2 hour after 3 hours have past.
    firstRound = false; // flag to let you know that this is on the past 3 hour mark. Initialized to 0 outside main loop.
  }

  int forecast = UNKNOWN;
  if (minuteCount < 35 && firstRound) //if time is less than 35 min on the first 3 hour interval.
  {
    forecast = UNKNOWN;
  }
  else if (dP_dt < (-0.25))
  {
    forecast = THUNDERSTORM;
  }
  else if (dP_dt > 0.25)
  {
    forecast = UNSTABLE;
  }
  else if ((dP_dt > (-0.25)) && (dP_dt < (-0.05)))
  {
    forecast = CLOUDY;
  }
  else if ((dP_dt > 0.05) && (dP_dt < 0.25))
  {
    forecast = SUNNY;
  }
  else if ((dP_dt >(-0.05)) && (dP_dt < 0.05))
  {
    forecast = STABLE;
  }
  else
  {
    forecast = UNKNOWN;
  }

  // uncomment when debugging
  //Serial.print(F("Forecast at minute "));
  //Serial.print(minuteCount);
  //Serial.print(F(" dP/dt = "));
  //Serial.print(dP_dt);
  //Serial.print(F("kPa/h --> "));
  //Serial.println(weather[forecast]);

  return forecast;
}


void setup() {
  metric = getControllerConfig().isMetric;  // was getConfig().isMetric; before MySensors v2.1.1
  Wire.begin(0x76); // Wire.begin(sda, scl)
}

void presentation()  {
  // Send the sketch version information to the gateway and Controller
  sendSketchInfo("BME280 Sensor", "1.6");

  // Register sensors to gw (they will be created as child devices)
  present(BARO_CHILD, S_BARO);
  present(TEMP_CHILD, S_TEMP);
  present(HUM_CHILD, S_HUM);
}

// Loop
void loop() {
  
  // need to read the NVM compensation parameters
  BME280.readCompensationParams();

  /* After taking the measurement the chip goes back to sleep, use when battery powered.
  // Oversampling settings (os1x, os2x, os4x, os8x or os16x).
  BME280.writeFilterCoefficient(fc_16);       // IIR Filter coefficient, higher numbers avoid sudden changes to be accounted for (such as slamming a door)
  BME280.writeOversamplingPressure(os16x);    // pressure x16
  BME280.writeOversamplingTemperature(os8x);  // temperature x8
  BME280.writeOversamplingHumidity(os8x);     // humidity x8

  BME280.writeMode(smForced);                 // Forced sample.  After taking the measurement the chip goes back to sleep
  */

  // Normal mode for regular automatic samples
  BME280.writeStandbyTime(tsb_0p5ms);         // tsb = 0.5ms
  BME280.writeFilterCoefficient(fc_16);       // IIR Filter coefficient 16
  BME280.writeOversamplingPressure(os16x);    // pressure x16
  BME280.writeOversamplingTemperature(os8x);  // temperature x8
  BME280.writeOversamplingHumidity(os8x);     // humidity x8
  
  BME280.writeMode(smNormal);
  
  while (1) {
    // Just to be sure, wait until sensor is done mesuring  
    while (BME280.isMeasuring()) {
  }
  
  // Read out the data - must do this before calling the getxxxxx routines
  BME280.readMeasurements();

  float temperature = BME280.getTemperatureMostAccurate();                    // must get temp first
  float humidity = BME280.getHumidityMostAccurate();
  float pressure_local = BME280.getPressureMostAccurate();                    // Get pressure at current location
  float pressure = pressure_local/pow((1.0 - ( ALTITUDE / 44330.0 )), 5.255); // Adjust to sea level pressure using user altitude
  int forecast = sample(pressure);
  
  if (!metric) 
  {
    // Convert to fahrenheit
    temperature = temperature * 9.0 / 5.0 + 32.0;
  }

  Serial.println();
  Serial.print("Temperature = ");
  Serial.print(temperature);
  Serial.println(metric ? " °C" : " °F");
  Serial.print("Humidity = ");
  Serial.print(humidity);
  Serial.println(" %");
  Serial.print("Pressure = ");
  Serial.print(pressure);
  Serial.println(" hPa");
  Serial.print("Forecast = ");
  Serial.println(weather[forecast]);
  Serial.println();


  if (temperature != lastTemp) 
  {
    send(tempMsg.set(temperature, 1));
    lastTemp = temperature;
  }


  if (humidity != lastHum) 
  {
    send(humMsg.set(humidity, 1));
    lastHum = humidity;
  }

  if (pressure != lastPressure) 
  {
    send(pressureMsg.set(pressure, 2));
    lastPressure = pressure;
  }

  if (forecast != lastForecast)
  {
    send(forecastMsg.set(weather[forecast]));
    lastForecast = forecast;
  }
  
  sleep(SLEEP_TIME);
  
}
}

And my serial output is:

Temperature = 27.80 ⸮C
Humidity = 57.31 %
Pressure = 1014.62 hPa
Forecast = stable

817497 GWT:TPS:TOPIC=domoticz/in/mymqtt/0/1/1/0/0,MSG SENT
817653 GWT:TPS:TOPIC=domoticz/in/mymqtt/0/2/1/0/1,MSG SENT
817715 GWT:TPS:TOPIC=domoticz/in/mymqtt/0/0/1/0/4,MSG SENT
817778 MCO:SLP:MS=60000,SMS=0,I1=255,M1=255,I2=255,M2=255
817838 MCO:SLP:WUP=-2

But when I try to subscribe to MySensors MQTT Gateway from Domoticz, I can't obtain a connection: every time I get the error "Error: MQTT: Failed to start, return code: 14 (Check IP/Port)"

In Domoticz I've added a "MySensors Gateway with MQTT Interface" Hardware, with IP = IP of NodeMCU/MySensors and Port = 1883 (standard one).

I really hope someone can help me figure out this "noob error". Thank you very much in advance for your time.

[Neo Mod]

I've just tried with a fresh install of domoticz: the MySensors MQTT Gateway is still unavailable.

2018-08-02 22:57:26.414 Error: MQTT: Failed to start, return code: 14 (Check IP/Port)

EDIT:

Running an "empty" sketch for MQTT Gateway seems to be working correctly: I'm wondering if this is a problem related to Domoticz.

28193 GWT:TPC:IP=192.168.1.44
28225 GWT:RMQ:MQTT RECONNECT
28292 GWT:RMQ:MQTT CONNECTED
28323 GWT:TPS:TOPIC=domoticz/out/MyMQTT/0/255/0/0/17,MSG SENT

EDIT 2:

I verified that MySensors MQTT is reporting correctly to Mosquitto Broker running on Pi 3. All values are reported correctly.

pi@raspberrypi:~ $ mosquitto_sub -h localhost -t "#" -v
domoticz/in/MyMQTT/0/1/1/0/0 29.3
domoticz/in/MyMQTT/0/2/1/0/1 59.9
domoticz/in/MyMQTT/0/0/1/0/4 1014.09

This is starting to feel more and more like a Domoticz "weird" problem.

[mfalkvidd]

@neo-mod a MQTT client (MY_GATEWAY_MQTT_CLIENT) connects to a mqtt broker. So your esp8266, which is configured as client, will connect to the mqtt broker at MY_CONTROLLER_IP_ADDRESS.

A client will not accept incoming connections. That's why Domoticz can't connect to the esp8266.

I've never used mqtt with MySensors so I don't know how to configure it with Domoticz infortunately, but maybe someone else can help.

[mfalkvidd]

Just a guess: maybe all you need is to use the rpi3 ip instead of the esp8266 ip?

[Neo Mod]

Thank you so much! You just saved me (a certified n00b, I know xD) from countless hours of headache!

@mfalkvidd said in ESP8266 MQTT Gateway: Can't Subscribe from Domoticz with Mosquitto running on Pi 3:

Just a guess: maybe all you need is to use the rpi3 ip instead of the esp8266 ip?