ESP8266 GW+Node (MQTT Transport): Won't Present Sketch or Actuator To Controller

[non_avg_joe]

Hi There Experts :)

If You Will, I'd Love Some Assistance w/ Getting My Capstone Project Ready :

My Software Setup is as Following -

Node-RED (Flow(s)+Dashboard) <-> Mosquitto MQTT-Broker (v1.4.10) <-> {ESP8266 Arduino-port (IDE : v1.6.12 ; ESP8266 port : latest github 'master'-branch pull) + MySensors Library (v2.1.0-beta - latest github 'master'-branch pull)}

As Per My Program ('Sketch'), I've Took The 'GatewayESP8266MQTTClient' Example & Elaborated On It a Bit - Merging In Mostly The 'DimmableLEDActuator' Example,
Along a mDNS+DNS-SD Based Name-Resolver - For The MQTT-Broker Hostname (Via The Library That's Included With The ESP8266 Arduino-port).

However, While My ESP8266 GW+Node Boots, Successfully Connects to Both WiFi Net. & The MQTT-Broker,
& Responds - for e.g. : To MySensors Lib. 'set' Commands (Sent Via MQTT) - Just As Expected (Including Sending Back an 'ACK' - If Reqested),

It Just Won't Present The Sketch Name+Version - OR - The Dimmable-LED-Actuator (Via MQTT),
Neither It'll Request a Node_Id (Via MQTT).

Sketch :

#include <EEPROM.h>
#include <Wire.h>
#include <SPI.h>

#include <ESP8266WiFi.h>
#include <ESP8266mDNS.h>

// a Dynamic Configuration struct
//struct __attribute__((packed)) My_Config_t
struct My_Config_t
{
String WLAN_STA_SSID = "";
String WLAN_STA_PASSWD = "";
String MQTT_Broker_FQDN = "";
IPAddress MQTT_Broker_IP;
uint16_t MQTT_Broker_Port = 0;
} My_Config;

//#define MDNS_DEBUG_ERR 1
//#define MDNS_DEBUG_TX 1
//#define MDNS_DEBUG_RX 1

/**
 * The MySensors Arduino library handles the wireless radio link and protocol
 * between your home built sensors/actuators and HA controller of choice. <...>
 *
 * Created by Henrik Ekblad <henrik.ekblad@mysensors.org>
 * Copyright (C) 2013-2015 Sensnology AB
 * Full contributor list: https://github.com/mysensors/Arduino/graphs/contributors
 *
 * Documentation: http://www.mysensors.org  |  Support Forum: http://forum.mysensors.org
 *
 * This program is free software; you can redistribute it and/or
 * modify it under the terms of the GNU General Public License
 * version 2 as published by the Free Software Foundation.
 *
 *******************************
 *
 * REVISION HISTORY
 * Version 1.0 - Henrik Ekblad
 * 
 * DESCRIPTION
 * The ESP8266 MQTT gateway sends radio network (or locally attached sensors) data to your MQTT broker.
 * The node also listens to MY_MQTT_TOPIC_PREFIX and sends out those messages to the radio network
 *
 * LED purposes:
 * - To use the feature, uncomment WITH_LEDS_BLINKING in MyConfig.h
 * - RX (green) - blink fast on radio message recieved. In inclusion mode will blink fast only on presentation recieved
 * - TX (yellow) - blink fast on radio message transmitted. In inclusion mode will blink slowly
 * - ERR (red) - fast blink on error during transmission error or recieve crc error  
 * 
 * See http://www.mysensors.org/build/esp8266_gateway for wiring instructions.
 *            
 * Not all ESP8266 modules have all pins available on their external interface.
 * This code has been tested on an ESP-12 module.
 * The ESP8266 requires a certain pin configuration to download code, and another one to run code:
 * - Connect REST (reset) via 10K pullup resistor to VCC, and via switch to GND ('reset switch')
 * - Connect GPIO15 via 10K pulldown resistor to GND
 * - Connect CH_PD via 10K resistor to VCC
 * - Connect GPIO2 via 10K resistor to VCC
 * - Connect GPIO0 via 10K resistor to VCC, and via switch to GND ('bootload switch')
 * 
 */

// Enable debug prints to serial monitor
#define MY_DEBUG
#define MY_DEBUG_VERBOSE
#define TRANSPORT_DEBUG

// for testing only - the controller should auto-allocate the nodeId
//#if defined(MY_NODE_ID)
//#undef MY_NODE_ID
//#define MY_NODE_ID 0
//#endif

// Use a bit lower baudrate for serial prints on ESP8266 than default in MyConfig.h
#define MY_BAUD_RATE 9600

// Enables and select radio type (if attached)
//#define MY_RADIO_NRF24
//#define MY_RADIO_RFM69

//#define MY_GATEWAY_FEATURE
#define MY_GATEWAY_ESP8266
#define MY_GATEWAY_MQTT_CLIENT

// Set this nodes subscripe and publish topic prefix
#define MY_MQTT_PUBLISH_TOPIC_PREFIX "root1/mys-gateway1-out"
#define MY_MQTT_SUBSCRIBE_TOPIC_PREFIX "root1/mys-gateway1-in"

// Set MQTT client id
#define MY_MQTT_CLIENT_ID  (String("mysensors-gw-esp_") + String(ESP.getChipId(), HEX)).c_str()
//#define MY_MQTT_CLIENT_ID "mysensors-gw1"

// Enable these if your MQTT broker requires usenrame/password
//#define MY_MQTT_USER "username"
//#define MY_MQTT_PASSWORD "password"

// Set WIFI SSID and password
#define MY_ESP8266_SSID (My_Config.WLAN_STA_SSID).c_str()
#define MY_ESP8266_PASSWORD (My_Config.WLAN_STA_PASSWD).c_str()

// Set the hostname for the WiFi Client. This is the hostname
// it will pass to the DHCP server if not static. 
//#define MY_ESP8266_HOSTNAME (String("ESP_") + String(ESP.getChipId(), HEX)).c_str()

// Enable MY_IP_ADDRESS here if you want a static ip address (no DHCP)
//#define MY_IP_ADDRESS (192, 168, 0, 10)

// If using static ip you need to define Gateway and Subnet address as well
//#define MY_IP_GATEWAY_ADDRESS (10, 0, 0, 100)
//#define MY_IP_SUBNET_ADDRESS (255, 255, 255, 0)

// MQTT-Broker IP Address
#define MY_CONTROLLER_URL_ADDRESS ((My_Config.MQTT_Broker_FQDN).c_str())
//#define MY_CONTROLLER_IP_ADDRESS (10, 0, 0, 1) // should be auto-configured using mDNS+DNS-SD !

// MQTT-Broker Port
#define MY_PORT (My_Config.MQTT_Broker_Port)


 /*
// Flash leds on rx/tx/err
#define MY_LEDS_BLINKING_FEATURE
// Set blinking period
#define MY_DEFAULT_LED_BLINK_PERIOD 300

// Enable inclusion mode
#define MY_INCLUSION_MODE_FEATURE
// Enable Inclusion mode button on gateway
#define MY_INCLUSION_BUTTON_FEATURE
// Set inclusion mode duration (in seconds)
#define MY_INCLUSION_MODE_DURATION 60 
// Digital pin used for inclusion mode button
#define MY_INCLUSION_MODE_BUTTON_PIN  3 

#define MY_DEFAULT_ERR_LED_PIN 16  // Error led pin
#define MY_DEFAULT_RX_LED_PIN  16  // Receive led pin
#define MY_DEFAULT_TX_LED_PIN  16  // the PCB, on board LED
*/

//#define MY_USE_UDP

// How many clients should be able to connect to this gateway (default 1)
#define MY_GATEWAY_MAX_CLIENTS 1

#if defined(MY_USE_UDP)
  #include <WiFiUdp.h>
#endif

#include <MySensors.h>
 
// ***

uint8_t MYS_childSensorId1 = 0;

#define MY_LED_PIN D1
#define MY_LED_FADE_DELAY 10  // Delay in ms for each percentage fade up/down (10ms = 1s full-range dim)

static uint8_t MY_LED_Light_Level = 0;  // Current dim. level

MyMessage MYS_dimmerMsg(MYS_childSensorId1, V_PERCENTAGE);
MyMessage MYS_statusMsg(MYS_childSensorId1, V_STATUS);


void setup_net()
{
//  sprintf(wifi_ap_ssid, "ESP-AP-%06X", ESP.getChipId());
  Serial.print("setup_net(): Hostname = ");
  Serial.println((String("ESP_") + String(ESP.getChipId(), HEX)).c_str());

  WiFi.hostname((String("ESP_") + String(ESP.getChipId(), HEX)).c_str());

  WiFi.begin((My_Config.WLAN_STA_SSID).c_str(), (My_Config.WLAN_STA_PASSWD).c_str());
  
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  
  Serial.println();
  Serial.print("setup_net(): Connected to ");
  Serial.print((My_Config.WLAN_STA_SSID).c_str());
  Serial.print(", IP address: ");
  Serial.println(WiFi.localIP());
}


void setup_mdns()
{
  if (!MDNS.begin((String("ESP_") + String(ESP.getChipId(), HEX)).c_str())) { Serial.println("setup_mdns(): Error Setting-Up mDNS Responder !"); }
  else { Serial.println("setup_mdns(): mDNS responder started"); }
}


void mdns_get_first_record_for_fqdn(String _FQDN, IPAddress &_IP, uint16_t &_Port)
{
  // trim the ".local" ending - for handing over to the mDNS Resolver
  _FQDN.replace(".local","");

  Serial.print("mdns_get_first_record_for_fqdn(): Non-FQDN Hostname = ");
  Serial.println(_FQDN.c_str());

  Serial.println("mdns_get_first_record_for_fqdn(): Sending mDNS query");
  
  // Send out query for mqtt tcp services
  uint8_t n = MDNS.queryService("mqtt", "tcp");

  Serial.println("mdns_get_first_record_for_fqdn(): mDNS query done");
  
  if (n == 0) {
    Serial.println("mdns_get_first_record_for_fqdn(): No Services Were Found");
  }
  else {
    Serial.print("mdns_get_first_record_for_fqdn(): ");
    Serial.print(n);
    Serial.println(" Service(s) Were Found :");

    for (uint8_t i = 0; i < n; ++i) {
      // Print details for each service found
      Serial.print(i); Serial.print(": ");
      Serial.print(MDNS.hostname(i)); Serial.print(" (");
      Serial.print(MDNS.IP(i)); Serial.print(":"); Serial.print(MDNS.port(i));
      Serial.println(")");

    if (_FQDN == MDNS.hostname(i)) {
        _IP = MDNS.IP(i);
        _Port = MDNS.port(i);
        Serial.print("mdns_get_first_record_for_fqdn(): mDNS 1st Matching Record# = ");
        Serial.println(i);
        break;
        // return the 1st matching record only
      }
     }
    }

}


// before() : Anything To Be Done Before mySensors Lib. Starts
void before() { 

MY_SERIALDEVICE.println("before() : Hello !");

// Here to be the reading of a json/cbor object from a file on a local storage (e.g. : SPIFFS over Serial NVRAM) & the parsing of it into the My_Config struct

My_Config.WLAN_STA_SSID = "My-WLAN";
My_Config.WLAN_STA_PASSWD = "My-Passwd";
My_Config.MQTT_Broker_FQDN = "hercules.local";

// a '.local' TLD -> to be resolved via mDNS
if ((My_Config.MQTT_Broker_FQDN).endsWith(".local"))
{
setup_net();
setup_mdns();
mdns_get_first_record_for_fqdn(My_Config.MQTT_Broker_FQDN, My_Config.MQTT_Broker_IP, My_Config.MQTT_Broker_Port);

Serial.print("before(): MQTT Broker IP:Port = ");
Serial.print(My_Config.MQTT_Broker_IP); Serial.print(":");
Serial.println(My_Config.MQTT_Broker_Port);

if (My_Config.MQTT_Broker_Port != 0) { My_Config.MQTT_Broker_FQDN = (My_Config.MQTT_Broker_IP).toString(); }

Serial.print("before(): My_Config.MQTT_Broker_FQDN = ");
Serial.println(My_Config.MQTT_Broker_FQDN);
}
// < mDNS Resolution

}


void setup() {

MY_SERIALDEVICE.println("setup() : Hello !");

request( MYS_childSensorId1, V_PERCENTAGE ); // request the formerly-set light-level from the controller

}


void presentation() {
  // Present locally attached sensors here    

    MY_SERIALDEVICE.println("presentation() : Hello !");

    present( MYS_childSensorId1, S_DIMMER );

    sendSketchInfo("Capstone_2016", "1.0");
}


void loop() {
  // Send locally attech sensors data here

    Serial.print("ESP.getFreeHeap() = ");
    Serial.println(ESP.getFreeHeap());

    sendHeartbeat();
  
//  delay(50); // a >100ms delay() will interfere with the WiFi Connectivity ! (& might also cause a system hang/reset)
    wait(15000); // Wait for a specified amount of time to pass. Keeps process()ing. This does not power-down the radio nor the Arduino. Because this calls process() in a loop, it is a good way to wait in your loop() on a node that listens to messages.
}


void receive(const MyMessage &message)
{
  if (message.type == V_STATUS) {
    Serial.print( "V_STATUS command received. Value is " ); Serial.println( message.data  );
    
    uint8_t lstate = atoi( message.data );
    Serial.print( "V_STATUS new state: " ); Serial.println( lstate );
    
    if ((lstate != 0) && (lstate != 1)) {
      Serial.println( "V_STATUS data invalid (should be 0/1)" );
      }
    }

if (message.type == V_PERCENTAGE) {
    Serial.print( "V_PERCENTAGE command received. Value is " ); Serial.println( message.data  );
    
    uint8_t dimvalue = atoi( message.data );
    
    Serial.print( "V_PERCENTAGE new state: " ); Serial.println( dimvalue );

    if ((dimvalue<0)||(dimvalue>100)) {
      Serial.println( "V_PERCENTAGE data Invalid (should be 0..100)" );
      }
    }
    
    if (message.type == V_STATUS || message.type == V_PERCENTAGE) {

        //  Retrieve the power or dim level from the incoming request message
        int requestedLevel = atoi( message.data );

        // Adjust incoming level if this is a V_STATUS variable update [0 == off, 1 == on]
        requestedLevel *= ( message.type == V_STATUS ? 35 : 1 );

        // Clip incoming level to valid range of 0 to 100
        requestedLevel = requestedLevel > 100 ? 100 : requestedLevel;
        requestedLevel = requestedLevel < 0   ? 0   : requestedLevel;

        Serial.print( "Changing level to " ); Serial.print( requestedLevel );
        Serial.print( ", from " ); Serial.println( MY_LED_Light_Level );

        fadeToLevel( requestedLevel );

        // Inform the gateway of the current DimmableLED's SwitchPower1 and LoadLevelStatus value...
        send(MYS_statusMsg.set(MY_LED_Light_Level > 0));

        // hek comment: Is this really nessesary?
        send( MYS_dimmerMsg.set(MY_LED_Light_Level) );
    }
}


/***
 *  This method provides a graceful fade up/down effect
 */
void fadeToLevel( int toLevel )
{
    int delta = (( toLevel - MY_LED_Light_Level ) < 0) ? -1 : 1;

    while ( MY_LED_Light_Level != toLevel ) {
        MY_LED_Light_Level += delta;
        analogWrite( MY_LED_PIN, (int)(MY_LED_Light_Level / 100. * 255) );
        delay( MY_LED_FADE_DELAY );
    }
}

Debugging Output - via Serial Port :

(~45 First Seconds)

0;255;3;0;9;MCO:BGN:INIT GW,CP=R-NGE--,VER=2.1.0-beta
0;255;3;0;9;MCO:BGN:BFR
before() : Hello !
setup_net(): Hostname = ESP_c04b5b
scandone
f 0, scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt 

connected with My-WLAN, channel 1
dhcp client start...
...ip:10.0.0.102,mask:255.255.255.0,gw:10.0.0.250
.
setup_net(): Connected to My-WLAN, IP address: 10.0.0.102
setup_mdns(): mDNS responder started
mdns_get_first_record_for_fqdn(): Non-FQDN Hostname = hercules
mdns_get_first_record_for_fqdn(): Sending mDNS query
mdns_get_first_record_for_fqdn(): mDNS query done
mdns_get_first_record_for_fqdn(): 1 Service(s) Were Found :
0: hercules (10.0.0.1:1883)
mdns_get_first_record_for_fqdn(): mDNS 1st Matching Record# = 0
before(): MQTT Broker IP:Port = 10.0.0.1:1883
before(): My_Config.MQTT_Broker_FQDN = 10.0.0.1
f r0, scandone
.IP: 10.0.0.102
0;255;3;0;9;MCO:BGN:STP
setup() : Hello !
0;255;3;0;9;MCO:BGN:INIT OK,TSP=NA
IP: 10.0.0.102
0;255;3;0;9;Attempting MQTT connection...
0;255;3;0;9;MQTT connected
presentation() : Hello !
ESP.getFreeHeap() = 39568
pm open,type:2 0
ESP.getFreeHeap() = 41168
ESP.getFreeHeap() = 41168

[non_avg_joe]

Update:

While Commit# [ https://github.com/mysensors/MySensors/commit/38013ce5acf7b5af23ce5e9dbc2920d38836a910 ] Seem To Have Fixed The Presentation Issue ( Thanks a Bunch ! :) ),
No Node_ID Is Ever Requested ( Is There a Way, Preferrably Reliable..., To Manually "Push" a Node_ID From The Controller ? ).

[tekka]

@non_avg_joe A gateway will always have ID=0, hence no ID request.
But there is something I need to adjust...hold on :)

PR submitted: https://github.com/mysensors/MySensors/pull/699

Thanks for reporting.

[Mickey]

@non_avg_joe
In the section where needed to populate the params from SPIFFS

// Here to be the reading of a json/cbor object from a file on a local storage (e.g. : SPIFFS over Serial NVRAM) & the parsing of it into the My_Config struct

My_Config.WLAN_STA_SSID = "My-WLAN";
My_Config.WLAN_STA_PASSWD = "My-Passwd";
My_Config.MQTT_Broker_FQDN = "hercules.local";

Did you ever got to implement this section?

[non_avg_joe]

@tekka: Thanks For All Your Efforts, Buddy ! :)

@Mickey:
Not Yet, Although I Plan On Utilizing [ https://github.com/tzapu/WiFiManager ] & a Similar Example Is Included There -
https://github.com/tzapu/WiFiManager/blob/master/examples/AutoConnectWithFSParameters/AutoConnectWithFSParameters.ino
+
https://github.com/tzapu/WiFiManager/blob/master/examples/OnDemandConfigPortal/OnDemandConfigPortal.ino