SSL support for MQTT on ESP8266



  • @hek @Anticimex First install https://github.com/esp8266/arduino-esp8266fs-plugin and upload the crt file to the ESP8266 file system.

    then issue these command from the folder in which MySensors is installed:
    sed -i 's/WiFiClient/WiFiClientSecure/g' Arduino/libraries/MySensors/core/MyGatewayTransportEthernet.cpp

    sed -i 's/WiFiClient/WiFiClientSecure/g' Arduino/libraries/MySensors/core/MyGatewayTransportMQTTClient.cpp

    change your GatewayESP8266MQTTClient to this:

    /**
     * The MySensors Arduino library handles the wireless radio link and protocol
     * between your home built sensors/actuators and HA controller of choice.
     * The sensors forms a self healing radio network with optional repeaters. Each
     * repeater and gateway builds a routing tables in EEPROM which keeps track of the
     * network topology allowing messages to be routed to nodes.
     *
     * Created by Henrik Ekblad <henrik.ekblad@mysensors.org>
     * Copyright (C) 2013-2015 Sensnology AB
     * Full contributor list: https://github.com/mysensors/Arduino/graphs/contributors
     *
     * Documentation: http://www.mysensors.org
     * Support Forum: http://forum.mysensors.org
     *
     * This program is free software; you can redistribute it and/or
     * modify it under the terms of the GNU General Public License
     * version 2 as published by the Free Software Foundation.
     *
     *******************************
     *
     * REVISION HISTORY
     * Version 1.0 - Henrik Ekblad
     * 
     * DESCRIPTION
     * The ESP8266 MQTT gateway sends radio network (or locally attached sensors) data to your MQTT broker.
     * The node also listens to MY_MQTT_TOPIC_PREFIX and sends out those messages to the radio network
     *
     * LED purposes:
     * - To use the feature, uncomment WITH_LEDS_BLINKING in MyConfig.h
     * - RX (green) - blink fast on radio message recieved. In inclusion mode will blink fast only on presentation recieved
     * - TX (yellow) - blink fast on radio message transmitted. In inclusion mode will blink slowly
     * - ERR (red) - fast blink on error during transmission error or recieve crc error  
     * 
     * See http://www.mysensors.org/build/esp8266_gateway for wiring instructions.
     * nRF24L01+  ESP8266
     * VCC        VCC
     * CE         GPIO4          
     * CSN/CS     GPIO15
     * SCK        GPIO14
     * MISO       GPIO12
     * MOSI       GPIO13
     *            
     * Not all ESP8266 modules have all pins available on their external interface.
     * This code has been tested on an ESP-12 module.
     * The ESP8266 requires a certain pin configuration to download code, and another one to run code:
     * - Connect REST (reset) via 10K pullup resistor to VCC, and via switch to GND ('reset switch')
     * - Connect GPIO15 via 10K pulldown resistor to GND
     * - Connect CH_PD via 10K resistor to VCC
     * - Connect GPIO2 via 10K resistor to VCC
     * - Connect GPIO0 via 10K resistor to VCC, and via switch to GND ('bootload switch')
     * 
      * Inclusion mode button:
     * - Connect GPIO5 via switch to GND ('inclusion switch')
     * 
     * Hardware SHA204 signing is currently not supported!
     *
     * Make sure to fill in your ssid and WiFi password below for ssid & pass.
     */
    #include "FS.h"
    #include <EEPROM.h>
    #include <SPI.h>
    
    // Enable debug prints to serial monitor
    #define MY_DEBUG 
    
    // Use a bit lower baudrate for serial prints on ESP8266 than default in MyConfig.h
    #define MY_BAUD_RATE 9600
    
    // Enables and select radio type (if attached)
    #define MY_RADIO_NRF24
    //#define MY_RADIO_RFM69
    
    #define MY_GATEWAY_MQTT_CLIENT
    #define MY_GATEWAY_ESP8266
    
    // Set this nodes subscripe and publish topic prefix
    #define MY_MQTT_PUBLISH_TOPIC_PREFIX "mygateway1-out"
    #define MY_MQTT_SUBSCRIBE_TOPIC_PREFIX "mygateway1-in"
    
    // Set MQTT client id
    #define MY_MQTT_CLIENT_ID "mysensors-1"
    
    // Enable these if your MQTT broker requires usenrame/password
    #define MY_MQTT_USER "admin" //your MQTT username
    #define MY_MQTT_PASSWORD "admin" //your MQTT password
    
    // Set WIFI SSID and password
    #define MY_ESP8266_SSID "yourwifissid"
    #define MY_ESP8266_PASSWORD "yourwifipassword"
    
    // Enable MY_IP_ADDRESS here if you want a static ip address (no DHCP)
    #define MY_IP_ADDRESS 172,16,0,120
    
    // If using static ip you need to define Gateway and Subnet address as well
    #define MY_IP_GATEWAY_ADDRESS 172,16,0,1
    #define MY_IP_SUBNET_ADDRESS 255,255,255,0
    
    
    // MQTT broker ip address.  
    #define MY_CONTROLLER_IP_ADDRESS 172, 16, 0, 95
    
    // The MQTT broker port to to open 
    #define MY_PORT 8883      // note this change its port 8883 as per standard for SSL over MQTT
    
     /*
    // Flash leds on rx/tx/err
    #define MY_LEDS_BLINKING_FEATURE
    // Set blinking period
    #define MY_DEFAULT_LED_BLINK_PERIOD 300
    
    // Enable inclusion mode
    #define MY_INCLUSION_MODE_FEATURE
    // Enable Inclusion mode button on gateway
    #define MY_INCLUSION_BUTTON_FEATURE
    // Set inclusion mode duration (in seconds)
    #define MY_INCLUSION_MODE_DURATION 60 
    // Digital pin used for inclusion mode button
    #define MY_INCLUSION_MODE_BUTTON_PIN  3 
    
    #define MY_DEFAULT_ERR_LED_PIN 16  // Error led pin
    #define MY_DEFAULT_RX_LED_PIN  16  // Receive led pin
    #define MY_DEFAULT_TX_LED_PIN  16  // the PCB, on board LED
    */
    
    #include <ESP8266WiFi.h>
    #include <MySensor.h>
    
    void setup() { 
      if (!SPIFFS.begin()) {
        Serial.println("Failed to mount file system");
        return;
      }
    
      File ca = SPIFFS.open("/ca.crt", "r"); //replace ca.crt with the ca file name
      if (!ca) {
        Serial.println("Failed to open ca file");
      }
      else
      Serial.println("Successfully opened ca file");
    
    if(_ethClient.loadCertificate(ca)) //_ethClient as defined in MyGatewayTransportMQTTClient.cpp
      Serial.println("loaded");
      else
      Serial.println("not loaded");
    }
    
    void presentation() {
      // Present locally attached sensors here    
    }
    
    
    void loop() {
      // Send locally attech sensors data here
    }
    
    
    
    

    NB: i used https://github.com/owntracks/tools/raw/master/TLS/generate-CA.sh for generating the ca files


  • Contest Winner

    I have no esp hw. And like I said, ssl and MQTT is specific to a certain architecture (from a gw and controller perspective) so I don't want it being mixed in with the existing security infrastructure that goes on every node.
    You are free to implement it as such of course, but I won't bring it into the signing solution.


  • Admin

    @Anticimex
    This has nothing to do with the message signing.

    @noelgeorgi
    Good!. Might be good to put this initialization in the optional before()-method instead of setup(), which runs before any other MySensors stuff.



  • @hek I didnt know there was an optional method. Thanks


  • Contest Winner

    @hek i know that perfectly well, but since I am adressed, I just want to make clear I do not consider myself involved 🙂



  • @hek pubsubclient has issues with ssl connection, frequent disconnections. Adafruit's MQTT library works like charm, trying to integrate into MySensors



  • @hek I somewhat succesfully integrated Adafruit MQTT library into MySensors, i am able to sent out data but not able to receive (subscribe). These are the changes i made:

    diff output

    • MySesnors.h
    < #if defined(MY_GATEWAY_SERIAL) || defined(MY_GATEWAY_W5100) || defined(MY_GATEWAY_ENC28J60) || defined(MY_GATEWAY_ESP8266) || defined(MY_GATEWAY_MQTT_CLIENT) || defined(MY_GATEWAY_MQTT_SECURE_CLIENT)
    ---
    > #if defined(MY_GATEWAY_SERIAL) || defined(MY_GATEWAY_W5100) || defined(MY_GATEWAY_ENC28J60) || defined(MY_GATEWAY_ESP8266) || defined(MY_GATEWAY_MQTT_CLIENT)
    55,59d54
    < #if defined(MY_GATEWAY_MQTT_SECURE_CLIENT)
    < 	#include "drivers/Adafruit_MQTT_Library/Adafruit_MQTT.cpp"
    < 	#include "drivers/Adafruit_MQTT_Library/Adafruit_MQTT_Client.cpp"
    < #endif
    < 
    180c175
    < #if defined(MY_GATEWAY_MQTT_CLIENT) || defined(MY_GATEWAY_MQTT_SECURE_CLIENT)
    ---
    > #if defined(MY_GATEWAY_MQTT_CLIENT)
    
    • keywords.txt
    < MY_GATEWAY_MQTT_SECURE_CLIENT LITERAL 1
    

    MyGatewayTransportMQTTClient.cpp

    36,40c36
    < 	#if defined(MY_GATEWAY_MQTT_SECURE_CLIENT)
    < 		#define EthernetClient WiFiClientSecure
    < 	#else 
    < 		#define EthernetClient WiFiClient
    < 	#endif
    ---
    > 	#define EthernetClient WiFiClientSecure
    54,277d49
    < #if defined(MY_GATEWAY_MQTT_SECURE_CLIENT)
    < 	Adafruit_MQTT_Client mqtt(&_ethClient
    < 				 #if defined(MY_CONTROLLER_IP_ADDRESS)
    <  				, _brokerIp 
    < 				#else
    < 				, MY_CONTROLLER_URL_ADDRESS
    < 				#endif				
    < 				, MY_PORT 
    < 				#if defined(MY_MQTT_USER) && defined(MY_MQTT_PASSWORD) 
    < 				, MY_MQTT_USER, MY_MQTT_PASSWORD
    < 				#endif
    < 				);
    < 
    < Adafruit_MQTT_Subscribe feed = Adafruit_MQTT_Subscribe(&mqtt,MY_MQTT_SUBSCRIBE_TOPIC_PREFIX "/+/+/+/+/+" );
    < Adafruit_MQTT_Subscribe *subscription;
    < bool _connecting = true;
    < bool _available = false;
    < char _convBuffer[MAX_PAYLOAD*2+1];
    < char _fmtBuffer[MY_GATEWAY_MAX_SEND_LENGTH];
    < MyMessage _mqttMsg;
    < 
    < bool gatewayTransportSend(MyMessage &message) {
    < 	if (!mqtt.connected())
    < 		return false;
    < 	snprintf_P(_fmtBuffer, MY_GATEWAY_MAX_SEND_LENGTH, PSTR(MY_MQTT_PUBLISH_TOPIC_PREFIX "/%d/%d/%d/%d/%d"), message.sender, message.sensor, mGetCommand(message), mGetAck(message), message.type);
    < 	debug(PSTR("Sending message on topic: %s\n"), _fmtBuffer);
    < 	return mqtt.publish(_fmtBuffer, message.getString(_convBuffer));
    < }
    < 
    < void incomingMQTT(char* topic, byte* payload,
    <                         unsigned int length)
    < {
    < 	debug(PSTR("Message arrived on topic: %s\n"), topic);
    < 	char *str, *p;
    < 	uint8_t i = 0;
    < 	uint8_t bvalue[MAX_PAYLOAD];
    < 	uint8_t blen = 0;
    < 	uint8_t command = 0;
    < 	for (str = strtok_r(topic, "/", &p); str && i <= 5;
    < 			str = strtok_r(NULL, "/", &p)) {
    < 		switch (i) {
    < 			case 0: {
    < 				// Topic prefix
    < 				if (strcmp(str, MY_MQTT_SUBSCRIBE_TOPIC_PREFIX) != 0) {
    < 					// Message not for us or malformed!
    < 					return;
    < 				}
    < 				break;
    < 			}
    < 			case 1: {
    < 				// Node id
    < 				_mqttMsg.destination = atoi(str);
    < 				break;
    < 			}
    < 			case 2: {
    < 				// Sensor id
    < 				_mqttMsg.sensor = atoi(str);
    < 				break;
    < 			}
    < 			case 3: {
    < 				// Command type
    < 				command = atoi(str);
    < 				mSetCommand(_mqttMsg, command);
    < 				break;
    < 			}
    < 			case 4: {
    < 				// Ack flag
    < 				mSetRequestAck(_mqttMsg, atoi(str)?1:0);
    < 				break;
    < 			}
    < 			case 5: {
    < 				// Sub type
    < 				_mqttMsg.type = atoi(str);
    < 				// Add payload
    < 				if (command == C_STREAM) {
    < 					blen = 0;
    < 					uint8_t val;
    < 					while (*payload) {
    < 						val = protocolH2i(*payload++) << 4;
    < 						val += protocolH2i(*payload++);
    < 						bvalue[blen] = val;
    < 						blen++;
    < 					}
    < 					_mqttMsg.set(bvalue, blen);
    < 				} else {
    < 					char* ca;
    < 					ca = (char *) payload;
    < 					ca += length;
    < 					*ca = '\0';
    < 					_mqttMsg.set((const char*) payload);
    < 				}
    < 				_available = true;
    < 			}
    < 		}
    < 		i++;
    < 	}
    < }
    < 
    < bool reconnectMQTT() {
    < 	debug(PSTR("Attempting MQTT connection...\n"));
    < 	// Attempt to connect
    < 		
    < int8_t ret;
    < 
    <   // Stop if already connected.
    <   if (mqtt.connected()) {
    <     return true;
    <   }
    < 
    <   //Serial.print("Connecting to MQTT... ");
    < 
    <   uint8_t retries = 3;
    <   while ((ret = mqtt.connect()) != 0) { // connect will return 0 for connected
    <        Serial.println(mqtt.connectErrorString(ret));
    <        Serial.println("Retrying MQTT connection in 5 seconds...");
    <        mqtt.disconnect();
    <        delay(5000);  // wait 5 seconds
    <        retries--;
    <        if (retries == 0) {
    <          // basically die and wait for WDT to reset me
    <          while (1);
    <        }
    <   }
    <   //Serial.println("MQTT Connected!");
    <  
    < 
    < 		 {
    < 		debug(PSTR("MQTT connected\n"));
    < 		// Once connected, publish an announcement...
    < 		//_client.publish("outTopic","hello world");
    < 		// ... and resubscribe
    < 		mqtt.subscribe(&feed);
    < 		return true;
    < 	}
    < 	return false;
    < }
    < 
    < bool gatewayTransportInit() {
    < 	_connecting = true;
    < 	//#if defined(MY_CONTROLLER_IP_ADDRESS)
    < 	//	_client.setServer(_brokerIp, MY_PORT);
    < 	//#else
    < 	//	_client.setServer(MY_CONTROLLER_URL_ADDRESS, MY_PORT);
    < 	//#endif
    < 
    < 	//_client.setCallback(incomingMQTT);
    < 
    <  if((subscription = mqtt.readSubscription(10))) {
    <     if (subscription == &feed) 
    <     {
    <       Serial.print(F("Got: "));
    <       Serial.println((char *)feed.lastread);
    <     }}
    < 
    < 
    <   	#if defined(MY_GATEWAY_ESP8266)
    < 		// Turn off access point
    < 		WiFi.mode (WIFI_STA);
    < 		(void)WiFi.begin(MY_ESP8266_SSID, MY_ESP8266_PASSWORD);
    < 		#ifdef MY_IP_ADDRESS
    < 			WiFi.config(_clientIp, _gatewayIp, _subnetIp);
    < 		#endif
    < 		while (WiFi.status() != WL_CONNECTED)
    < 		{
    < 			delay(500);
    < 			MY_SERIALDEVICE.print(".");
    < 			yield();
    < 		}
    < 		MY_SERIALDEVICE.print("IP: ");
    < 		MY_SERIALDEVICE.println(WiFi.localIP());
    < 	#else
    < 		#ifdef MY_IP_ADDRESS
    < 			Ethernet.begin(_clientMAC, _clientIp);
    < 		#else
    < 			// Get IP address from DHCP
    < 			if (!Ethernet.begin(_clientMAC))
    < 			{
    < 				MY_SERIALDEVICE.print("DHCP FAILURE...");
    < 				_connecting = false;
    < 				return false;
    < 			}
    < 			MY_SERIALDEVICE.print("IP: ");
    < 			MY_SERIALDEVICE.println(Ethernet.localIP());
    < 		#endif
    < 
    < 		// give the Ethernet interface a second to initialize
    < 		// TODO: use HW delay
    < 		wait(1000);
    < 	#endif
    < 	_connecting = false;
    < 	return true;
    < }
    < 
    < bool gatewayTransportAvailable() {
    < 	if (_connecting)
    < 		return false;
    < 
    < 	//keep lease on dhcp address
    < 	//Ethernet.maintain();
    < 	if (!_ethClient.connected()) {
    < 		//reinitialise client
    < 		if (gatewayTransportInit())
    < 			reconnectMQTT();
    < 		return false;
    < 	}
    < 	//_ethClient.loop();
    < 
    < 
    < 
    < 	return _available;
    < }
    < 
    < MyMessage & gatewayTransportReceive() {
    < 	// Return the last parsed message
    < 	_available = false;
    < 	return _mqttMsg;
    < }
    < 
    < 
    < 
    < 
    < 
    < 
    < #else
    279d50
    < 
    456c227
    < #endif
    ---
    > 
    
    
    • GatewWay Code
    /**
     * The MySensors Arduino library handles the wireless radio link and protocol
     * between your home built sensors/actuators and HA controller of choice.
     * The sensors forms a self healing radio network with optional repeaters. Each
     * repeater and gateway builds a routing tables in EEPROM which keeps track of the
     * network topology allowing messages to be routed to nodes.
     *
     * Created by Henrik Ekblad <henrik.ekblad@mysensors.org>
     * Copyright (C) 2013-2015 Sensnology AB
     * Full contributor list: https://github.com/mysensors/Arduino/graphs/contributors
     *
     * Documentation: http://www.mysensors.org
     * Support Forum: http://forum.mysensors.org
     *
     * This program is free software; you can redistribute it and/or
     * modify it under the terms of the GNU General Public License
     * version 2 as published by the Free Software Foundation.
     *
     *******************************
     *
     * REVISION HISTORY
     * Version 1.0 - Henrik Ekblad
     * 
     * DESCRIPTION
     * The ESP8266 MQTT gateway sends radio network (or locally attached sensors) data to your MQTT broker.
     * The node also listens to MY_MQTT_TOPIC_PREFIX and sends out those messages to the radio network
     *
     * LED purposes:
     * - To use the feature, uncomment WITH_LEDS_BLINKING in MyConfig.h
     * - RX (green) - blink fast on radio message recieved. In inclusion mode will blink fast only on presentation recieved
     * - TX (yellow) - blink fast on radio message transmitted. In inclusion mode will blink slowly
     * - ERR (red) - fast blink on error during transmission error or recieve crc error  
     * 
     * See http://www.mysensors.org/build/esp8266_gateway for wiring instructions.
     * nRF24L01+  ESP8266
     * VCC        VCC
     * CE         GPIO4          
     * CSN/CS     GPIO15
     * SCK        GPIO14
     * MISO       GPIO12
     * MOSI       GPIO13
     *            
     * Not all ESP8266 modules have all pins available on their external interface.
     * This code has been tested on an ESP-12 module.
     * The ESP8266 requires a certain pin configuration to download code, and another one to run code:
     * - Connect REST (reset) via 10K pullup resistor to VCC, and via switch to GND ('reset switch')
     * - Connect GPIO15 via 10K pulldown resistor to GND
     * - Connect CH_PD via 10K resistor to VCC
     * - Connect GPIO2 via 10K resistor to VCC
     * - Connect GPIO0 via 10K resistor to VCC, and via switch to GND ('bootload switch')
     * 
      * Inclusion mode button:
     * - Connect GPIO5 via switch to GND ('inclusion switch')
     * 
     * Hardware SHA204 signing is currently not supported!
     *
     * Make sure to fill in your ssid and WiFi password below for ssid & pass.
     */
    
    #include <EEPROM.h>
    #include <SPI.h>
    
    // Enable debug prints to serial monitor
    #define MY_DEBUG 
    
    // Use a bit lower baudrate for serial prints on ESP8266 than default in MyConfig.h
    //#define MY_BAUD_RATE 9600
    
    // Enables and select radio type (if attached)
    #define MY_RADIO_NRF24
    //#define MY_RADIO_RFM69
    
    #define MY_GATEWAY_MQTT_SECURE_CLIENT
    #define MY_GATEWAY_ESP8266
    
    // Set this nodes subscripe and publish topic prefix
    #define MY_MQTT_PUBLISH_TOPIC_PREFIX "mygateway1-out"
    #define MY_MQTT_SUBSCRIBE_TOPIC_PREFIX "mygateway1-in"
    
    // Set MQTT client id
    #define MY_MQTT_CLIENT_ID "mysensors-1"
    
    // Enable these if your MQTT broker requires usenrame/password
    #define MY_MQTT_USER "user"
    #define MY_MQTT_PASSWORD "pass"
    
    // Set WIFI SSID and password
    #define MY_ESP8266_SSID "ssid"
    #define MY_ESP8266_PASSWORD "password"
    
    // Enable MY_IP_ADDRESS here if you want a static ip address (no DHCP)
    #define MY_IP_ADDRESS 172,16,0,120
    
    // If using static ip you need to define Gateway and Subnet address as well
    #define MY_IP_GATEWAY_ADDRESS 172,16,0,1
    #define MY_IP_SUBNET_ADDRESS 255,255,255,0
    
    
    // MQTT broker ip address.  
    #define MY_CONTROLLER_URL_ADDRESS "domain.com"
    
    // The MQTT broker port to to open 
    #define MY_PORT 8883      
    
     /*
    // Flash leds on rx/tx/err
    #define MY_LEDS_BLINKING_FEATURE
    // Set blinking period
    #define MY_DEFAULT_LED_BLINK_PERIOD 300
    
    // Enable inclusion mode
    #define MY_INCLUSION_MODE_FEATURE
    // Enable Inclusion mode button on gateway
    #define MY_INCLUSION_BUTTON_FEATURE
    // Set inclusion mode duration (in seconds)
    #define MY_INCLUSION_MODE_DURATION 60 
    // Digital pin used for inclusion mode button
    #define MY_INCLUSION_MODE_BUTTON_PIN  3 
    
    #define MY_DEFAULT_ERR_LED_PIN 16  // Error led pin
    #define MY_DEFAULT_RX_LED_PIN  16  // Receive led pin
    #define MY_DEFAULT_TX_LED_PIN  16  // the PCB, on board LED
    */
    
    #include <ESP8266WiFi.h>
    #include <MySensor.h>
    
    void setup() { 
    }
    
    void presentation() {
      // Present locally attached sensors here    
    }
    
    
    void loop() {
      // Send locally attech sensors data here
    }
    
    
    
    

    Any help would be really appreciated???


  • Admin

    Isn't incomingMQTT() ever called?

    You seem to have commented away this line:
    //_client.setCallback(incomingMQTT);



  • @hek i am trying to do that in void loop{} will update once it gets working properly and will do a pull request, the problem is that i can subscribe to a main topic but not its subfeeds



  • @hek Adafruit's MQTT libarary doesnt have wildcard subscriptions switching to https://github.com/256dpi/arduino-mqtt SSL seems to be working and library has callback support



  • @noelgeorgi so do you have a working code already? It would be great to try it out.


  • Hardware Contributor

    Also looking at this... was this ever implemented? And where your trials on server encryption or also in client authentication?

    Thank You




Log in to reply
 

Suggested Topics

  • 1
  • 3
  • 10
  • 2
  • 9
  • 59
  • 6
  • 183

207
Online

9.6k
Users

10.2k
Topics

106.2k
Posts